Subject: [clisp-list] Ensuring call of cleanup ("destructor") routine for "C" object

From: "John K. Hinsdale" <[email-masked]>
Date: Wed Jun 26 12:30:05 2002

Thought I'd give an update on my adventures integrating Oracle access
w/ CLISP.  I'm almost there.  I ended up ditching the approach (used
by UnCommon SQL and others) of importing the entire Oracle client
library (OCI) with all its 200 or so functions and instead opted for a
much simpler interface with a few routines written in "C".  It's
pretty fast as well.  I've "sealed off" the Oracle bloat completely
from the Lisp side, which sees a single "C" structure and just a few
functions seen by CLISP.  This is what 99.9% of users will need.

But now I'm having a bit of a problem integrating it into CLISP via
the foreign function interface (FFI).  Specificaly it is not clear to
me how I can guarantee that once I create my "database object" (i.e.,
the "C" structure above) I can be guaranteed it eventaully gets
cleaned up via *my* cleanup routine - which releases a whole bunch of
state and resources, e.g., database connections, query cursors alloc'd
buffers, etc.)

The FFI seems to have a built-in way for cleanup of malloc/free
storage, however I need more than that.  I have to call specific
Oracle functions to release resources when the database object is
garbage collected.  I may also have to roll back or commit a database
transaction.

Using the "regexp" wrapper as a hint, I made a toy test FFI module
that just store a string as its state and prints it when it's accessed
- see below.  However, I can't seem to get the cleanup ("destroy")
function to get called.  I tried wrapping my initialization
("constructor") routine with a call to "ext:finalize" asking it to run
my cleanup when the object is GC'd but that did not seem to do it.

Is there a way to tell CLISP:

	(1) when it is GC'ing my database object to call my cleanup

and	(2) to immediately GC my database object?  This is because
	    I don't want my Lisp Oracle client leaving database
	    connections lying around - they consume a lot of
	    resources on the Oracle server side.  I need my objects
	    to get expunged immediately after they cease to be
	    visible to the program.

Any ideas?

---
John Hinsdale, Alma Mater Software, Inc., Tarrytown, NY 10591-3710 USA
[email-masked] | http://www.alma.com/staff/hin | +1 914 631 4690

; -------------------------------------------------------------------------
; Test

(defpackage "TEST"
  (:documentation
   "Testing the FFI")
  (:use "LISP" "FFI")
  (:export "NEWFN" "OPFN" "DESFN" ))

(in-package "TEST")

; Alloc, init and return a pointer to new object
(def-c-call-out newfn (:arguments (s c-string))
                      (:return-type c-pointer))

; Destroy an object
(def-c-call-out desfn (:arguments (ptr c-pointer))
                      (:return-type int))

; Operate on an object
(def-c-call-out opfn (:arguments (ptr c-pointer) (s c-string))
		     (:return-type int))

; Make an object, ensuring cleanup (DOES NOT WORK???)
(defun make-obj (s)
       (let (obj)
            (ext:finalize obj #'desfn)
	    (setq obj (newfn s))))

; -------------------------------------------------------------------------

---

From: "Thomas F. Burdick" <[email-masked]>
Date: Wed Jun 26 12:41:02 2002

> 	    to get expunged immediately after they cease to be

You don't want to leave this up to the GC.  Have a function accessible
from Lisp that destroys these objects, and the resources they consume.
GCs are good for managing resources that aren't tight, like memory.
For most uses of database handles, you only need the object to have
dynamic extent, in which case, a WITH-... style macro works wonderfully:

  (defmacro with-database-handle ((var &rest args) &body body)
    (let ((object (gensym)))
      `(let ((,object (make-obj ,@args)))
         (unwind-protect
             (let ((,var ,object)) ,@body)
           (destroy-object ,object)))))

If the object's lifetime is more complicated than this, you just need
to keep track of it, the GC is not an appropriate tool for cleaning it
up.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'

---

From: Kaz Kylheku <[email-masked]>
Date: Wed Jun 26 12:51:02 2002

On Wed, 26 Jun 2002, John K. Hinsdale wrote:

> Date: Wed, 26 Jun 2002 15:29:57 -0400
> From: John K. Hinsdale <[email-masked]>
> To: [email-masked]
> Subject: [clisp-list] Ensuring call of cleanup ("destructor") routine for
>     "C" object
> 
> 

Common Lisp has no finalization routine feature (though some
implementations do, according to a recent comp.lang.lisp
discussion). 

But really, releasing resources such as open files, sockets or database
connections at garbage collection time is probably too late! Such things
should be explicitly released.

One model to look at are the WITH-* standard macros in Common Lisp.
For example WITH-OPEN-FILE associates a new stream with a file,
evaluates some forms and then esures that the stream is closed.

You could have open and close type routines in your FFI, and then
write some WITH-* style macros to ensure that closes are matched
with opens.

These macros are typically based on UNWIND-PROTECT.

In CLISP, here is what an expansion of WITH-OPEN-FILE looks like:

[1]> (macroexpand '(with-open-file (stream args) form))

(LET ((STREAM (OPEN ARGS))) (DECLARE (SYSTEM::READ-ONLY STREAM))
 (UNWIND-PROTECT
  (MULTIPLE-VALUE-PROG1 (PROGN FORM) (WHEN STREAM (CLOSE STREAM)))
  (WHEN STREAM (CLOSE STREAM :ABORT T)))) ;

So you see, the stream is closed as soon as the WITH-OPEN-FILE
form finishes evaluating, but the stream object persists until
it is garbage collected.

Regarding rollback and commit, you could probably hack up some
macros that provide a convenient notation for doing it.

---

From: "John K. Hinsdale" <[email-masked]>
Date: Fri Jun 28 06:10:02 2002

> "Thomas F. Burdick" <[email-masked]> writes:

 > Kaz Kylheku <[email-masked]> writes:

Gents, thanks for the hints.  The with-... macro is what I want.  I
like Kaz's idea of using it to deal w/ the commit/abort aspect of my
modules as well.  [Update: I'm doing the part of my Oracle lib.  that
involves gluing it to CLISP - woo-hoo, I can get to my date.]

BTW, I later noticed Dan Weinreb answered my same question back in
1991:

  http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&selm=1991May20.055952.4991%40odi.com

For general Lisp stuff I'll do more Googling first so as not to ask
11-year old questions.  :)

---