Subject: [clisp-list] :malloc-free allocation -- here: instance methods

From: Aurelio Bignoli <[email-masked]>
Date: Sun Apr 20 11:54:09 2003

Hoehle, Joerg-Cyril writes:

 > I still believe modules is a way to go to interface to foreign
 > stuff. That implies C code, be it automatically generated or
 > not. This C code should then contain the offsets and other
 > architecture/platform dependent forms. Let the .fas file and Lisp
 > code remain free of that.

good point! I'll follow this approach.

 > >returned function prototype could be a problem: every DB fuction has
 > >its own parameter list. Probably the C factory function should return
 > I don't understand this.

well, different DB functions have different argument lists. For example:

int DB->get(DB *db, DB_TXN *txnid, DBT *key, DBT *data, u_int32_t flags);

int DB->open(DB *db, DB_TXN *txnid, const char *file,const char *database, 
             DBTYPE type, u_int32_t flags, int mode);

In C the factory function could probably be declared as

typedef int (*db_function) (/* we can't specify argument list! */);
db_function ff_factory (DB *dbp, int function_type)
{
 [...]
}

It would be better to specify arguments in the db_function type
definition, but the C compiler can work without them. How should such
a foreign function be defined in CLISP? What return type should we use?

(def-call-out ff-factory
  (:name "ff_factory")
  (:arguments (db (c-ptr DB :in)) (function-type int :in))
  (:return-type ???))

---

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Thu Apr 24 01:25:04 2003

Aurelio Bignoli wrote:
>well, different DB functions have different argument lists. 

> How should such

One way would be to use C-POINTER just to get the address, and then construct foreign-function objects with this address.

Another would be the following (unportable, not recommended): return a pointer to the offset in the DB structure where the functions lay (or duplicate the whole structure in Lisp), and use
  (:arguments
   (db-ffs                              :out
     (c-struct list
      (get (c-function (:arguments (db c-pointer) ...) ...))
      (put (c-function ...))))
The FFI would then construct the foreign-function objects.

I suggest the following. I recently said that this DB->get is like accessing foreign functions off a dynamic library base pointer.

You need the following function signatures:
DB * function-name -> foreign-address
foreign-address * foreign-function-type -> foreign-function-object

The former you have to write, the latter can be used as FOREIGN-ADDRESS-FUNCTION, which was both in my ffi hacks toolbox, in my dynload patch, both sent to this list, but not yet in CVS CLISP.
You could also merge these two functions into one. That's what e.g. FFI::lookup-foreign-function does in foreign.d for the functions found in a module:
DB * function-name * ff-type -> ff-object

The former allows more work to be done at Lisp-level, which you may appreciate. The difference is really minor.

Here's the first function
enum {get, put, ...}
// I hereby choose to have Lisp pass known integers as representing the different functions.
// An alternative would be to have the C code dispatch on symbols such as GET, PUT etc. utils/MODPREP helps writing modules that access Lisp objects.
// You choose.
void* bdb_entry_point(DB* db, enum) {
  switch (enum) {
    case get: return &(DB->get);
    case put: return &(DB->put);
}

Here's some matching Lisp
(defun db-factory (DB name)
  (declare (type foreign-address DB)) ; maybe use (foreign-pointer type)?
  (declare-or-assert symbol name)
  (ffi:foreign-address-function
    ;; address
    (bdb-entry-point (name-to-bdb-index name))
    ;; c-type
    (ecase name
      (GET
       (load-time-value
        (ffi::parse-c-function
         '(c-function
            (:arguments (db c-pointer) (db-txn c-pointer) #)
            (:return-type ...) (:language :stdc))))))
    ;; name for printing #<FOREIGN-FUNCTION ...> (informative)
    name))

Memoization is omited here. Please use VALIDP to detect whether an old foreign-function-object is still valid or comes from an old Lisp image or whether the library has been closed and you need to regenerate the address.

Another issue not yet covered: It would be good if all these foreign-function s would share their pointer base with the corresponding DB base pointer. Then, (setf (validp DB) nil) (after closing it of course) would invalidate all functions in one statement.

There are two ways to achieve this
1. write the above bdb_entry_point() as a LISPFUNN() instead of going through the FFI to call it and have it return the appropriate FOREIGN-ADDRESS object.
2. or call bdb_entry_point via FFI, but then use the (SET-FOREIGN-BASE) I recently talked about to achieve this sharing. Sam implemented my proposal in CVS as (SETF (FOREIGN-POINTER) #) but I have yet to check what he did.
(let ((address (bdb-entry-point (name-to-bdb-index name))))
  (setf (foreign-base address) db)
  address)
;possibly a one-liner: (setf (foreign-base (bdb-entry-point #) db)
which still yields the new address, not db.

I'd go for 1. - I'm a fan of modules, and one has to write a little C code (bdb_entry_point) anyway. Then it's just easy to pack it as a LISPFUN (use allocate_faddress()).
And if it's a LISPFUN, you can just as well have it return the appropriate FOREIGN-FUNCTION object with little more (double?) code (i.e. case where the two functions mentioned above are merged into one).

Oh well, maybe I should provide more sample code?

Hope this helps,
	Jorg Hohle.