Subject: [clisp-list] design question: foreign-variable objects vs symbol macros

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Thu Jan 23 03:06:03 2003

Hi,

So far, CLISP managed more or less well to hide the FOREIGN-VARIABLE objects that the FFI uses for operations on foreign memory (e.g. CAST, DEREF, ELEMENT etc.).

I'm wondering whether it wouldn't be as easy to let programmers operate on FOREIGN-VARIABLE objects, rename FFI::%CAST to FFI:CAST* etc., document them and avoid
a) the below mentioned risk of dererencing when not intended and
b) half a dozen WITH- macrolet introducers.

So far, the user seemingly only manipulated variables (symbol macros defined via MACROLET), introduced via DEF-C-VAR or WITH-C-VAR.

This is fine with lexical scoping:
  (ffi:with-c-var (dest `(c-array uint8 ,destlen)) ; no init
    (multiple-value-bind (status actual)
        (zlib-compress-string (ffi:c-var-address dest) destlen source sourcelen)
      ...)

I nevertheless find it potentially dangerous, because when you pass the variable as an argument to any function, you instead dereference the whole foreign object.
(zlib-compress-string dest ...) ; WRONG
(zlib-compress-string (ffi:c-var-address dest) ...) ; RIGHT

The first form
a) creates a Lisp array of :element-type '(unsigned-byte 8) with :initial-contents taken from the address dest points to,
b) allocates again a foreign array on the stack,
c) fills it from the Lisp array and
d) passes that to the function (if type declarations would fit, which they do not in this particular case).

Maybe this shows why Bruno Haible doesn't consider the FFI - he designed himself back in 1995 - as a good match for todays objectoriented programming.
My feeling: the CLISP FFI can be made to match OO use. Otherwise I wouldn't have invested that much effort in it.

MIHO, with OO and/or foreign libraries, you pretty much work with references to foreign objects (that is what FOREIGN-VARIABLE objects are), instead of constantly converting foreign structures to similar looking Lisp structures. I need references to foreign objects. I don't want the X/MS/GDI window structure converted to a Lisp structure!
The latter is IMHO even dangerous, because EQ-ness is lost: EQUALP is not EQ!

Now consider
FOREIGN/SIMPLE-CALLOC type &key count read-only

This function will return a FOREIGN-VARIABLE object. So the result is currently unusable with any other documented FFI function, as these do not operate on that type of object (but on symbol macros instead).

So I must define yet another symbol-macro defining macro
WITH-FOREIGN-ADDRESS (var addresss &optional type) &body

This can be understood similar to CLOS' WITH-SLOTS: it makes the slots visible/accessible.

(setq memory (FOREIGN-CALLOC 'uint8 :count 50))

(with-foreign-address (buffer memory)
  (list buffer (element buffer 0))) ; return whole array (as Lisp array) and the first element (as fixnum)

So I can still mostly hide FOREIGN-VARIABLE objects and provide an interface still based on CAST, DEREF, ELEMENT that FFI users have known for long.

But I'm getting tired of all these macro wrappers: WITH-FOREIGN-OBJECT, WITH-C-VAR, WITH-FOREIGN-ADDRESS, WITH-FOREIGN-STRING and more to come.

And I still dislike the risk of automatic conversion with those symbol macros:
(with-foreign-address (buffer memory)
  (unix:unix-read fd buffer 50) ; WRONG again, need (c-var-address buffer)

Instead, I could provide a patch to FOREIGN-CALL-OUT to accept FOREIGN-VARIABLE objects in addition to FOREIGN-ADDRESS where C-POINTER type declarations have been specified. But that is just part of the iceberg.
  (unix:unix-read fd memory 50) ; WITH-SLOTS and buffer variable made superfluous

Thanks for your comments,
	Jörg Höhle.