Subject: [clisp-list] design question: internal and external representation

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Thu Jan 23 03:00:05 2003

Hi,

for several years, CLISP's FFI has done quite a good job at hiding the internal representation of a foreign function or variable type specification.

External: (c-function (:name "name_in_C") (:arguments (foo c-pointer :out :none) ...) (:return-type ffi:int))

Internal: #<some vector>

Witness, FFI::parse-c-type and its converse ffi::deparse-c-type are not even documented or exported from the FFI package.

I believe this works well as long as most use of the FFI is static, like in static (and top-level) declarations à la DEF-CALL-OUT.

I get headaches when it comes to dynamic stuff. Nevertheless, so far, I managed to provide a couple of macros (even more to come) to work around this and still hide the internals: e.g. WITH-C-VAR

But this comes at cost to implementors: Many, if not all functions must be duplicated, and calls to PARSE-C-TYPE inserted in between.
That's part of why there are macros, which do this, and Lisp function in foreign1.lisp as pure wrappers to built-in function in foreign.d.

I want to provide a dynamic foreign-function constructor (cf. alien-funcall etc. in CMUCL). It will be similar to:
(defun FOREIGN-ADDRESS-FUNCTION (address typespec &optional C-name)
  (ffi::%FOREIGN-ADDRESS-FUNCTION address (ffi::parse-c-type typespec) C-name))
Use e.g. in:
(funcall (foreign-address-function (foreign-address unix:unix-read) `(c-function (:arguments ((buffer (c-array uint16 ,len))) ...)))
 fd #(#xABDD #x1234 ...) ,len)
; contrieved example, don't do variable arrays this way (too much overhead)

I'm somewhat tired of these wrapping exercises.

As I want to avoid the overhead of PARSE-C-TYPE at run-time as much as possible, and most type declarations are static like (quote (c-function ...)) anyway, there's room for either DEFINE-COMPILER-MACRO or macro based optimisations which would do constant folding in appropriate places. FOREIGN-ADDRESS-FUNCTION might end up as a macro (evaluating all arguments).

E.g. use of
(CAST var '(c-array character 32))
has a huge run-time overhead, which can be completely eliminated, using either LOAD-TIME-VALUE or by building the internal representation at compile-time (possible for all simlpe cases).

I wonder whether it should be better to
1) open the design:
2) document PARSE-C-TYPE and its converse, and present them like a typical MAKE-* constructor
3) Document the functions that operate on the internal representation only (all built-in functions) and
4) let the programmer insert PARSE-C-TYPE wher needed.

The goals I pursue are:
a) flexibility in the hands of the programmer (if needed)
b) compile-time optimisation
c) understandability

Thank you very much in case you're still reading this!

Regards,
	Jörg Höhle.