Subject: [clisp-list] ffi::parse-c-type optimization (was: critique of default-foreign- language)

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Thu Apr 25 04:54:04 2002

Hi,

Sam wrote:
> actually, in this particular case, there is no reason not to call
> PARSE-C-TYPE at macroexpansion time e.g., 
> 
>   (defmacro offset (place offset type &environment env)
>     (setq place (macroexpand place env))
>     (if (foreign-place-p place 'FOREIGN-VALUE)
>       `(FOREIGN-VALUE (%OFFSET ,(second place) ,offset 
> ,(M-PARSE-C-TYPE type)))
>       (err `(offset ,place ,offset ,type))))
> 
>   (defmacro M-PARSE-C-TYPE (type)
>     (if (constantp type) (PARSE-C-TYPE (eval type)) 
> `(PARSE-C-TYPE ,type)))

This folding is a premature optimization IMHO. It works with simple cases (c-array uint8) and may break with others (DEF-C-STRUCT), as then a function object is stored in the signature.

>(ffi::parse-c-type 'regmatch_t)
#(C-STRUCT #(RM_SO RM_EO) #<COMPILED-CLOSURE :LAMBDA> INT INT)

I didn't find enough time to think about the relationship between compilation and load environments, nor the exact nature of this function object. CLISP's ability to output a closure to a .fas file doesn't mean that it's the correct thing to rely on here. Greetings from EQ/EQUALP/UNDECIDABLE-FUNCTION-EQUAL.
Havoc can happen when resurrecting this way (c-struct class #) a closure emanating from ancient compile-times.

CLISP can inline (c-struct LIST #) or (c-struct VECTOR #) safely,
because #<SYSTEM-FUNCTION LIST> (not #'LIST!) is saved
in .fas as #.(SYS::%FIND-SUBR 'common-lisp::list) and will
thus be loaded without duplication.

I believe DEFINE-COMPILER-MACRO is the appropriate idiom for
the discussed optimization, together with LOAD-TIME-VALUE. That's
how I came to recently submit a bug w.r.t. COMPILER-MACRO.

When in the interpreter, evaluating a (DEF-xyz-CALL) or CAST
form, PARSE-C-TYPE needs to be called in any case, so there's
no point in delaying it or calling it early.

My idea was along:
1. simple cases: inline expand (like you do above):
  (C-ARRAY uint8 2) -> '#(C-ARRAY #).
2. more convoluted, yet constant cases:
  (LOAD-TIME-VALUE (PARSE-C-TYPE #))
  helps with (defun ... (cast ...)) example.
3. Maybe special-case `(c-array uintx ,(length foobar)) to
  (VECTOR 'c-array 'uintx (length #)) -- the next most common
  case after constant types, I suppose. That why I recently
  criticized that a backquote expansion is not portably
  defined, unlike Scheme.
4. Other: PARSE-C-TYPE (resp. PARSE-C-FUNCTION) at run-time,
  i.e. refuse optimizing.

(defun foo (x)
  (CAST ... '(C-ARRAY UINT8 2))
In this case, an optimization that works for the interpreter would help it as well as the compiler, as otherwise, PARSE-C-TYPE is called by the interpreter for each invocation of FOO.
LOAD-TIME-VALUE is #'identity for the interpreter.
But for interpretation of DEF-*-CALL (declarative, top-level forms), PARSE-C-TYPE will have to be called anyway, so inlining (constant folding) doesn't help the interpreter.

> Actually, is it possible to extract the calling convention from the
> library (foo.so) itself?  Then all this would be much easier.
It's not. Definitely not. Even having a CORBA IDL wouldn't be enough.

> 3. when the type is non-constant, we should require :LANGUAGE argument
>    anyway, whether by using separate forms or requiring it explicitly,
>    and I prefer the latter.
This area is the one I thought about when I said earlier that CLISP makes no guarantees whatsoever -so far(?)- about what is parsed at what time. So the programmer cannot reliably know when it's needed.

> Another thing is: suppose the library he is interfacing with was
> recompiled with a different calling convention.
Quite unlikely -- Never heard of. The possibility I see is specific to CLISP: turning the module based linux-libc6.lisp into a dynamic library call.

> Then he would have to go over _all_ his code and replace all
> DEF-STDC-STDCALL-CALL-OUT with DEF-STDC-CALL-OUT or modify all his
> :LANGUAGE arguments.
This is a trivial M-x query-replace ... ! or vi /// exercise.
It even works with MS-Windows' pitiful Notepad.

Regards,
	Jorg Hohle.