Subject: [clisp-list] Re: array of strings?

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Thu Feb 26 06:24:02 2004

Hi,

Sam Steingold needs to interface to the obscure matGetDir funtion of MathWorks/Matlab?
<http://www.mathworks.com/access/helpdesk/help/techdoc/apiref/engoutputbuffer.shtml>
I went to that page and I understand next to nothing. Great documentation!

1. "int *num" seems to be an :out variable. But is it possibly :in-out?

2. Sam will have to invoke mxFree() on the result (the callee calls mxCalloc() internally). Therefore, a return type like (c-array c-string N) cannot be used, since the original pointer must be preserved.

This function is a typical candidate where I believe a higher-level Lisp look&feel API would be valuable (aka "thick" interface in Ada), instead of C interface weirdness brought to Lisp. The nice Lisp-visible function internally needs several calls to low-level code to accomplish desired functionality (e.g. get-dir + mxFree).

CLISP's FFI allows relatively complex protocols to be covered (e.g. :malloc-free convention), but it cannot yet declare everything in one swoop. Existing libraries are too diverse for that.

I once thought about declarable resources. This get-dir API function would fall into that schema: one could declare the finalizing function, mxFree() in the present case, or e.g. CloseSocket/Window etc. Very handy for the Amiga. Even more when GC-collectable.

Here's my guess at what the documentation may mean:
The code remembers def-var-call for variable sized arguments, with the addition of freeing resources.

(def-var-call-out %get-dir-internal
  (:library "foo.library")
  (:language :stdc)
  (:name "matGetDir")
  (:arguments (MAT-file c-pointer)
              (num-entries int :out))
  (:guard (foreign-pointer-not-null return)) ; not-yet in CLISP, ignore
  ;;(:return-type (c-pointer (c-array c-string num-entries))) ; not in CLISP
  (:return-type c-pointer))

(def-call-out mx-free
  (:library "mathworks.library")
  (:language :stdc)
  (:name "mxFree")
  (:arguments (memory c-pointer))
  (:return-type nil)) ; guessed

(defun mw:get-dir (MAT-file)
  ;;(declare (type (or foreign-pointer foreign-variable) MAT-file))
  (multiple-value-bind (membuf num-entries)
       (get-dir-internal MAT-file)
    (when (and/or? (foreign-pointer-not-null membuf); superfluous?
               (> 0 num-entries))       ;TODO? need mxfree when zero num-entries?
      ;;(setq membuf (set-pointer-base membuf :new))
      ;;(ffi-finalize membuf #'mx-free)
      (unwind-protect
          ;;(coerce 'list #) ; coerce array to Lisp if you wish
          (with-c-var (names `(c-ptr (c-array c-string ,num-entries)))
            ;; or use (c-ptr (c-array-max char ,(1+ mxMAXNAM))) instead of c-string
            (setf (cast names 'c-pointer) membuf)
            names)
        (mxFree membuf)))))
;;TODO? error out if num-entries <0 (or NULL pointer)

Disclaimer: Completely untested, abundant typos, etc.

(defun ffi-finalize (x terminator &key (mark-invalid t))
  ;; This relies on correct management of pointer-valid information
  (when (validp x)
    (unwind-protect
        (funcall terminator x)
      (when mark-invalid (setf (validp x) nil)))))

Kaz wrote:
> They must do some funny allocation to allow you to blow this away with
> just one call to their mxFree() function.
Typical C: Once you know the size of all strings, allocate enough room for them and the array and fill that memory, building up the pointer structure for the array. A single free() is felt to be very convenient among C programmers.

>maybe the MATFile object owns these strings,
Indeed, there's also a completely distinct interpretation. The array returned could be just that: an array. The string's characters could reside in the MAT-file itself, and share lifetime with that object. We just don't know.
C (and C++) documentation is traditionally silent about lifetime. It gets on my nerves.

Hope this help,
	Jorg Hohle

---

From: Sam Steingold <[email-masked]>
Date: Thu Feb 26 08:47:11 2004

> * Hoehle, Joerg-Cyril <[email-masked]> [2004-02-26 15:13:49 +0100]:
>
> This function is a typical candidate where I believe a higher-level
> Lisp look&feel API would be valuable (aka "thick" interface in Ada),
> instead of C interface weirdness brought to Lisp. The nice
> Lisp-visible function internally needs several calls to low-level code
> to accomplish desired functionality (e.g. get-dir + mxFree).

yes, but they do not provide that.
we need to write a C (not FFI) module to accomplish this.

> CLISP's FFI allows relatively complex protocols to be covered
>(e.g. :malloc-free convention), but it cannot yet declare everything in
>one swoop. Existing libraries are too diverse for that.

indeed. this makes it an imperative to offer a very low-level FFI layer
which would be suitable to port UFFI to CLISP.
More and more software relies on UFFI.
We must support it.

> I once thought about declarable resources. This get-dir API function
> would fall into that schema: one could declare the finalizing
> function, mxFree() in the present case, or e.g. CloseSocket/Window
> etc. Very handy for the Amiga. Even more when GC-collectable.

yes, a def-c-type should take a :FINALIZER option.

> Here's my guess at what the documentation may mean:

indeed, this works.
thanks!

-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.honestreporting.com>
MS: Brain off-line, please wait.