Subject: [clisp-list] modules and FFI dependency (was: Compiling clisp-2.30: No rule to make target `clisp-module')

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Tue Nov 26 01:48:04 2002

Hi,

For completeness, the below message from "./configure" is wrong. I consider it more to be describing a limitation of configure's --with-module extension or the clisp-link sh script.

The truth is that modules are a lower-level entity than the FFI. You can create and link modules without -DDYNAMIC_FFI.

Examples: the Amiga AFFI module and probably also Dan Stanger's GDI module. All modules written by hand as in my cookbook. FOREIGN-POINTER objects are available in CLISP without DYNAMIC_FFI.

It's the FFI that may use CLISP's module system, not the other way round.
E.g., compilation of DEF-CALL-IN/OUT forms generates C code as a side-effect. The emitted C code depends on CLISP's external module system.

Example: the regexp module and probably all the rest.

The FFI must not necessarily use the module system: with dynamic or shared libraries (the day they get into CLISP), you would not need C code anymore if you like.

     FFI
      | \
      |  modules
      |   |
      CLISP

This also means that the day CLISP will have dynamic libraries, the configure script will have to distinguish "link modules" (with Lisp->C->link+image) from "modules" (or packages?) which would be entities that can be loaded and activated on demand, without requiring relinking (like any other .lisp/.fas files). A priori, the modules/ subdirectory could contain both of them.

The quoted text below should say no more than:

The modules/ subdirectory contains external CLISP modules. On UNIX, modules can be linked to CLISP using the clisp-link shell script. This is what configure --with-module=X will prepare. Modules using DEF-CALL-IN/OUT forms depend on the FFI.

It is a limitation of the configure [and clisp-link?] script that using --with-module=X turns on --with-dynamic-ffi [but who would build CLISP without it except for testing?]

Pascal Bourguignon a écrit:
>Or alternatively,  have a  little note somewhere  about the  fact that
>modules should be found deeply.  Perhaps in configure, since I infered
>(cd modules ; for f in * ; ... ) from:
>
>         --with-module=MODULE  build with add-on MODULE
>                               this requires dynamic FFI and will turn it on
>                               if the FFI checks pass
>-->
>         --with-module=MODULE  build with add-on MODULE, name of directory,
>                               or one of the mutually exclusive
>                               directory/subdirectory, in ./modules/
>                               This requires dynamic FFI and will turn it on
>                               if the FFI checks pass
What's mutually exclusive? Postgres 6.x and 6.y? Why? Incompatible link libraries?

Thanks for trying to compile as much modules as possible and reporting.

Regards,
	Jörg Höhle.

---

From: Pascal Bourguignon <[email-masked]>
Date: Tue Nov 26 04:20:06 2002

Hoehle, Joerg-Cyril writes:
> Hi,
> 
> For completeness, the below message from "./configure" is wrong. I
> consider it more to be describing a limitation of configure's
> --with-module extension or the clisp-link sh script.
> [...] 
> The modules/ subdirectory contains external CLISP modules. On UNIX,
> modules can be linked to CLISP using the clisp-link shell script. This
> is what configure --with-module=X will prepare. Modules using
> DEF-CALL-IN/OUT forms depend on the FFI.
> 
> It is a limitation of the configure [and clisp-link?] script that
> using --with-module=X turns on --with-dynamic-ffi [but who would
> build CLISP without it except for testing?]
> 
> 

Perhaps, the fact that both modules export the same SQL package?

Or   the  same   LINUX   and  LINUX-AUX   packages   defined  by   the
bindings/linuxlibc* modules?

Granted, one perhaps  could compile all of them at  once, but we could
not build a system  with all of them.

The point is  that a naive user cannot  blindly configure and compile.
Perhaps it should be added in  the link.sh scripts tests to see if the
module can  be compiled for the  current target (or this  could be the
job  of  configure),  so  amigaos  would not  include  itself  to  the
compilation when  the amigaos libraries  are not available, and  so on
for the  linuxlibcs.  Thus, the  right bindings for the  target system
could be selected automatically.

> Thanks for trying to compile as much modules as possible and reporting.

Thanks for providing us with these modules! 

-- 
__Pascal_Bourguignon__                   http://www.informatimago.com/
----------------------------------------------------------------------
There is no worse tyranny than to force a man to pay for what he does not
want merely because you think it would be good for him.  -- Robert Heinlein

---

From: Pascal Bourguignon <[email-masked]>
Date: Tue Nov 26 04:45:04 2002

Hoehle, Joerg-Cyril writes:
> The truth is that modules are a lower-level entity than the FFI. You
> can create and link modules without -DDYNAMIC_FFI.
>
> Examples: the Amiga AFFI module and probably also Dan Stanger's GDI
> module. All modules written by hand as in my cookbook. FOREIGN-POINTER
> objects are available in CLISP without DYNAMIC_FFI.
>
> It's the FFI that may use CLISP's module system, not the other way
> round.  E.g., compilation of DEF-CALL-IN/OUT forms generates C code as
> a side-effect. The emitted C code depends on CLISP's external module
> system.
>
> The FFI must not necessarily use the module system: with dynamic or
> shared libraries (the day they get into CLISP), you would not need C
> code anymore if you like.
>
>      FFI
>       | \
>       |  modules
>       |   |
>       CLISP

I guess you mean this diagram:

             +---------------------+   +-------------+
             |    some modules     |   | some other  |
             | like postgresql642  |   |  modules    |
             +---------------------+   +-------------+
                  |       |                  |
              +---+       |                  |
              |           |                  |
              v           |                  |
           +-----+        |                  |
           | FFI |        |                  |
           +-----+        |   +--------------+
             | |          |   |
             | +------+   |   |
             |        |   |   |
             |        v   v   v
             |  +----------------+
             |  | module-system  |
             |  +----------------+
             |        |
             |        |
             v        v
          +---------------+
          |     CLISP     |
          +---------------+

because regarding this dependency on FFI, it seems that it's a problem
of dependency (configure? makefiles?) that makes it fail with:

full/lisp.run -B . -M full/lispinit.mem -norc -q -i clx/new-clx/clx clx/new-clx/image oracle/oracle postgresql642/postgresql bindings/linuxlibc6/linux bindings/linuxlibc6/wrap regexp/regexp wildcard/wildcard -x (saveinitmem "full/lispinit.mem")
;; Loading file /local/src/clisp-2.30/src/clx/new-clx/clx.fas ...
;; Loaded file /local/src/clisp-2.30/src/clx/new-clx/clx.fas
;; Loading file /local/src/clisp-2.30/src/clx/new-clx/image.fas ...
;; Loaded file /local/src/clisp-2.30/src/clx/new-clx/image.fas
;; Loading file /local/src/clisp-2.30/src/oracle/oracle.fas ...
;; Loaded file /local/src/clisp-2.30/src/oracle/oracle.fas
;; Loading file /local/src/clisp-2.30/src/postgresql642/postgresql.fas ...
*** - Incomplete FFI type ConnStatusType is not allowed here.
[1]>

when I configure and compile  with postgresql642 at once, because when
I did:

  - configure all modules I can but postgresql642, and compile ok,
  - then in the same build tree, I configure anew with only postgresql642,
  - and then compile it, it succeeded.

So, it seems that postgresql642 needs  FFI, but FFI is not compiled or
integrated  before postgresql642.  I need  to  configure/compile first
without it, and once again with it. See attached script.

Also, it's surprizing that when  you re-configure with a different set
of modules, those  that are not configured in the  later run are still
included (just because their object files are still there).  We'd need
at least a Makefile with a distclean target at top level...

> This also means that the day CLISP will have dynamic libraries, the
> configure script will have to distinguish "link modules" (with
> Lisp->C->link+image) from "modules" (or packages?) which would be
> entities that can be loaded and activated on demand, without requiring
> relinking (like any other .lisp/.fas files). A priori, the modules/
> subdirectory could contain both of them.