Subject: [clisp-list] package-lock and module interaction, saving disk space

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Fri Mar 14 03:39:17 2003

Hi,

>thanks for your patches - at this time CVS has __EVERYTHING__ you ever
>sent me.  It's a __VERY__ good time to sync!
Ok, I'll byte :) - got the 7MB snapshot, not yet installed.

BTW, this patch is missing:
http://sourceforge.net/tracker/index.php?func=detail&aid=549049&group_id=1355&atid=301355

CLISP's "new" package locking behaviour leads to an annoying

> lisp.run -M lispinit.mem

** - Continuable Error
INTERN("VECTOR-POSITION"): #<PACKAGE EXT> is locked
If you continue (by typing 'continue'): Ignore the lock and proceed
1. Break [1]> continue
  i i i i i i i       ooooo    o        ooooooo   ooooo   ooooo
  I I I I I I I      8     8   8           8     8     o  8    8
[...]
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2002

[1]> 

What I want is a mechanism to simply add new functions via modules, as I did from 1995 on, and documented in my article. Package-locking breaks this (except for stuff added to the free USER package) for the packages SYS and EXT, which I typically used.

The UNIX shell link-kit script may look easy to use. It's certainly powerful. But it's not what I call simple. Especially it's against my goal of saving disk space and keeping a single memory image for all possible module configurations, just switching lisp.exe if needed, an advice I've also given in my article.

Let me explain how I've been working for years:
I keep the original lisp.run and lispinit.mem from the original build, containing 0 modules.
I then build some lispx.run containing as many modules as I want, e.g. regexp, dynload, ffihacks, hinbug, gdi.

I invoke lispx.run -M lispinit.mem. The modules mostly lay dormant in the executable. Each is quite small, compared to the CLISP executable size.

If I need some functionality, I kiss the module awake:
(load "clisp:modules/regexp/regexp")
-> loading regexp.fas
which then provides the functionality, and full package, as needed.

I'd only generate image files containing e.g. gdi or regexp if the amount of .fas files became huge or if I had to deliver an application to some customer (which has not yet happened). So far, .fas file loading speed has been completely acceptable to me, and I by far prefer this speed/space trade-off.
Loading .fas files instead of working with an image is also nice to developers: work on what's current, not old definitions.

Summary: it is possible to use plenty of modules with just one lispinit.mem and a single extra lisp.run. (I could even delete the original lisp.run) -
I do it all the time.
No module sets, link kits etc. The value of a directory per module is in my eyes to provide
1. automation and
2. a Makefile etc. framework for C/Lisp compilation.

So what's missing now in CLISP?
1. In my eyes, the ability for the C code to create a package as needed for the module's functions. The full package definition and exports will be provided later by a Lisp/fas file - if ever loaded.
2. a home for modules such small that they don't need an own package. I used SYS or EXT so far, which package-lock breaks.

That's what my patches attempted to address. Maybe all that's additionaly needed is a variable to hold the packages that were unlocked during module initialization, and restore their state afterwards.

Regards,
	Jorg Hohle.
Remembers me about METAFONT: it is plain superfluous to dump a cmbase image. The single plain.base suffices, loading cmbase.mf into it is trivially fast.

---

From: Sam Steingold <[email-masked]>
Date: Fri Mar 14 07:05:06 2003

> * In message <[email-masked]>
> * On the subject of "[clisp-list] package-lock and module interaction, saving disk space"
> * Sent on Fri, 14 Mar 2003 12:38:34 +0100
> * Honorable "Hoehle, Joerg-Cyril" <[email-masked]> writes:
>

I do not consider this patch to be finished.
I write there:

why is the lock problem here at all?!
a module _should_ live in it's own package, and all new
packages are created unlocked (they _should_ be locked after
the module is created, indirectly, by being pushed on
*system-package-list* - I should mention this in the docs!).
SO: what is that package we are unlocking?!

Another issue is the automatic package creation.
I don't like it either (what if you misspell the package
name?  you will get an extra package, which you will notice
until much later, and some weird errors that will take
forever to debug).

You said:
I consider this issue still open, but low-priority.

> CLISP's "new" package locking behaviour leads to an annoying

CLISP does not lock new packages.

> What I want is a mechanism to simply add new functions via modules, as
> I did from 1995 on, and documented in my article.

what article?

> Package-locking breaks this (except for stuff added to the free USER
> package) for the packages SYS and EXT, which I typically used.

you should not use SYS & EXT.
You should use JCH-EXT and do

(without-package-lock ("EXT")
  (use-package '("JCH-EXT") "EXT")
  (re-export "JCH-EXT" "EXT"))

> The UNIX shell link-kit script may look easy to use. It's certainly
> powerful. But it's not what I call simple. Especially it's against my
> goal of saving disk space and keeping a single memory image for all
> possible module configurations, just switching lisp.exe if needed, an
> advice I've also given in my article.

This does not look like a good idea.
Memory images are under 2MB - cheap these days.

> which then provides the functionality, and full package, as needed.

so why can't you do that now?

> 1. In my eyes, the ability for the C code to create a package as
> needed for the module's functions. The full package definition and
> exports will be provided later by a Lisp/fas file - if ever loaded.

pushSTACK(my_name);
funcall(L(make_package),1);

> 2. a home for modules such small that they don't need an own
> package. I used SYS or EXT so far, which package-lock breaks.

No user code is allowed to touch anything in *SYSTEM-PACKAGE-LIST*.
If you are sure you know what you are doing, you may use
WITHOUT-PACKAGE-LOCK in LISP and
mark_pack_unlocked()/mark_pack_locked() in C.

Making it hard to modify packages in *SYSTEM-PACKAGE-LIST* is
__INTENDED__.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
If a train station is a place where a train stops, what's a workstation?