Subject: [clisp-list] compiling clisp-2.27 on cygwin

From: "Billinghurst, David (CRTS)" <[email-masked]>
Date: Mon Nov 5 04:46:06 2001

Just a note to say that I have built clisp-2.27 on windows 2000 using
cygwin.  I was then able to compile a maxima, and it "sort of works".

 There were two minor problems.

configure detects IPv6 due to some incomplete headers in the cygwin
distribution.  This can be worked around by editing $(build)/unixconf.h and
replacing 
#define HAVE_IPV6 1
with
#undef HAVE_IPV6
A better solution would be to fix improve the IPv6 detection, as was done
for gcc-3

cygwin doesn't have getservent().  This was worked around by modifying
src/socket.d near line 847, changing 
#ifndef WIN32
to
#if !defined(WIN32) || ! defined (__CYGWIN__)
I guess an autoconf test for getservent() would be preferable.

The build then proceded normally.  

There was one testsuite failure in excepsit.erg.

+++++++++++++++++++++++++++++++++++++++++
(Mr) David Billinghurst
Comalco Research Centre
PO Box 316, Thomastown, Vic, Australia, 3074
Phone:	+61 3 9469 0642
FAX:		+61 3 9462 2700
Email:	[email-masked]

---

From: Russell Senior <[email-masked]>
Date: Thu Nov 8 23:01:03 2001

I am working on a CGI program using CLISP to handle file uploads.  A
form completed by the client includes:

  <form enctype="multipart/form-data" 
        action="/cgi-bin/clisp-upload" 
        method="post">

and 

  <input type="file" name="foo">

The client sends a multipart/form-data to the clisp-upload program,
where the data is delivered by way of standard input: boundaries,
headers, payload and all.  Since the payload is binary, I need to read
it as binary.  According to impnotes that accompanies clisp-2.27
(under "Line Terminators", in the "Extensionts-1.4. Encodings"
section):

   The line terminator mode is relevant only for output (writing to a
   file/pipe/socket). During input, all three kinds of line
   terminators are recognized. If you do not want this, i.e., if you
   really want to distinguish LF, CR and CR/LF, you have to resort to
   binary input (function READ-BYTE).

I _do_ need to distinguish, since such line terminator squashing isn't
reversible in arbitrary binary data.  Instead of the function
read-byte, I am trying to use the function read-byte-sequence instead
(which I assume is equivalent under the circumstances), but am having
the following difficulty: I can't seem to call it on *standard-input*.
I am getting the message:

   *** - READ-BYTE on #<IO TERMINAL-STREAM> is illegal

If I try to setf the stream-element-type to '(unsigned-byte 8), as
suggested in the "Streams Dictionary" section ("Together with (SETF
STREAM-ELEMENT-TYPE), this function permits mixed character/binary
input from a stream."), I get:

   *** - (SETF STREAM-ELEMENT-TYPE) on #<IO TERMINAL-STREAM> is illegal

Er, so how do I get what I want?

Thanks in advance!

-- 
Russell Senior         ``The two chiefs turned to each other.        
[email-masked]      Bellison uncorked a flood of horrible       
                         profanity, which, translated meant, `This is
                         extremely unusual.' ''

---

From: Sam Steingold <[email-masked]>
Date: Fri Nov 9 06:29:06 2001

> * In message <[email-masked]>
> * On the subject of "[clisp-list] stream trouble"
> * Sent on 08 Nov 2001 23:00:46 -0800
> * Honorable Russell Senior <[email-masked]> writes:
>
yep.

>    *** - (SETF STREAM-ELEMENT-TYPE) on #<IO TERMINAL-STREAM> is illegal
yep.

how come you are reading from a terminal stream?
don't use terminal streams.
CLISP can open internet sockets - read from them, change their
STREAM-ELEMENT-TYPE &c.

-- 
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
non-smoking section in a restaurant == non-peeing section in a swimming pool

---

From: Russell Senior <[email-masked]>
Date: Fri Nov 9 11:21:01 2001

>>>>> "Sam" == Sam Steingold <[email-masked]> writes:

>> * In message <[email-masked]> * On the subject of
>> "[clisp-list] stream trouble" * Sent on 08 Nov 2001 23:00:46 -0800

Sam> yep.

>> *** - (SETF STREAM-ELEMENT-TYPE) on #<IO TERMINAL-STREAM> is

Sam> yep.

No doubt.

Sam> how come you are reading from a terminal stream?  don't use
Sam> terminal streams.  

I am reading from *standard-input*, as in:

  (ext:read-byte-sequence buffer *standard-input* :end buffer-size)

since that is where Apache (in this case) puts the data.  Is it
possible to accomodate this behavior and, if so, how do people
typically do it?  I have reviewed the mailing list traffic going back
to roughly June 1996 and haven't found the question addressed.

Thanks.

---

From: Sam Steingold <[email-masked]>
Date: Fri Nov 9 13:12:01 2001

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] stream trouble"
> * Sent on 09 Nov 2001 11:20:13 -0800

try this:
(let ((true-input (two-way-stream-input-stream
                   (symbol-value
                    (synonym-stream-symbol *standard-input*)))))
  (setf (stream-element-type true-input) '(unsigned-byte 8))
  (read-byte-sequence buffer true-input))

-- 
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
Bus error -- driver executed.

---

From: lin8080 <[email-masked]>
Date: Fri Nov 9 23:54:01 2001

Hallo

[1] I am new in this list and my english is not good
[2] Because I am new in lisp, I do not read all documents 
    (but 3 books and what I can find online)
[3] I run the release 2.27 (2001-07-17) of GNU CLISP
[4] I work private on a kind of neural network for experiments
[5] The ()-structure is modular, each () has up to 3kb stored in a
separate file and load on demand. 
[6] The question(s) :

[7]
When I do (load "modul-03.lisp"), I could find no way to unload this
module. My thought is to free up all variables for other modules. This
way I will be able to use the same variable names in different modules.
So, is there a way, to make the load-process unhapened ? (function
unload - undefined function it says). What is a usual way in this case ?

[8]
An other problem is (say it simply) a way like a command
(bound-variable-xyz) and (unbound-variable-xyz). I found setq and setf,
but no unsetq, unsetf. Only the possibility like a new (setq variable 0)
is known, but I wish that the variable is not known by the system after
use. If there are commands in this direction, please let me know, so I
can read somewhere about this. (there are some comments in the
news-group comp.lang.lisp under: On the implementation of lexical
variables..)

[9]
For example, I have hugh lists filled with numbers (the modules), all in
the same structure, for example modul-18: 
   (47 53 29) (n 17 38 42) 
   (36 28 51) (s 23 58 29)
   ... 20.000 more (these are star-positions taken from a catalog)
So I want to ask, whether in clisp is a possibility to make something
like a car and a cdr of "modul-18.lisp"- the file (like load-read(that
is: bind a variable)-unload). 

At the moment I load the whole modul-18 (there are 6 other modules like
this) and make cons-pairs like (...).(...), and then I get the runtime
problem, finding the position of 8732 inside this module, to work with
8731 and 8733. This needs to much time. Alternatively I thought about
big 2-dimensional arrays, but witch is the better way to handle
positions in this case ? (the aim is a short runtime)

For testing, I write "modul-18" in the editor, typing position-numbers,
to find easy one position again, this looks like
   (12456 ((47 53 29) (n 17 38 42)))
but doing so, I get errors from the other modules (lambdas they want).
Seems like I have to rewrite these modules, but this is much work and
will take some time.

[10]
The ready programm should give me the position of a star in 6 different
coordinate systems, so I typed in one position and get back 5 equivalent
coordinate places. And because it is neuronal, there is that
experimental test, whether the programm can learn to transform the
positions.

[11] last words:
Please, I do not expect complete solutions. Only some tips or comments
should be helpful, because working with lisp is so magnificent and
funny. I see myself confronted with an ocean and swimming does so good
...

stefan

---

From: Harley Gorrell <[email-masked]>
Date: Sat Nov 10 09:23:01 2001

> Hallo

   Guten Tag Stefan!  Mein Deutsch is nicht als gut als seine English.

   I cant answer all your questions but I can answer some.

> When I do (load "modul-03.lisp"), I could find no way to
> unload this module.

   Lisp is a garbage collected language.  When something is
not referenced anymore, the GC will reclaim it.  So
generally dont dont need up "unload".

;; define the var to be special
(defvar *bigarr* nil)
;; bigarr now uses lots of space
(setf *bigarr* (make-array 10000))
*BIGARR*
;; now *bigarr* has no reference to the array, it will be collected
(setf *bigarr* nil)

> My thought is to free up all variables for other modules. This

Write function that sets all the vars you care about to some
know value. (like nil)

> (bound-variable-xyz) and (unbound-variable-xyz).

   (makunbound '*bigarr*)

   http://www.xanalys.com/software_tools/reference/HyperSpec/

> is: bind a variable)-unload).

   Read the file into a var and walk across it.

(defun read-file (fname)
  ;; :eof should be an uninterened symbol
  (let ((fval :eof))
    (with-open-file (fstream fname)
     (loop
       do (setf fval (read fstream nil :eof))
       until (eq fval :eof)
       collect fval))))

hope this helps,
harley.

---

From: David Morse <[email-masked]>
Date: Sat Nov 10 09:59:02 2001

> unload - undefined function it says). What is a usual way in this case ?

You're right, its hard.  Look at "unintern" and "delete-package" though.

> but no unsetq, unsetf. 

"makunbound".  ("fmakunbound" for functions.)

---

From: Russell Senior <[email-masked]>
Date: Tue Nov 13 01:33:02 2001

>>>>> "Sam" == Sam Steingold <[email-masked]> writes:

Sam> try this: 

Sam>    (let ((true-input (two-way-stream-input-stream
Sam>                       (symbol-value (synonym-stream-symbol
Sam>                       *standard-input*))))) 
Sam>       (setf (stream-element-type true-input) '(unsigned-byte 8))
Sam>       (read-byte-sequence buffer true-input))

I ran the following at the Linux command line (clisp-2.27):

  $ clisp -q -norc -x '(let ((true-input (two-way-stream-input-stream (symbol-value (synonym-stream-symbol *standard-input*))))) (describe true-input)) (exit)'

  *** - SYNONYM-STREAM-SYMBOL: argument #<INPUT STRING-INPUT-STREAM> should be a stream of type SYNONYM-STREAM

Starting CLISP normally and playing around, I find:

  [1]> (describe *standard-input*)

  #<IO SYNONYM-STREAM *TERMINAL-IO*> is an input/output-stream.

  [2]> (two-way-stream-input-stream *terminal-io*)

  *** - TWO-WAY-STREAM-INPUT-STREAM: argument #<IO TERMINAL-STREAM> should be a stream of type TWO-WAY-STREAM
  1. Break [3]> 

If *terminal-io* isn't of type TWO-WAY-STREAM, what is an input/output-stream?

---

From: Sam Steingold <[email-masked]>
Date: Tue Nov 13 06:54:08 2001

> * In message <[email-masked]>
> * Sent on 13 Nov 2001 01:32:17 -0800

yes, when started with "-x", CLISP puts the argument into a string and
reads from it.

>   *** - TWO-WAY-STREAM-INPUT-STREAM: argument #<IO TERMINAL-STREAM> should be a stream of type TWO-WAY-STREAM

of course: when both 0 and 1 are terminals, then *terminal-io* is a
TERMINAL-STREAM and not a TWO-WAY-STREAM.

my trick should work when you read _from a file_:

$ echo '(let ((true-input (two-way-stream-input-stream (symbol-value (synonym-stream-symbol *standard-input*))))) (describe true-input)) (exit)' > z
$ clisp -q -norc < z
#<INPUT BUFFERED FILE-STREAM CHARACTER #P"/dev/fd/0" @1> is an input-stream.
$

Would you like to write a paper describing what you are doing?
We can put it on our web site, like we did with
<http://clisp.cons.org/clash.html>. 

-- 
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
Between grand theft and a legal fee, there only stands a law degree.

---

From: lin8080 <[email-masked]>
Date: Tue Nov 13 10:59:07 2001

David Morse, Harley Gorrell schrieben:

Thank you for your tips. The next problem is not far, but I try and
learn.

greetings
stefan

---

From: Russell Senior <[email-masked]>
Date: Tue Nov 13 14:06:02 2001

>> $ clisp -q -norc -x '(let ((true-input (two-way-stream-input-stream
>> (symbol-value (synonym-stream-symbol *standard-input*)))))
>> (describe true-input)) (exit)'
>> 
>> *** - SYNONYM-STREAM-SYMBOL: argument #<INPUT STRING-INPUT-STREAM>
>> should be a stream of type SYNONYM-STREAM

Sam> yes, when started with "-x", CLISP puts the argument into a
Sam> string and reads from it.

What would be the "approved" way of executing lisp code "out-of-band"
so that standard input is reserved for data?  Is it even possible?

Putting the code into a file (test.lisp):

  cat > test.lisp
  (let ((true-input (two-way-stream-input-stream (symbol-value (synonym-stream-symbol *standard-input*))))) (describe true-input))
  ^D

 and running:

  clisp -q -norc test.lisp

or 

  clisp -q -norc -i test.lisp

gives me the message:  

while:

  clisp -q -norc < test.lisp

"works", but as far as I can tell that is going to interfere with the
way the data is sent to the CGI program.  

Is there another way?

A while ago I could see that one solution is to make my CGI script
look like:

  #!/bin/sh

  INCOMING=/tmp/$$.incoming-data

  cat > $INCOMING
 
  clisp -q -norc ...

where clisp reads from $INCOMING in some fashion, obviously no longer
constrained by the *standard-input* intricacies.  I thought that there
might be a way of avoiding that kind of punting.  However, having
beaten my head against a wall for a while, I am not sure there is.

>> [more russell fumbling around elided]

Sam> of course: when both 0 and 1 are terminals, then *terminal-io* is
Sam> a TERMINAL-STREAM and not a TWO-WAY-STREAM.

Sam> my trick should work when you read _from a file_:

Sam> $ echo '(let ((true-input (two-way-stream-input-stream
Sam> (symbol-value (synonym-stream-symbol *standard-input*)))))
Sam> (describe true-input)) (exit)' > z $ clisp -q -norc < z #<INPUT
Sam> BUFFERED FILE-STREAM CHARACTER #P"/dev/fd/0" @1> is an
Sam> input-stream.  $

Sam> Would you like to write a paper describing what you are doing?
Sam> We can put it on our web site, like we did with
Sam> <http://clisp.cons.org/clash.html>.

That seems to be a step-by-step cookbook on how to use clisp as a
shell.  I'd be happy to do that for my problem once I've figured out
how to make it work.  In the meantime, I'll try to put together a more
thorough explanation from start to finish of what I am trying to do.

---

From: "Thomas F. Burdick" <[email-masked]>
Date: Tue Nov 13 14:40:07 2001

Russell Senior writes:

 > In the meantime, I'll try to put together a more thorough
 > explanation from start to finish of what I am trying to do.

I can try to do that for you here, since I don't know the answer,
either.  I've come across the same problem, but avoided it with the
same technique I used to improve performance: I had CLISP open a
socket, and stay running, and served CGI requests with a very small C
program that forwarded the request to the CLISP socket, and sent the
response back to the web server.

The problem he's having is this: CGI requests can come in a number of
forms; one of them is a byte stream on standard input.  The problem is
that the request is a stream of bytes, but in lisp, *standard-input*
is a character stream.  Reading characters from it and getting the
value of the byte by calling char-code would happen to work, but it's
wrong and fragile.  How should one deal with this byte stream?

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'

---

From: Sam Steingold <[email-masked]>
Date: Wed Nov 14 06:46:06 2001

> * In message <[email-masked]>
> * Sent on 13 Nov 2001 14:05:16 -0800

$ cat z
(defun foo ()
  (let ((true-input (two-way-stream-input-stream
                     (symbol-value
                      (synonym-stream-symbol *standard-input*)))))
    (describe true-input)
    (format t "~&line: ~s~%" (read-line true-input))
    (setf (stream-element-type true-input) '(unsigned-byte 8))
    (format t "~&byte: ~s~%" (read-byte true-input)))
  (exit))
(foo)
$ clisp -q -norc -i z < z
;; Loading file z ...
 
line: "(defun foo ()"
byte: 32
$

Please note that this will _not_ be necessary in the next CLISP release
(*standard-input* and *standard-input* will be real streams, not
synonym streams, but _ONLY_ when appropriate, i.e., when not a terminal).

watch <clisp-devel> for the CVS commit message about implementing this,
if you want to test this feature.

> I'd be happy to do that for my problem once I've figured out how to
> make it work.  In the meantime, I'll try to put together a more

good.
when/if you get around to doing a text, please use clash.html as a
template (use the same header/footer, CSS, xhtml &c).

-- 
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
A professor is someone who talks in someone else's sleep.

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Jan 26 10:30:06 2002

Some CLISP code of mine was crashing, but only when run from a loaded
memory image, not when loaded directly as .lisp or .fas files.

It turns out that it was failing to detect a null pointer coming out
of a glibc2 function (I'm using a CLISP built using
--with-modules=bindings/linuxlibc6). And so it then passed the
null pointer to the library, which tries to dereference a structure
member in it, causing a fault at address 0x10.

My program tries to capture a null pointer by doing this
ugly hack:

(defconstant *null-pointer* (linux:realloc (linux:malloc 1) 0))

This gives you an object which prints like this:

    #<FOREIGN-ADDRESS #x00000000>

It compares positively to other such null pointers under
the EQUALP function, so you can use it, for instance, to detect
linux:readdir returning NULL.

Now here is the rub. After having defined *null-pointer*, if
you save the memory image with ext:saveinitmem, and then
reload, the value of *null-pointer* now prints like this:

    #<INVALID FOREIGN-ADDRESS #x00000000>

     ^^^^^^^^

This object no longer compares EQUALP to newly computed null pointers.
Something in its representation has changed on save and reload, which
is disturbing.

I'd like to know what the recommended way is to test for
null pointers coming out of the FFI. Should I convert
#<... #x00000000> to a string and parse out the hex or what? 
Or compile in a C function to do it?

For now I have a workaround, which is to make *null-pointer* a
variable, and setf a new value to it right after loading the memory
image.

---

From: Sam Steingold <[email-masked]>
Date: Sun Jan 27 11:25:03 2002

> * In message <[email-masked]>
> * On the subject of "[clisp-list] Strangeness involving FFI pointers and ext:saveimage."
> * Sent on Sat, 26 Jan 2002 10:29:50 -0800 (PST)
> * Honorable Kaz Kylheku <[email-masked]> writes:
>

I have no idea what _was_ the right way before, so I just added a
function FFI:FOREIGN-ADDRESS-NULL which will check for a NULL pointer.

(in the CVS now)

-- 
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
Linux - find out what you've been missing while you've been rebooting Windows

---

From: Kaz Kylheku <[email-masked]>
Date: Sun Jan 27 12:57:03 2002

> function FFI:FOREIGN-ADDRESS-NULL which will check for a NULL pointer.

Thank you very much for this! Good name too, no -P suffix, perfectly
consistent with the standard NULL predicate. ;)

> (in the CVS now)
          ^^^

Speaking of which, i've developed a Lisp layer over CVS called MCVS.
(Currently requires CLISP and Linux, but should be portable to
other Lisps and systems). This is the project in which I was having
the little FFI problem above.

Remember that little discussion we had a while back in the GNU mailing
list, concerning file renaming in the gnu-cvs mailing list?

MCVS versions the directory structure. It requires no modifications
to CVS at all. And it's very close to ready for a 0.0 alpha release.

MCVS is packaged up as a shell utility, which you can use like this:

    mcvs mv *.lisp src		 # move all the lisp files into a src/ subdir
    mcvs *.txt *.html doc	 # move text and HTML files into doc/
    emacs doc/readme.html	 # also make changes to a file
    vi doc/readme.txt 		 # or two (keeping things politically neutral)
    mcvs diff | mcvs filt	 # view diffs, decode machine-generated names
    mcvs ci			 # commit all of the above

Developers can even change the directory structure in parallel and then
merge. What's more, it's possible to distribute patches which are made
against the MCVS representation of your project. So in principle good
old diff and patch are amplified to do filesystem restructuring too.

I'm already using MCVS to version its own sources. And I've already used
the move command more than once! You don't realize just how great
it is to have this until you have it.

Lisp is about to shake up the CVS world a little. I hope. :)

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Sep 21 13:03:01 2002

I use a command similar to cvs -c mcvs-main to compile Meta-CVS.
this *always* compiles mcvs-main.lisp, but for all the modules
that this one depends on, it only compiles if the .fas files don't
exist or are out of date, which is nice, sensible behavior.

Would it be braindamaged in some way if -c treated the root module
in the same way as the others?

---

From: Sam Steingold <[email-masked]>
Date: Mon Sep 23 06:40:02 2002

> * In message <[email-masked]>
> * On the subject of "[clisp-list] Re: clisp -c always compiles"
> * Sent on Sat, 21 Sep 2002 13:02:15 -0700 (PDT)

-c means "compile this file, no matter what".
COMPILE-FILE recompiles the REQUIREd files, when necessary.
see <http://clisp.cons.org/impnotes/system.html#require>.

> in the same way as the others?

Yes. 

Use make(1) or defsystem or roll your own check (see
CLOCC/CLLIB/fileio.lisp, function LOAD-COMPILE-MAYBE).

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.3 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Why use Windows, when there are Doors?