Subject: [clisp-list] Re: Building CLISP for Windows

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Fri Mar 8 02:07:03 2002

Hi,

Sam Steingold wrote:
> see CLISP/win32*/
> I used msvc6 with CLISP/win32msvc/makefile.msvc5 and followed the
> instructions in CLISP/win32msvc/INSTALL
> everything worked just dandy.
I can confirm this, using MSVC6.

However, here are some observations of mine:

o (queens 21.0s0) still crashes.
--------
o genclisph.exe and as a consequence clisp.h are not generated and there's even no makefile target for them.
Did anyone ever build a module on MS-windows?

What is the wgenclisph.exe that makefile target clean0 may remove?

I used the following to complete the makefile (somebody please add this to makemake):
genclisph.exe : genclisph.obj
	$(CC) $(CFLAGS) $(CLFLAGS) genclisph.obj /Fegenclisph.exe
clisp.h : genclisph.exe
	genclisph.exe > clisp.h

--------
clisp.h is not quite correct:
mtest.c (generated from my private mtest.lisp) includes clisp.h
clisp.h(24) : error C2632: 'long' gefolgt von 'long' ist unzulaessig
clisp.h(25) : error C2632: 'long' gefolgt von 'long' ist unzulaessig
It seems like genclisp.d doesn't reproduce lispbibl.d which says:
#elif defined(MICROSOFT)
  typedef __int64           SLONGLONG;
  typedef unsigned __int64  ULONGLONG;
  #define HAVE_LONGLONG
#endif
and just outputs:
typedef long long           SLONGLONG;
for Microloft VC, which is not accepted.

--------
o when linking lisp.exe
Befehlszeilenwarnung D4002 : Unbekannte Option '-lm' wird ignoriert
Probably makemake needs to be changed?

--------
o dirkey.lisp:
The following functions were used but not defined:
 LDAP::TEXT
TEXT seems to come from package SYSTEM instead (SYSTEM::TEXT) and used
in producing error messages (IIRC it's a marker for GETTEXT).

--------
o One of dirkey.lisp or dirkey.c should be renamed (like affi|1,
foreign|1, rexx|1.{d,lisp}) This will avoid dirkey.c to get
overwritten the day dirkey.lisp will contain some FFI definition, when
it's supposed to be generated from dirkey.d. Furthermore, makefile
(MSVC6.0) get confused and wants to rebuild my dirkey, maybe because
of this?? or maybe because somehow dirkey.c disappeared?

--------
Seen after I added MODULE(mtest) to modules.h:
modules.i.c
modules.h(1) : warning C4113: 'void (__cdecl *)()' weicht in der
Parameterliste von 'void (__cdecl *)(struct module_ *)' ab
modules.h(1) : warning C4113: 'void (__cdecl *)()' weicht in der Parameterliste
von 'void (__cdecl *)(struct module_ *)' ab
"x deviates in the parameter list from y."
Looks like something was missing or not expanded. That doesn't look like a good definition.

I also have 5KB worth of compiler warnings from varying C files (in German, sorry). Where should I send that?

Regards,
	Jorg Hohle.

---

From: Sam Steingold <[email-masked]>
Date: Fri Mar 8 11:02:03 2002

> * In message <DFD875E85664D3118FA6080006277DE705216404@U8PN2>
> * On the subject of "[clisp-list] Re: Building CLISP for Windows"
> * Sent on Fri, 8 Mar 2002 11:06:10 +0100
> * Honorable "Hoehle, Joerg-Cyril" <[email-masked]> writes:
>
> Sam Steingold wrote:
> > see CLISP/win32*/

msvs has a pretty decent debugger - maybe you could track this down?
you will have to recreate makefile:
look at Makefile.devel and add "debug" to the makemake options.

> o genclisph.exe and as a consequence clisp.h are not generated and
> there's even no makefile target for them.

IIUC, you need add an interface to LoadLibrary (win32 version of dlopen)
to spvw.d:dynload_modules().
Bruno proposed using libltdl from GNU libtool.
would you like to do this?

> What is the wgenclisph.exe that makefile target clean0 may remove?

Bruno?

> 	$(CC) $(CFLAGS) $(CLFLAGS) genclisph.obj /Fegenclisph.exe

this was disabled on win32 together will the rest of the modules
interface.
I enabled it - re-create your makefiles and try it
(you will need some UNIX utils, like sed, for some other module
targets, and I am not sure that nmake supports things like $@ &c.)

> for Microloft VC, which is not accepted.

fixed, thanks.

> of this?? or maybe because somehow dirkey.c disappeared?

another mystery solved!!! thanks!
fixed too.

> I also have 5KB worth of compiler warnings from varying C files (in German, sorry). Where should I send that?

Please fix them and send in a patch instead! :-)

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
Bus error -- driver executed.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sat Mar 23 01:35:02 2002

Hi,

Friday, March 08, 2002, 8:06:10 PM, you wrote:

Joerg-Cyril> o (queens 21.0s0) still crashes.
I investigated the stack handling in win32. It seems that all
works ok, so hash table handling code just eats too much
program stack. Thus, it is not a crash, but a nice stack
overflow handling.

To increase maximal hashtable size you may relink lisp with other
stack size instead of current 3145728 (it's set in Makefile by
editbin utility when building in MSVC). This memory doesn't being
allocated at once but only when it's needed through exception
mechanism, so I find it safe to change 3M value to, say, 16M
(16777216). Maybe we should ask Sam to change Makefile.in ?

I'm not a unix expert and cannot say what stack size available
on FreeBSD version. But the following function

(defun a(x) (print x) (+ x (a (+ x 1))))

being called as (a 1) counts until 3287 in win NT and approx.
until 15000 in FreeBSD. I wonder whether it's because of another
aligment or because of greater stack limit. It's interesting,
what will print linux version of clisp.

Question to honorable Sam: You recently maked weak-hash-tables,
may be you know, why hash tables are so program-stack hungry ?
Max. HT size seems roughly proportional to program stack size,
i.e. for 1.5M stack max HT is 55000, for 3M stack max HT size is
123000.

-- 
Best regards,
 Arseny                            mailto:[email-masked]

---

From: Sam Steingold <[email-masked]>
Date: Sat Mar 23 09:44:03 2002

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] Re: Building CLISP for Windows"
> * Sent on Sat, 23 Mar 2002 19:34:46 +1000
> * Honorable Arseny Slobodjuck <[email-masked]> writes:
>
> I investigated the stack handling in win32. It seems that all works
> ok, so hash table handling code just eats too much program
> stack. Thus, it is not a crash, but a nice stack overflow handling.
> 
> stack size instead of current 3145728 (it's set in Makefile by editbin
> utility when building in MSVC).

actually, you o not have to re-link - you can do this with the existing
lisp.exe. 

> This memory doesn't being allocated at once but only when it's needed
> through exception mechanism, so I find it safe to change 3M value to,
> say, 16M (16777216). Maybe we should ask Sam to change Makefile.in ?

how will this affect the lesser woe32 platforms, like win95?

> being called as (a 1) counts until 3287 in win NT and approx.  until
> 15000 in FreeBSD. I wonder whether it's because of another aligment or
> because of greater stack limit. It's interesting, what will print
> linux version of clisp.

this depends on "ulimit -s".

> Question to honorable Sam: You recently maked weak-hash-tables, may be
> you know, why hash tables are so program-stack hungry ?  Max. HT size
> seems roughly proportional to program stack size, i.e. for 1.5M stack
> max HT is 55000, for 3M stack max HT size is 123000.

I don't think it is:

[24]> (make-hash-table :size (ash most-positive-fixnum -1) )
#S(hash-table eql)
[25]> (make-hash-table :size (1+ (ash most-positive-fixnum -1) ))
*** - Hash table size 8388608 too large

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
I don't want to be young again, I just don't want to get any older.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sat Mar 23 16:12:11 2002

Hello Sam,

>> say, 16M (16777216). Maybe we should ask Sam to change Makefile.in ?

Sam> how will this affect the lesser woe32 platforms, like win95?

I didn't think about it. I'll try to check on win95.

>> max HT is 55000, for 3M stack max HT size is 123000.

Sam> I don't think it is:

[24]>> (make-hash-table :size (ash most-positive-fixnum -1) )
Sam> #S(hash-table eql)
[25]>> (make-hash-table :size (1+ (ash most-positive-fixnum -1) ))
Sam> *** - Hash table size 8388608 too large

On winnt all just the same, but if you'll begin to work with big HTs
(limits I have roughly pointed out), "Program stack overflow. RESET"
arises. It's not necessary to fill all such a big HT, usually 5000
elements are enough.

---

From: Sam Steingold <[email-masked]>
Date: Sat Mar 23 19:02:08 2002

Hi Arseny,

> * In message <[email-masked]>
> * On the subject of "Re[2]: [clisp-list] CLISP stack & hash tables"
> * Sent on Sun, 24 Mar 2002 10:11:58 +1000

[1]> (setq h (make-hash-table :size 100000)
)
#S(HASH-TABLE EQL)
[2]> (dotimes (i 100000) (setf (gethash i h) i))
NIL
[3]> (setq *print-array* nil)
NIL
[4]> (defun z (n) (let ((h (make-hash-table :size n))) (dotimes (i n) (setf (gethash i h) i))))
Z
[5]> (z 10000)
NIL
[6]> (defun z (n) (let ((h (make-hash-table :size n))) (dotimes (i n h) (setf (gethash i h) i))))
Z
[7]> (z 10000)
#<HASH-TABLE EQL 10000 #x2058F0BD>
[8]> (z 100000)
#<HASH-TABLE EQL 100000 #x20777509>
[9]> (z 100000)
#<HASH-TABLE EQL 100000 #x20960509>
[10]> (z 1000000)
#<HASH-TABLE EQL 1000000 #x21C73209>
[11]> (z 10000000)

*** - Hash table size 10000000 too large
1. Break [12]> 
[13]> (z (ash most-positive-fixnum -1))
#<HASH-TABLE EQL 8388607 #x2BC42001>
[14]> (z (1+ (ash most-positive-fixnum -1)))

1. Break [15]> 

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
MS Windows vs IBM OS/2: Why marketing matters more than technology...

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sun Mar 24 06:23:06 2002

Hello Sam,

>> arises. It's not necessary to fill all such a big HT, usually 5000

[1]>> (setq h (make-hash-table :size 100000)
Sam> )
Sam> #S(HASH-TABLE EQL)
[2]>> (dotimes (i 100000) (setf (gethash i h) i))
Sam> NIL
[3]>> (setq *print-array* nil)
Sam> NIL
[4]>> (defun z (n) (let ((h (make-hash-table :size n))) (dotimes (i n) (setf (gethash i h) i))))
Sam> Z
[5]>> (z 10000)
Sam> NIL
[6]>> (defun z (n) (let ((h (make-hash-table :size n))) (dotimes (i n h) (setf (gethash i h) i))))
Sam> Z
[7]>> (z 10000)
Sam> #<HASH-TABLE EQL 10000 #x2058F0BD>
[8]>> (z 100000)
Sam> #<HASH-TABLE EQL 100000 #x20777509>
[9]>> (z 100000)
Sam> #<HASH-TABLE EQL 100000 #x20960509>
[10]>> (z 1000000)
Sam> #<HASH-TABLE EQL 1000000 #x21C73209>
[11]>> (z 10000000)

Sam> *** - Hash table size 10000000 too large
1. Break [12]>> 
[13]>> (z (ash most-positive-fixnum -1))
Sam> #<HASH-TABLE EQL 8388607 #x2BC42001>
[14]>> (z (1+ (ash most-positive-fixnum -1)))

1. Break [15]>> 

Cool. Now I think that problem sits in GC. When I periodically call GC
in process of filling the HT, stack overflow doesn't appear. (But it
S-L-O-W ! It seems slower than just by GC time, may be CPU cache does
something with this...) GC differs for different OSes ... May be GC
being overloaded by garbage ? What do you think ?

Another problem: when I limited the stack by 'ulimit -s' in FreeBSD,
my endless-recursion test caused coredump instead of detected stack
overflow. Value of counter achieved is comparable hovewer with win32
version (4000 for 3M stack in FreeBSD, 3200 in win NT).

---

From: Sam Steingold <[email-masked]>
Date: Sun Mar 24 11:02:06 2002

> * In message <[email-masked]>
> * On the subject of "Re[4]: [clisp-list] CLISP stack & hash tables"
> * Sent on Mon, 25 Mar 2002 00:22:05 +1000

good. now recompile with -g and run under gdb.
when you get the crash, print the CLISP stack with "p C_show_stack" (or
something :-) and try to figure out what is filling it.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
When you are arguing with an idiot, your opponent is doing the same.

---

From: Peter Burwood <[email-masked]>
Date: Sat Apr 13 05:53:02 2002

Hi,

This is just a FYI email.

Basically, Clisp from CVS compiled with on Cygwin on Win98SE has no problems
with hash tables or Joerg's nines problem.

> o (queens 21.0s0) still crashes.

Works with Cygwin/Win98SE OOTB.

[20]> (nines 21.0s0)

(450 2 450 1 450 4 450 13 9450 22 9450 33 9450 103 198450 195)

> *** - Hash table size 8388608 too large

Cygwin/Win98SE is the same, i.e., works OOTB.

Peter