Subject: [clisp-list] Relatively new to Clisp

From: Jeremy Doolin <[email-masked]>
Date: Wed Feb 27 16:04:04 2002

Well, hello to all on this list.

I've been messing around with writing a shell in Common Lisp using
CLISP.  So far I've managed to make pretty good progress. Most of what
I've done is basic string processing.  So here is what my shell can do:

-execute commands, change directory, variables, environment variables,
aliases, configurable prompt, evaluate S-expressions and probably a few
minor things I can't think of right now.  

And here is what I'd like it to do:

-pipes, I/O redirection, a complete syntax (it will not be bash syntax),
and (the hardest part) job control.

Here is what is wrong with it so far:

I am using the (execute ..) function for executing commands.  Regardless
if I use this or (run-program ..), my shell is dependent upon another
shell.  This is rather... disgusting.  Now, I'm well aware that Lisp is
not an OS level language.  But I've noticed that (cd ..) is implemented
in CLISP.  I can think of no other way this could be implemented than a
system call to 'chdir'.  So perhaps an 'exec' type function could be
implemented in the same way.  A 'fork' command would also be helpful.  

Another problem is when I give the shell an S-expression with an error.
For example, (car "foo").  If this happens, the entire shell exits. 
That's inconvenient.  I suppose I could disallow s-expressions, but can
you imagine how cool it would be to something like:

(+ 2 $FOO)  in the shell?  Obviously, expressions like this would
require me to make my own syntax.  

So here is what I'm asking: 

First of all, I'd be perfectly willing to implement an exec command in
CLISP.  However, I need to know how 'cd' and 'getenv' were done (and
where in the source tree).  Unless there is another way to do this, of
course.  

Second, I'm looking for advice with any of the other problems/issues
I've described.  (like pipes and the bad S-expression problem).  

Also, keep in mind that I've not been programming in Lisp for long.  So
forgive any obvious ignorance that I've exhibited.  I'm perfectly
willing to take criticism for things I do wrong, just don't point out
"how stupid he is for doing something like that".  The reason I say this
is that I've had that experience before.  

Thanks!

Jeremy Dolin

-- 
May all your disgraces be private

---

From: Peter Wood <[email-masked]>
Date: Thu Feb 28 00:04:02 2002

Hi,

> I've done is basic string processing.  So here is what my shell can do:

Cool.  I have been playing around with something similar.

> implemented in the same way.  A 'fork' command would also be helpful.  

You can do these things with the libc bindings.  You don't mention
what your OS is, but if you're using Linux or AmigaOs (!) bindings are
supplied.  The problem with exec is you need a C definition with the
correct number of args.  linux.lisp includes 3 execlp definitions, so
you would be limited to a small number of args.  I 'worked around this'
by defining a larger number of execlp's in linux.lisp, and on the lisp
side have a macro construct the call to the correct execlp.  It's ugly
and primitive, and your number of args is still limited, so it's
pretty unsatisfactory, even though it works.

For some reason, wait() and waitpid() are not included in linux.lisp.
I asked about this before on clisp-list, but nobody replied.  I don't
think fork() is useable without wait() so it's a bit strange.  I added

(def-c-call-out wait
  (:arguments
   (empty (c-ptr int)))
  (:return-type pid_t))

and it seems to work ok.  I also added a definition for waitpid(), but
I think it will need a wrapper.

> require me to make my own syntax.  

I take it your shell is a program Clisp is running?  My approach
was/is to use Clisp directly, and extend it to provide the needed
functionality of a shell.  See http://clisp.sourceforge.net/clash.html

I abandoned the idea of a read-macro for entering commands, since
running programs from execlp is simple enough to not need to pamper
the user.  You can also define readline functions that insert text
'invisibly', so you can provide the user with the illusion that your
shell has two seperate syntaxes.  As an example, I have a readline
function 'clash-run-wait' which wraps whatever the user has typed in
with (runw "....")\n. Where the dots are replaced with the user's
input.  #'runw calls a macro which builds the call to the appropriate
execlp call in the linux package.  It forks and waits.  The upshot is
you can run programs using an identical syntax to Bash, from the
user's point of view.  

> where in the source tree).  Unless there is another way to do this, of

getenv is in misc.d.  But I would advise you to try the libc bindings
first.  Good luck :-)

> I've described.  (like pipes and the bad S-expression problem).  

Clisp has pipes, but they use the shell (I think).  See,
impnotes.html#shell.
 
Have fun.

Regards,
Peter

---

From: Sam Steingold <[email-masked]>
Date: Thu Feb 28 10:50:08 2002

> * In message <1014854608.32240.95.camel@zeus>
> * On the subject of "[clisp-list] Relatively new to Clisp"
> * Sent on 27 Feb 2002 19:03:28 -0500
> * Honorable Jeremy Doolin <[email-masked]> writes:
>
> I am using the (execute ..) function for executing commands.
> Regardless if I use this or (run-program ..), my shell is dependent
> upon another shell.

this is not true.
EXECUTE on UNIX calls vfork/execv/wait2

> For example, (car "foo").  If this happens, the entire shell exits.

maybe you should handle the errors in your shell?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
I'd give my right arm to be ambidextrous.

---

From: Sam Steingold <[email-masked]>
Date: Thu Feb 28 11:03:07 2002

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] Relatively new to Clisp"
> * Sent on Thu, 28 Feb 2002 08:46:17 +0100
> * Honorable Peter Wood <[email-masked]> writes:
>

try the appended patch (with the CVS CLISP) and see wrap.lisp for usage
of the :out args.
if it does what you want, I will commit the patch.

> Clisp has pipes, but they use the shell (I think).

shell is cheap (100k /bin/ash).
i/o redirection is not quite trivial.
I do not want to re-implement BASH in the CLISP's C substrate.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
Trespassers will be shot.  Survivors will be SHOT AGAIN!

Index: linux.lisp
===================================================================
RCS file: /cvsroot/clisp/clisp/modules/bindings/linuxlibc6/linux.lisp,v
retrieving revision 1.5
diff -u -w -b -u -b -w -i -B -r1.5 linux.lisp
--- linux.lisp	28 Feb 2002 18:36:53 -0000	1.5
+++ linux.lisp	28 Feb 2002 18:50:59 -0000
@@ -1822,6 +1822,18 @@
                       (:return-type pid_t)
 )
 
+(c-lines "#include <sys/wait.h>~%")
+
+(def-c-call-out wait
+    (:arguments (status (c-ptr int) :out :alloca))
+  (:return-type pid_t))
+
+(def-c-call-out waitpid
+    (:arguments (pid pid_t)
+                (status (c-ptr int) :out :alloca)
+                (options int))
+  (:return-type pid_t))
+
 (def-c-call-out ttyname (:arguments (fd int))
                         (:return-type c-string)
 )

---

From: Peter Wood <[email-masked]>
Date: Thu Feb 28 23:22:05 2002

Hi,

Thanks.

> I do not want to re-implement BASH in the CLISP's C substrate.

I understand that.

Regards,
Peter

---

From: Sam Steingold <[email-masked]>
Date: Fri Mar 1 08:10:10 2002

> * In message <[email-masked]>
> * Sent on Fri, 1 Mar 2002 08:02:57 +0100

do you mean that the patch works?

> > I do not want to re-implement BASH in the CLISP's C substrate.

BTW, how is "&" implemented in a shell?
when I do vfork/exec/wait, I am blocked till the sub-process is done.
do I just drop wait(2)?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
My inferiority complex is not as good as yours.

---

From: Randall R Schulz <[email-masked]>
Date: Fri Mar 1 08:27:15 2002

Sam,

At its essence, the shells' detachment operator ('&') does indeed amount to 
just deferring the wait(2) call.

However, that call cannot be omitted entirely. Processes are not fully gone 
until they've been wait()-ed for. If their immediate parent exits without 
wait()-ing, then the system's "init" process (process ID 1) inherits and 
cleans up after these processes.

If your process is long-running, then you must enable (catch) the SIGCLD 
signal so you know when a child process's state changes. Unless you're 
using the kernel debugging interfaces, "state change" just means process 
termination. When you get a SIGCLD you then issue the wait call, which is 
guaranteed in that circumstance not to block (assuming the SIGCLD is not 
fictitious). With the return value from wait(2) (which includes the process 
ID and information about how it terminated) you can do what ever internal 
bookkeeping is necessary to clean up after and report the termination of 
the sub-process.

Randall Schulz
Mountain View, CA USA

At 08:05 2002-03-01, Sam Steingold wrote:
>...
>
>BTW, how is "&" implemented in a shell? when I do vfork/exec/wait, I am 
>blocked till the sub-process is done. do I just drop wait(2)?
>
>--
>Sam Steingold (http://www.podval.org/~sds)

---

From: Peter Wood <[email-masked]>
Date: Fri Mar 1 23:51:02 2002

> do you mean that the patch works?

I have now tested it as far as possible, and I think it works fine.
My understanding is there is a whole bunch of __W* macros in
bits/waitstatus.h and W* macros in sys/wait.h which are needed for
accessing the status value, so I have not been able to check that part
of it yet.  That is, I can't get information about the child's exist
status.

Thanks again.

Peter

ps.  I will have a go at doing ffi definitions for the W* macros, but
it won't be soon.

---

From: Peter Wood <[email-masked]>
Date: Sat Mar 2 04:44:21 2002

Hi,

There is not actually much to the  W* macros.  So I'm not sure its
worth adding them to  linux.lisp.

This works very nicely:

(defun test-wait2 ()
  (let ((pid (linux:fork)))
    (cond ((= pid -1) (error "fork failed"))
	  ((/= pid 0) (multiple-value-bind (childpid status)
			 (linux:wait)
			   (progn (format t "Child pid=~a has finished~%" childpid)
				  (if (zerop (logand status #x7f))
				      (format t "...with exit code=~a~%" 
					      (ash (logand status #xff00) -8))
				    (format t "Child bombed~%")))))
	  (t (progn 
	       (format t "~&Hello, Pop~%")
	       (linux:exit 37))))))

[1](test-wait2)

Hello, pop
Child pid=3048 has finished
...with exit code=37
NIL
[2]

Regards,
Peter

---

From: Sam Steingold <[email-masked]>
Date: Sat Mar 2 12:45:12 2002

> * In message <[email-masked]>
> * Sent on Sat, 2 Mar 2002 13:28:45 +0100

I did anyway.

(defun mesg (&rest fmt-args)
  (format t "~&[~d] " (linux:getpid))
  (apply #'format t fmt-args))
(defun test-wait2 ()
  (let ((pid (linux:fork)))
    (cond ((= pid -1) (error "fork failed"))
	  ((/= pid 0)
           (mesg "Started child ~d~%" pid)
           (multiple-value-bind (childpid status) (linux:wait)
             (mesg "Child ~d has finished~%" childpid)
             (if (zerop (logand status #x7f))
                 (mesg "with exit code=~a~%" (linux:WEXITSTATUS status))
                 (mesg "Child bombed ~d / ~d~%" (linux:WTERMSIG status)
                       (linux:WEXITSTATUS status)))))
	  (t (sleep 1)          ; let dad go first
             (mesg "Hello, Pop (~d), I am ~d~%"
                   (linux:getppid) (linux:getpid))
             (linux:exit 37)))))
(test-wait2)

[17857] Started child 17858
[17858] Hello, Pop (17857), I am 17858
[17857] Child 17858 has finished
[17857] with exit code=37

thanks!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
If Perl is the solution, you're solving the wrong problem. - Erik Naggum