Subject: Multi-return value functions and with-input-stream-do

From: Oliver Andrich <[email-masked]>
Date: Wed, 15 Feb 95 12:33:30 MET

Hello everybody!

After playing around with CLISP for a while, I think that I have
reached now a point of knowledge to be able to solve some more
ambitious problem than playing around with lists, streams and so on.

I want to develop a program to lemmatise a german text, but that is
not the topic of this mail.

I want that my program has an nice X-Windows userinterface, and
inspired by an article in Linux Journal I thought "Why shouldn't I do
it the same way?". This guy simply opened a pipe to a wish Task (the
Tcl/Tk-interpreter) and send this task via the pipe the required
commands and makes all data processing within a C-program. And I want
to do the same with LISP, yesterday I tried it and all works fine.
That means I opened a pipe with make-pipe-io-stream to the wish task
and I also managed to send all information for displaying the main
window. But then my problems started, I wanted to read something from
the in-stream and parse it, but I got really messed up with all these
input functions provided by LISP. Then I discovered this nice macro
with-open-stream and thought that this macro would solve my problems.
But I doesn't solve anything, cause make-pipe-io-stream returns a
multi-returnvalue. How do I use with-open-stream with make-pipe-io-stream?

Or how do I solve the following task? I want to constantly read from a
pipe similar to read-char-no-hang, but I want to read a whole line and
reading stops when an EOF is received, then all pipes produced by
make-pipe-io-stream should be closed.

I hope, that some could help me.

Bye, Oliver

P.M.: Excuse my bad english, but I have to learn writing english all
over again. I lacked some experience in the past.

---

From: Matthias Lindner <[email-masked]>
Date: Wed, 15 Feb 95 19:30:32 +0100

Oliver Andrich writes:
 > ...

I used the following code in the Plopp! planning system to solve the problem 
you described:

(defvar *WISH-PROG* 	  "wish")
(defvar *WISH-ARGS* 	  (list "-name" "Plopp"))
(defvar *WISH-EXIT-MSG*   "after 500 exit")
(defvar *WISH*            NIL  "The two-way-stream to the wish process")

;;; --------------------------------------------------------------------------
;;; WITH-WISH (macro)
;;; --------------------------------------------------------------------------

(defmacro WITH-WISH (&body body)
  `(let ((*wish* (run-program *wish-prog* 
                              :arguments *wish-args*
			      :input     :stream
			      :output    :stream)))
     (unwind-protect
	 (progn (read-until *wish*)
		,@body)
       (when *wish*
	 (format *wish* "~A~%" *wish-exit-msg*)
	 (force-output *wish*)
	 (setq *wish* NIL)))))

;;; --------------------------------------------------------------------------
;;; READ-UNTIL (function)
;;; --------------------------------------------------------------------------

(defun READ-UNTIL (stream &optional (prompt ""))
  (flet ((read-to-end ()
           (do ((s nil (cons (read-char-no-hang stream NIL NIL) s)))
               ((and (consp s) (null (car s)))
                (coerce (nreverse (cdr s)) 'string)))))
    (do* ((lp  (length prompt))
          (str (read-to-end) (concatenate 'string str (read-to-end)))
          (ls  (length str)  (length str)))
        ((or (zerop lp)
             (and (>= ls lp) (string= prompt (subseq str (- ls lp)))))
         (subseq str 0 (- ls lp))))))

Within the body of WITH-WISH you can communicate with the tcl/tk process
writing to *wish* and reading from it. READ-UNTIL reads from a stream 
as much as possible or until a prompt is found. (the implementation is weird 
and could be much simpler, as wishes do not send prompts to pipes - anyway it
works ... at least for me ;) )
Remember to call "(force-output *wish*)" on the CLisp side and "flush stdout"
on the wish side after each output operation.

 > I hope, that some could help me.

Hope it helped.

--Matthias

------------------------------------------------------------------------------
Matthias Lindner
FG Intellektik, FB Informatik
Technische Hochschule Darmstadt
Alexanderstr.10, D-64283 Darmstadt
TEL: +49 6151 166651
FAX: +49 6151 165326
NET: [email-masked]
WWW: http://aida.intellektik.informatik.th-darmstadt.de/~matthias/
------------------------------------------------------------------------------

---

From: [email-masked] (Peter Dudey Drake)
Date: Wed, 15 Feb 95 11:02:49 PST

I've been using a cheap hack wherein I launch Lisp from a wish
process, set up a read-eval-print loop there, and have Tcl read and
write to the Lisp process.  (This requires two small files, below.)
Your method sounds more elegant, but I don't really know anything
about pipes.  Tell us if you figure it out...

To use my stuff, start wish and do like this:

% source join.tcl
% start_lisp
Lisp process started
Loading join.lisp
Starting event loop
Lisp ready
% lisp_eval "(+ 2 2)"
4
% lisp_eval "(defun foo (x) (+ x x))"
FOO
% lisp_eval "(foo 3)"
6
% lisp_eval "(quit)"
% exit

The tcl_to_lisp and lisp_to_tcl procs are quite nice, but be warned
that you may sometimes have to add or remove a set of braces to get
the right list in and out of tcl.

Enjoy!

+-     PETER DUDEY DRAKE      279-D SE Lilly Ave.,  Corvallis, OR  97333     -+
|       MS student in Artificial Intelligence, Oregon State University        |
|                   Where I come from, we're ALL like this.                   |
+-  [email-masked]        (finger [email-masked])  -+

Here's my Tcl file, join.tcl.  It assumes that you have a symbolic
link called "lisp" in the current directory that runs your Lisp.

proc start_lisp {} { 
  global Lisp_pipe
  set Lisp_pipe [open {|lisp} r+ ]
  after 2000
  puts "Lisp process started"
  write_lisp "(load \"join.lisp\")"
  puts "Loading join.lisp"
  read_till_prompt
  write_lisp "(talk-to-tcl)"
  puts "Starting event loop"
  read_till_prompt
  puts "Lisp ready"
}

proc write_lisp {string} {
  global Lisp_pipe

  puts $Lisp_pipe $string
  flush $Lisp_pipe
}

proc read_lisp {} {
  global Lisp_pipe

  return [gets $Lisp_pipe]
}

proc read_till_prompt {} {
  global Lisp_pipe
  set text {}
  while {[string compare [string range $text 0 4] READY]} {
	gets $Lisp_pipe text
#       Uncomment the next line for debugging
#	puts $text
		}
}

proc lisp_eval {string} {
#    puts "Evaluating $string"
    write_lisp $string
    if {$string != "(quit)"} {
	set result [read_lisp]
#	puts $result
	read_till_prompt
	return $result
    }
}

proc tcl_to_lisp {list} {
    ## Replaces braces with parentheses
    regsub -all "\{" [list $list] "\(" result
    regsub -all "\}" $result "\)" result
    return $result
}

proc lisp_to_tcl {list} {
    ## Replaces parentheses with braces
    regsub -all {\(} $list "\{" result
    regsub -all {\)} $result "\}" result
    return [lindex $result 0]
}

And here's join.lisp:

(format t "~%READY~%")
(force-output t)

(defun talk-to-tcl ()
 (format t "~%READY~%")
 (force-output t)
  (do ((text (read) (read)))
      (())
      (format t "~a~%READY~%" (write-to-string (eval text)))))

---

From: [email-masked] (Peter Dudey Drake)
Date: Wed, 15 Feb 95 11:04:32 PST

I tried editing compiler.lsp and init.lsp and recompiling them, but
that didn't seem to do the trick.  I'm thinking it may require
re-building the memory image, and wasn't that pre-built?

Thanks in advance,

---

From: "Harvey J. Stein" <[email-masked]>
Date: Wed, 15 Feb 1995 23:21:27 +0200

Of course, another solution to the problem of communication between a
lisp process & a wish process is to just use STk instead of clisp.

For those of you aren't familiar with it, STk is a scheme interpreter
with Tk compiled in.  It allows you to write scheme scripts to access
Tk for windowing instead of having to write Tcl scripts.  For example,
I wrote a version of tetris in it.

Harvey J. Stein
[email-masked]

---

From: Oliver Andrich <[email-masked]>
Date: Thu, 16 Feb 95 8:56:49 MET

Hi Harvey!

> I wrote a version of tetris in it.

I tried STk too. But I found that it doesn't have the nice features of
Common Lisp and needs as much resources as CLISP and a wish-task. I
like all powerful features Common Lisp provides. :-)

And STk is slow in comparison to CLISP, and even slower in comparison
to compiled CLISP-programs.

Bye, Oliver

---

From: Oliver Andrich <[email-masked]>
Date: Thu, 16 Feb 95 9:07:25 MET

Hello Peter!

> write to the Lisp process.  (This requires two small files, below.)

Hm... I think your method is rather easy, but I want hte Lisp program
to have the total control, that means Tcl/Tk is only used to do the
displaying for my lisp program.

By the way, can you use the whole power of lisp when using your
method? As I understand you write a Tcl/Tk program that calls Lisp to do
some nice things, that means you have to write program in Tcl/Tk doing
the displaying and you have to write a program in Lisp to do the rest.
Is that right? If that is the fact, then I no that this isn't the way
I want to do it. I want to write a Lisp program that simply sends
messages to the wish-Task, so that I have to learn as little as
possible of Tcl/Tk. (It's hard enough to fight Lisp. :-)

> about pipes.  Tell us if you figure it out...

Ok, I will post my results. I also thought about something like
developing a CLOS classlibrary for this stuff. But first I have to do
it in normal LISP.

Bye Oliver

---

From: Oliver Andrich <[email-masked]>
Date: Thu, 16 Feb 95 9:16:48 MET

Hello Matthias! 

> I used the following code in the Plopp! planning system to solve the problem 

[code]

Well, thanks a lot, cause your code seems to do everything I want.

By the way, what is Plopp!. Can you send me some information?

> works ... at least for me ;) )

Ok, that really seams to be the thing I want to do.

> on the wish side after each output operation.

Yeah, after fighting with Tcl/Tk and Lisp for a whole night, I learned
my lesson. :-) 

Bye, Oliver

---

From: Matthias Lindner <[email-masked]>
Date: Thu, 16 Feb 95 12:31:48 +0100

> By the way, what is Plopp!. Can you send me some information?

Plopp! is a prototype implementation of an assumption-based assistance planner.
It's an essential part of my dissertation. I've implemented it in Common Lisp
and interfaced it with a Tcl/Tk GUI. My WWW-homepage (address below) contains
hints to some papers about this beast and a pointer to the place where
you can get it by ftp.

Have fun

--Matthias

---

From: [email-masked] (Peter Dudey Drake)
Date: Thu, 16 Feb 95 09:12:16 PST

From: Oliver Andrich <[email-masked]>

I believe so.  Whatever Tcl sends to the Lisp process is sent through
EVAL.

There are three limitations that I know of to the technique I'm using:

- If there's an error on the Lisp end, the whole deal dies hideously.
This could probably be fixed from Lisp by altering the error behavior.

- Wish assumes there will be a fairly quick response.  I'm not sure
what happens if it tries to read from the Lisp process when nothing
has been returned yet.

- Wish reads only the first line that gets printed out as the return
value, although this may be arbitrarily long.  One should watch out
for pretty printing or shorthand (e.g., # for a deep sublist).

Well, this should work fine, if your task is event-driven.  (A
reasonable presumption if you're using Tcl/Tk.)  Every time an event
goes off, Tcl asks Lisp what to do with it, and Lisp replies.

---

From: "Harvey J. Stein" <[email-masked]>
Date: Thu, 16 Feb 1995 21:59:57 +0200

> like all powerful features Common Lisp provides. :-)

Of course, if you need the power, you're better off using Common Lisp
than re-implementing everything in scheme.  On the other hand, if you
don't, it's nice to have the smaller footprint.  However, the internet
Lisp archives are quite a bit more extensive than the scheme archives.
On the other hand, for basic added functionality, there's SLIB.

Perhaps you'd like winterp?  It's a lisp interface to Motif.  On the
plus side, it's lisp.  On the minus side it's Motif :).  The other
minus is that it's not common lisp, it's only xlisp :(.

It would be great if someone would develop a Tk interface to clisp.
Maybe someone can yank the Tk interface out of STk & adapt it to
clisp.  Or, perhaps if we wait for guile (the FSF's entry in the GUI
scripting wars), we can either use it, or rip out the Tk interface for
clisp.  The latter might be cleaner than using STk's interface,
because the FSF is presumably cleaning up the interface to make it
less Tcl like.

 > to compiled CLISP-programs.

I didn't find that...  But I didn't do extensive testing.  I tried
doing 1000 factorial in each and found STk to beat out clisp slightly
(even when factorial was compiled).

Harvey J. Stein
[email-masked]

---

From: Oliver Andrich <[email-masked]>
Date: Fri, 17 Feb 95 9:30:32 MET

> This could probably be fixed from Lisp by altering the error behavior.

Hm.... I think that that is a problem, I won't experience. Cause Lisp
parses every output from Tcl/Tk. 

Bye, Oliver

---

From: Oliver Andrich <[email-masked]>
Date: Fri, 17 Feb 95 9:40:59 MET

> On the other hand, for basic added functionality, there's SLIB.

I think that the SLIB makes Scheme really useful, cause the SLIB
provides a lot of nice features.

> minus is that it's not common lisp, it's only xlisp :(.

Well, I don't use Motif, cause I am using Linux on a machine with a
14" monitor. And because of the monitor I am using X-Windows not so
often. I only use it for tasks, where I need it. Something like doing
LaTeX. 

> because the FSF is presumably cleaning up the interface to make it

I think that this would be quite nice.

> (even when factorial was compiled).

Well, I used the gabriel-benchmarks, and then I found clisp to be
faster as STk.

Bye, Oliver