Subject: [clisp-list] How to get the call stack?

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Fri Oct 4 08:24:05 2002

John K. Hinsdale wrote:
>I would still like to take snapshot of the stack, _before_ it's
>unwound, and include as part of my message.  So I need to somehow
>override (or at least have some hook into) the built-in Lisp error
>thrower, if that makes any sense.  How do I do that?

That's the difference between HANDLER-BIND and HANDLER-CASE.

Regards,
	Jorg Hohle.

---

From: "John K. Hinsdale" <[email-masked]>
Date: Fri Oct 4 16:10:01 2002

> That's the difference between HANDLER-BIND and HANDLER-CASE.

Got it, thanks.  just what I wanted.  My call stack is now getting
(approximately) displayed, so I won't go back to perl or Java :)

---

From: Kaz Kylheku <[email-masked]>
Date: Fri Oct 4 16:27:01 2002

On Fri, 4 Oct 2002, John K. Hinsdale wrote:

> Date: Fri, 4 Oct 2002 19:09:10 -0400
> From: John K. Hinsdale <[email-masked]>
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [clisp-list] How to get the call stack?
> 
> 

It was said by Kent Pitman that HANDLER-CASE is a construct for people
coming from languages in which the only thing that you can do with an
exception is to contain it. ;)

You have RESTART-CASE, RESTART-BIND, HANDLER-CASE and HANDLER-BIND.
The two that you usually want are RESTART-CASE and HANDLER-BIND.

The reason for that is that the selected restart site is where
unwinding ultimately happens. Consequently, most restarts want to
perform a non-local exit to the restart point, thus the case construct
is most convenient. With restart-bind, the restart function would have
to set up a non-local exit. Or it could even return back to the error
handler, which would probably be confused as heck, not expecting the
evaluation of (invoke-restart ...) to return!

Whereas at the condition handling site, you don't want to unwind right
away, because you want to scan the dynamic context for available
restarts, which might be established at a or lower level of nesting
than the handler.  Unwinding is explicitly initiated by invoking a
restart in one of various ways. Moreover, it makes perfect sense for a
handler function to just return; by doing so it declines to handle
the error, creating an opportunity for the error search to continue,
without any unwinding having taken place which would throw away useful
information.

With HANDLER-CASE, you completely ignore restarts; in effect you
provide your own restart in the form of the implicit non-local exit.

---

From: Kaz Kylheku <[email-masked]>
Date: Wed Feb 12 16:27:01 2003

The problem with this is that normally, when you run programs from the
shell, they get SIG_DFL, which has different behavior from SIG_IGN.
The exec() system call causes SIG_IGN and SIG_DFL settings to be
inherited by the new process image. 

On Linux, signal(SIGCHLD, SIG_IGN) seems to cause the waitpid()
semantics described by the Single UNIX Specification. If the processes
terminate before the waitpid() call is reached, they are reaped
internally without becoming zombies. Then the waitpid() returns -1 with
ECHILD. If the waitpid is reached while the child is still running,
then waitpid() waits normally and returns the pid.

This is a problem, because lots of programs---CVS being one
example---are not written to defend against this obscure behavior and so
the -1/ECHILD takes them by surprise; they just expect SIGCHLD to be
SIG_DFL on startup. 

Sample run:

[4]> (defvar *x* (ext::make-pipe-input-stream "./show-sig"))
*X*
[5]> (read-line *x*)
"SIGCHLD == SIG_IGN"

The ext::run-program function is not affected (on the system I'm
using), because it is based on the system() function which indirects
through the shell. The shell is probably written defensively against
inheriting bad signal settings, so everything is cool.

/* showsig source code */
#include <signal.h>
#include <stdio.h>

int main(void)
{
    void (*existing)(int) = signal(SIGCHLD, SIG_DFL);

    if (existing == SIG_DFL)
	puts("SIGCHLD == SIG_DFL");
    else if (existing == SIG_IGN)
	puts("SIGCHLD == SIG_IGN");
    else
	puts("SIGCHLD == <handled>");
   
    return 0;
}

Here is a program that on Linux 2.2.22 shows the unreliable behavior.
If you remove the sleep() call, it works every time I try it: waitpid
returns the pid. If you put back the sleep() call, you get -1 return,
with errno being 10 (ECHILD) every time. Then if you set SIGCHLD to
SIG_DFL, that goes away, and waitpid returns positive again.

#include <signal.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <sys/types.h>

int main(void)
{
   pid_t pid = fork();

   /* Empty bowels before fork() to prevent double vision. */
   fflush(NULL); 

   signal(SIGCHLD, SIG_IGN);

   if (pid == 0) {
      char *argv[2] = { "show-sig", 0 };
      execvp("./show-sig", argv);
      printf("exec failed\n");
      exit(0);
   } else if (pid > 0) {
      int status;
      int r;
      sleep(1);
      r = waitpid(pid, &status, 0);
      printf("waitpid returned %d, errno == %d\n", r, errno);
   } else {
      puts("fork failed");    
   }

   return 0;
}

---

From: Sam Steingold <[email-masked]>
Date: Wed Feb 12 20:11:04 2003

> * In message <[email-masked]>
> * On the subject of "[clisp-list] Re: Issue: SIG_IGN setting for SIGCHLD inherited by children of CLISP."
> * Sent on Wed, 12 Feb 2003 16:24:59 -0800 (PST)
> * Honorable Kaz Kylheku <[email-masked]> writes:
>

note that EXECUTE does not suffer from this problem, only pipe
functions do.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
What garlic is to food, insanity is to art.

--- stream.d.~1.339.~	2003-01-29 14:36:06.000000000 -0500
+++ stream.d	2003-02-12 23:08:29.000000000 -0500
@@ -13467,9 +13473,11 @@
     var DYNAMIC_ARRAY(command_data,char,command_length);
     begin_system_call();
     memcpy(command_data,command,command_length);
+    begin_want_sigcld();
     # build pipe:
     if (!( pipe(handles) ==0)) {
-      FREE_DYNAMIC_ARRAY(command_data); OS_error();
+      FREE_DYNAMIC_ARRAY(command_data);
+      end_want_sigcld(); OS_error();
     }
     # Everything, that is stuffed in handles[1], resurfaces at handles[0]
     # again. We will utilize this as follows:
@@ -13495,6 +13503,7 @@
       _exit(-1); # if this fails, finish child-process
     }
     # This piece of code is again executed by the caller:
+    end_want_sigcld();
     if (child==-1)
       # Something failed, either on vfork or on execl.
       # In both cases errno was set.
@@ -13678,8 +13687,10 @@
     var DYNAMIC_ARRAY(command_data,char,command_length);
     begin_system_call();
     memcpy(command_data,command,command_length);
+    begin_want_sigcld();
     if (!( pipe(handles) ==0)) {
-      FREE_DYNAMIC_ARRAY(command_data); OS_error();
+      FREE_DYNAMIC_ARRAY(command_data);
+      end_want_sigcld(); OS_error();
     }
     # Everything, that is stuffed in handles[1], resurfaces at handles[0]
     # again. We will utilize this as follows:
@@ -13705,6 +13716,7 @@
       _exit(-1); # if this fails, finish child-process
     }
     # This piece of code is again executed by the caller:
+    end_want_sigcld();
     if (child==-1)
       # Something failed, either on vfork or on execl.
       # In both cases errno was set.

---

From: Sam Steingold <[email-masked]>
Date: Wed Feb 12 20:13:05 2003

note that EXECUTE does not suffer from this problem, only pipe
functions do.
please try the appended patch.

--- stream.d.~1.339.~	2003-01-29 14:36:06.000000000 -0500
+++ stream.d	2003-02-12 23:11:29.000000000 -0500
@@ -13467,9 +13473,11 @@
     var DYNAMIC_ARRAY(command_data,char,command_length);
     begin_system_call();
     memcpy(command_data,command,command_length);
+    begin_want_sigcld();
     # build pipe:
     if (!( pipe(handles) ==0)) {
-      FREE_DYNAMIC_ARRAY(command_data); OS_error();
+      FREE_DYNAMIC_ARRAY(command_data);
+      end_want_sigcld(); OS_error();
     }
     # Everything, that is stuffed in handles[1], resurfaces at handles[0]
     # again. We will utilize this as follows:
@@ -13495,6 +13503,7 @@
       _exit(-1); # if this fails, finish child-process
     }
     # This piece of code is again executed by the caller:
+    end_want_sigcld();
     if (child==-1)
       # Something failed, either on vfork or on execl.
       # In both cases errno was set.
@@ -13678,8 +13687,10 @@
     var DYNAMIC_ARRAY(command_data,char,command_length);
     begin_system_call();
     memcpy(command_data,command,command_length);
+    begin_want_sigcld();
     if (!( pipe(handles) ==0)) {
-      FREE_DYNAMIC_ARRAY(command_data); OS_error();
+      FREE_DYNAMIC_ARRAY(command_data);
+      end_want_sigcld(); OS_error();
     }
     # Everything, that is stuffed in handles[1], resurfaces at handles[0]
     # again. We will utilize this as follows:
@@ -13705,6 +13716,7 @@
       _exit(-1); # if this fails, finish child-process
     }
     # This piece of code is again executed by the caller:
+    end_want_sigcld();
     if (child==-1)
       # Something failed, either on vfork or on execl.
       # In both cases errno was set.
@@ -13830,11 +13842,14 @@
     var DYNAMIC_ARRAY(command_data,char,command_length);
     begin_system_call();
     memcpy(command_data,command,command_length);
+    begin_want_sigcld();
     # build Pipes:
     if (!( pipe(in_handles) ==0)) {
-      FREE_DYNAMIC_ARRAY(command_data); OS_error();
+      FREE_DYNAMIC_ARRAY(command_data);
+      end_want_sigcld(); OS_error();
     }
     if (!( pipe(out_handles) ==0))
+      end_want_sigcld();
       OS_error_saving_errno({
         CLOSE(in_handles[1]); CLOSE(in_handles[0]);
         FREE_DYNAMIC_ARRAY(command_data);
@@ -13873,6 +13888,7 @@
       _exit(-1); # if this fails, finish child-process
     }
     # This piece of code is again executed by the caller:
+    end_want_sigcld();
     if (child==-1)
       # Something failed, either on vfork or on execl.
       # In both cases errno was set.

---

From: Kaz Kylheku <[email-masked]>
Date: Fri Mar 7 20:48:03 2003

I've known about this issue quite some time. It's either a bug,
or there is some subtlety about backquotes that I don't understand.

;;; expected result:  (FOO `(BAR (BAZ 'A A) (BAZ 'B B) (BAZ 'C C)))
;;; CLISP result:     (FOO `(BAR NIL NIL NIL NIL))

(defun breaks-on-clisp ()
  (let ((list '(a b c d)))
    `(foo `(bar ,@',(mapcar #'(lambda (sym)
				`(baz ',sym ,sym))
			    list)))))

;;; moving (mapcar ...) to a function fixes it:

(defun factored-out (list)
  (mapcar #'(lambda (sym) `(baz ',sym ,sym))
	  list))

(defun does-not-break ()
  (let ((list '(a b c d)))
    `(foo `(bar ,@',(factored-out list)))))

It looks like backquote is confused; it's doing something wrong
with the `(baz ',sym ,sym) even though that occurs in a doubly-unquoted
context. You can see that MAPCAR iterates the right number of times,
but the lambda is returning NIL for some bizarre reason.

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Mar 8 19:02:04 2003

> or there is some subtlety about backquotes that I don't understand.

I have more information that may be useful. Firstly, here is a smaller
piece of code that reproduces the problem:

 (let ((x 3)) ``,,`,x) 

 ==> `,NIL

Secondly, I have a suspicion that this has to do with the third
parameter of the SYSTEM::BACKQUOTE form being deleted.

Check this out; suppose we manually construct a backquote form like
this:

  (list 'system::backquote (list 'system::unquote 3))

  ==> `,3

It looks good when printed, right? But a third element is missing that
is needed by the system::backquote evaluator, but not the
pretty-printer, so when we do this:

 (eval (list 'system::backquote (list 'system::unquote 3)))

 ==> NIL

Ooops, the value should be 3! Now let's put in that missing third
element, but let's lie with a wrong value just for dramatic effect:

 (eval (list 'system::backquote 
             (list 'system::unquote 3)
             42))

 ==> 42

Clearly, the value comes from the third element.

My suspicion is that when a form like 

 (let ((x 3)) ``,,`,x) 

is processed, the inner-most `,x expression is somehow translated into
a SYSTEM::BACKQUOTE form with a missing third element. 

In the backquote.lisp implementation file there is a remove-backquote-third
function which does the job of recursively stripping these third elements;
perhaps it is involved in some way, such as being inappropriately applied
or whatever.

Hope this helps. I'm trying to debug this myself, but I can't read
German, so I'm slow! ;)

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Mar 8 19:28:04 2003

> piece of code that reproduces the problem:

Even smaller, doh:

 ``,,`,3

 ==> `,NIL

Now for some really strange stuff: what is hiding behind the above
`,NIL form? Here is a trick: if we coerce it to a vector, the guts
become visible:

  (coerce ``,,`,3 'vector)

  ==> #(SYSTEM::BACKQUOTE (SYSTEM::UNQUOTE NIL) 3)

The third element *is* there, but the unquote form is screwed! So this
actually works:

  (eval ``,,`,3)
  ==> 3

Whee! (eval `,NIL) is 3.

---

From: Kaz Kylheku <[email-masked]>
Date: Sun Mar 9 12:32:10 2003

> parameter of the SYSTEM::BACKQUOTE form being deleted.

Indeed this is the problem, witness:

Bug, where is the (LIST X)?

    ``,,(coerce '`(,x) 'vector))

    ==> `,#(SYSTEM::BACKQUOTE ((SYSTEM::UNQUOTE X)))

Normal behavior, third element is there:

    (coerce '`(,x) 'vector) 

    ==> #(SYSTEM::BACKQUOTE ((SYSTEM::UNQUOTE X)) (LIST X))

So it looks like there are at least two bugs: in some cases the third
element of the SYSTEM::BACKUOTE form is being eaten, and in some other
cases, the second element is mangled to contain a NIL:

    (coerce ``,,`,3 'vector)  ; note: backquote is evaluated 

This second bug is not quite as serious; it must means that the printed
representation of the backquote is wrong, but it evaluates right.  But
obviously if you are spitting data out to text and expecting to read it
back, and it contains a backquote that was wrongly printed as ',NIL
rather than ',<whatever>, that is a problem.

---

From: Sam Steingold <[email-masked]>
Date: Sun Mar 9 14:44:05 2003

> * In message <[email-masked]>
> * On the subject of "Re: Strange backquote bug, or so I think."
> * Sent on Sat, 8 Mar 2003 19:00:47 -0800 (PST)
> I'm trying to debug this myself, but I can't read German, so I'm slow!
> ;)

Mirian Lennox <[email-masked]> translated the comments in
backquote.lisp on 2003-01-18.
thus, the current CVS head has comments in English.

Good luck!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Man has 2 states: hungry/angry and sate/sleepy.  Catch him in transition.

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Mar 15 01:47:04 2003

I have reimplemented backquotes in CLISP from scratch; the work is
advanced far enough that I can rebuild the patched CLISP and pass
``make test''.

The aim is to produce a smaller, simpler, more elegant implementation
that has zero defects.

A few visible differences are introduced.

Firstly, the reader macros no longer perform expansion. Consequently,
the SYSTEM::BACKQUOTE macro does not have a second argument that
specifies the expanded form.  The job of expansion, rather, is moved
into the macro itself.  So we now have a thick macro, but a thin reader
macro rather than the other way around.

The structure of the SYSTEM::BACKQUOTE forms has changed in another
way. The ,@FORM syntax corresponds to (SPLICE FORM) rather than (SPLICE
(UNQUOTE FORM)). The special case (SPLICE FORM) to represent ,@'FORM is
gone, this is just (SPLICE (QUOTE FORM)).  In other words, this is more
like the Scheme scheme.  Of course, I adjusted the pretty printer
accordingly by removing code that parses the two cases of the SPLICE
argument.

Any code that depends on CLISP's representation of ,@ will break.

The SYSTEM::ADD-BACKQUOTE and SYSTEM::ADD-UNQUOTE interfaces continue
to work. CLISP uses these internally, for instance in DEFSTRUCT,
to generate backquotes.

I removed the dependency from the array and struct readers on the
SYSTEM::*BACKQUOTE-LEVEL* variable. A different trick is used to
detect that an unquote has occured while reading a form other than a
simple vector or list. This trick is not as perfect,  in the
sense that there are more opportunities for the sub-reader to become
confused by the embedded UNQUOTE or SPLICE form, but but on the other
hand, in other cases we can produce a more accurate error message than
that a comma has occured outside of a backquote.

I still have a few items to do: 

- the ,. destructive splicing syntax is not implemented as destructive
  syntax; it's semantically equivalent to ,@ right now.
- the code generated by the backquote is not optimized; it 
  does lots of atrocious consing like (append (list '1) (list '2)).
- I have to conform the error messages to the internalization
  system in CLISP. Probably I will just reuse the existing ones,
  for the most part.
- write some automated tests that exercise complex backquotes,
  like the ones that bugged out on me.

If anyone is interested, I can provide a clean patch against the 2.30
release.

By the way. Let's try something:

;;; expected result:  (FOO `(BAR (BAZ 'A A) (BAZ 'B B) ...))
;;; CLISP result:     (FOO `(BAR NIL NIL NIL NIL))

[1]> (breaks-on-clisp) 
(FOO (APPEND (LIST 'BAR) '((BAZ 'A A) (BAZ 'B B) (BAZ 'C C) (BAZ 'D D))))

Woo hoo! Not bad for two evenings of hacking. ;)

How about the pretty printer breakage, whereby the backquote is
printed wrongly, but still evals right?

``,,`,3  ==>  `,NIL (stock CLISP)

``,,`,3  ==>  3     (hacked CLISP)

Score again! In fact, under the new hack, the ``,,`,FOO is completely
reduced to FOO, watch this;

  (macroexpand '``,,`,FOO)
  ==> FOO ; T

Ha!

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Mar 15 02:26:10 2003

> If anyone is interested, I can provide a clean patch against the 2.30

http://users.footprints.net/~kaz/clisp-2-30-backquote-2003-04-15.diff

---

From: Sam Steingold <[email-masked]>
Date: Sat Mar 15 07:30:19 2003

> * In message <[email-masked]>
> * On the subject of "[clisp-list] Re: New, from-scratch implementation of backquote."
> * Sent on Sat, 15 Mar 2003 01:45:18 -0800 (PST)

Great!
Congratulations!

> The aim is to produce a smaller, simpler, more elegant implementation

Did you check that the two defects are gone:

1. in backquote.lisp;

;; Bug: With nested backquotes some partial forms are evaluated several
;; times (e.g. in the primary evaluation: forms, which are needed for
;; the interpretation of secondary evaluation forms) and should
;; therefore be free from side-effects.

2. <http://clisp.cons.org/impnotes.html#backquote>:

        Multiple backquote combinations like ,,@ or ,@,@ are not
        implemented. Their use would be confusing anyway.

> macro rather than the other way around.

this appears to be the more standard approach.

> If anyone is interested, I can provide a clean patch against the 2.30

released under GNU GPL?

> reduced to FOO, watch this;

Great!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Good programmers treat Microsoft products as damage and route around it.

---

From: Kaz Kylheku <[email-masked]>
Date: Sat Mar 15 10:35:06 2003

> ;; therefore be free from side-effects.

Need to dig into this more to understand the problem.

>         implemented. Their use would be confusing anyway.

But a naive test of ,@,@ appears to works in stock CLISP:

``(,@,@(list '(+ 2 2)))

==> `(,@(+ 2 2))
==> (APPEND (+ 2 2))  ;; my version

The reason my version redues to APPEND is because the outer
backquote generates another backquote, which is recursively
expanded.

Again, will look deeper into the issue to understand
the real problem.

> > If anyone is interested, I can provide a clean patch against the 2.30

I will probably just assign the copyright (of the donated version of
the code) to you or Bruno, or else make it BSD like.

---

From: Kaz Kylheku <[email-masked]>
Date: Sun Mar 16 07:48:06 2003

> But a naive test of ,@,@ appears to works in stock CLISP:

I've been doing more tests, and it appears that my backquote
produces the same result with various ,@,@ situations as does
the stock one. So nothing breaks at least. 

However, one would expect the behavior to be like this:

   ``(,@,@'((list '+ 1 2) 3 4))

on the first evaluation, the outer ,@ splices in the list 
((list '+ 1 2) 3 4) leaving this:

  `(,@(list '+ 1 2) 3 4)

and so in the second round of expansion, the inner ,@ 
operator works only on the first element:

   ((+ 1 2) 3 4)

However, from both backquote implementations I get ``3 is not a list''.

Assuming that the behavior I described is the right one, you can 
of course get it this way, by separately treating the CAR and CDR
of the list.

  ``(,,(first '((list '+ 1 2) 3 4))
     ,@',(rest '((list '+ 1 2) 3 4)))

Note the extra quote in the ,@', which suppresses the additional
evaluation of the results of the REST which would otherwise take place,
in accordance with the semantics that once the tail section is spliced
in by the outer ,@ it should not be evaluated again by the inner ,@.

I clearly have to do more research on ,@,@ and perhaps take
it up on comp.lang.lisp.

---

From: Kaz Kylheku <[email-masked]>
Date: Mon Mar 17 00:34:11 2003

New patch:

http://users.footprints.net/~kaz/clisp-2-30-backquote-2003-04-17.diff

The backquote optimizer is in, and CLISP builds with it.

Optimization can be turned off by setting the variable
system::*backquote-optimizer* to NIL.

The approach is to build the naive APPEND arguments as described in the
HyperSpec, and then post-filter these. This way it is possible to write
a very small, simple backquote expander, coupled with optimizer for the
APPEND function that can be verified independently.

(system::bq-optimize-append '((list 1 2) (list 3 4)))
--> '(1 2 3 4)

(system::bq-optimize-append '((list 'a) (list b) c))
--> (LIST* 'A B C)

(system::bq-optimize-append '((list 'a) (list b) '(c)))
--> (LIST* 'A B C)

(system::bq-optimize-append '((list 'a) (list 'b) (list 'c) 'd)))
--> (A B C . D)

etc. 

In principle, we could use this as a basis for a compiler-macro over
the APPEND function.

Thus if some backquote bug appears, if it goes away when optimization
is turned off, it's a bug in BQ-OPTIMIZE-APPEND. If it does not, then
it's a bug in the semantics implemented by the macro.

---

From: Kaz Kylheku <[email-masked]>
Date: Fri Mar 21 23:46:02 2003

> ;; therefore be free from side-effects.

After a little experimentation, I understand what this is now. For
example, if we do this:

  (eval ````,',',',(print 3))

the form (print 3) is evaluated four times! Ouch! 

Yes, this bug goes away in the new implementation; the unquoted
forms are evaluated just once.

>        implemented. Their use would be confusing anyway.

I have requested a c.l.l discussion about these.

Regarding the unquote combination ,,@:

How this works in my implementation  is that the evaluation implied by
the right comma is distributed into the spliced list.

So whereas

 `(,@(list '(+ 2 2) '(+ 4 4)) d))

produces the splice

 ((+ 2 2) (+ 4 4) D)

adding a one more level of backquoting with an unquote around the
splice:

 ``(,,@(list '(+ 2 2) '(+ 4 4)) d))

produces

 (LIST (+ 2 2) (+ 4 4) 'D)

which when evaluated again produces

 (4 8 D)

So a kind of distributive law applies, as if the rightmost
comma went into every element form inserted by the splice, to
request its evaluation.

I have no idea whether this behavior is implied in any way by ANSI CL,
but it has a certain mathematical elegance to it. ;)

The above nested backquote does not work in stock CLISP at all; an
error is produced on second evaluation.

----

Regarding ,@,@:

A newbie might expect a naive text-like substitution, whereby
``(,@,@form) means: on first evaluation, splice in the text
representation of the list produced by form with parentheses removed,
to obtain `(,@x1 x2 x3 ...). Then on second evaluation, x1 must yield a
list which is spliced: (x11 x12 ... x2 x3 ...). But this interpretation
is laughable in addition to being quite useless.

My implementation does not support ,@,@ at all; but rather than
diagnosing it as an error, it unfortunately translates it into invalid
code. What's worse, the optimizer doesn't validate the bad code, and
sometimes turns it into working code. I.e. we have degenerate behavior.

My gut instinct is to just leave this alone until someone provides
a semi-authoritative interpretation of the spec on this matter.
Or else I can make it reject ,@,@ syntax with a diagnostic.

---

From: Kaz Kylheku <[email-masked]>
Date: Sun Mar 23 23:00:14 2003

> the right comma is distributed into the spliced list.

LispWorks does it this way too. And Timothy Moore showed me a macro
from McCLIM which ,,@ and depends on this behavior. Good!

LispWorks also seems to provide nice distributive behavior for ,@,@.

That is to say:

;; in LW
(eval ``(1 ,,@'((list 2 3) (list 4 5)) 6))
==> (1 (2 3) (4 5) 6)

(eval ``(1 ,@,@'((list 2 3) (list 4 5)) 6))
==> (1 2 3 4 5 6)

My implementation gets the above one right by fluke; but some
other cases (like if we remove the 1 and 6) confuse the optimizer.

(eval ``(,@,@'((list 2 3) (list 4 5))))

agrees with LispWorks if I disable the optimizer, but with
the optimizer turned on, an illegal reduction of `(APPEND ,@FORM)
to `,@FORM happens, which results in an error. Normally it's okay
to optimize (APPEND X) to X, but not when X has special semantics
for another round of subsequent expansion.

I think it's wortwhile hacking on this to nail down the ,@,@
behavior.

---

From: Sam Steingold <[email-masked]>
Date: Mon Mar 24 06:44:03 2003

> * In message <[email-masked]>
> * On the subject of "Re: New, from-scratch implementation of backquote."
> * Sent on Sun, 23 Mar 2003 22:58:24 -0800 (PST)

indeed!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Perl: all stupidities of UNIX in one.

---

From: Kaz Kylheku <[email-masked]>
Date: Tue Mar 25 09:26:17 2003

Try it!

  => (1 (2 3) (4 5) 6)

  => (1 2 3 4 5 6)

  (eval (eval ```(0 ,@,@,@'((list ''(1) ''(2)) (list ''(3) ''(4))) 5)))

  => (0 1 2 3 4 5)

So are the implementation notes are simply wrong in this regard,
or is there some subtlety that I might be missing?

Anyway, I have completed the patch. It builds cleanly, the entire 
testsuite passes, the ,@,@ stuff works, unquoting over #A arrays is 
disallowed. My confidence in the implementation is quite high.

All I need to do is add a whole lot more tests to tests/backquot.tst,
and then someone needs to take a look at the error messages in
src/new-backquote.lisp and translate them to French, German, etc.

http://users.footprints.net/~kaz/clisp-backquote-patch.html

---

From: Sam Steingold <[email-masked]>
Date: Tue Mar 25 12:48:14 2003

> * In message <[email-masked]>
> * On the subject of "Re: Actually ,,@ and ,@,@ does work in existing CLISP!"
> * Sent on Tue, 25 Mar 2003 09:24:40 -0800 (PST)

good.
I will integrate the patch.

> All I need to do is add a whole lot more tests to tests/backquot.tst,

that would be nice.

> src/new-backquote.lisp and translate them to French, German, etc.

don't worry about that for now.

> http://users.footprints.net/~kaz/clisp-backquote-patch.html

<http://users.footprints.net/~kaz/clisp-2-30-backquote-2003-04-24.diff>:

1. who is "Pei-Yin Lin"?

2. please add a (C) note and the licensing terms.

3. you need a define_variable() for *READING-ARRAY* in init_reader()

4. why do you call bq-optimize in places.lisp?
   I thought that optimizations were controlled by a variable.

5. you will need to write a single ChangeLog entry for the whole thing
   (nor an incremental one).

6. It would be nice if you could send the patches against CVS head, not 2.30.
   
Thanks!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat8 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Any programming language is at its best before it is implemented and used.