Subject: [clisp-list] Pathname tests

From: Christophe Rhodes <[email-masked]>
Date: Fri, 24 Aug 2001 16:57:38 +0100

Dear all,

As was requested a while back, I've made a sorta test suite for
pathname handling, available from
<URL:http://www-jcsu.jesus.cam.ac.uk/~csr21/pathname-tests.lisp"> or
cCLan CVS; I hope you don't mind me saying that clisp 2.27's behaviour
isn't too great...

To see the results, load the above file into a clisp session running
on Unix (there are just one or two unix-specific bits), and type
(run-tests). Diagnostics and information about failures will be sent
to the *trace-output* stream.

Thanks,

Christophe
-- 
Jesus College, Cambridge, CB5 8BL                           +44 1223 510 299
http://www-jcsu.jesus.cam.ac.uk/~csr21/                  (defun pling-dollar 
(str schar arg) (first (last +))) (make-dispatch-macro-character #\! t)
(set-dispatch-macro-character #\! #\$ #'pling-dollar)

---

From: Sam Steingold <[email-masked]>
Date: 24 Aug 2001 12:16:53 -0400

> * In message <[email-masked]>
> * On the subject of "[clisp-list] Pathname tests"
> * Sent on Fri, 24 Aug 2001 16:57:38 +0100
> * Honorable Christophe Rhodes <[email-masked]> writes:
>
> cCLan CVS;

the link is broken (the one you sent c.l.l is broken too)

> I hope you don't mind me saying that clisp 2.27's behaviour
> isn't too great...

only if your tests check ANSI compliance and not CMUCL compliance.

-- 
Sam Steingold (http://www.podval.org/~sds)
Support Israel's right to defend herself! <http://www.i-charity.com/go/israel>
Read what the Arab leaders say to their people on <http://www.memri.org/>
((lambda (x) (list x (list 'quote x))) '(lambda (x) (list x (list 'quote x))))

---

From: Christophe Rhodes <[email-masked]>
Date: Fri, 24 Aug 2001 17:39:40 +0100

> the link is broken (the one you sent c.l.l is broken too)

Thanks -- I believe this is now fixed. Apologies for that.
 
> only if your tests check ANSI compliance and not CMUCL compliance.

Again, if you[*] feel that you disagree with my interpretation of the
specification or that my tests are buggy, please let me know; a buggy
set of tests is probably worse than no tests at all.

Note that these tests were developed with the HyperSpec as sole
reference; while I admit I could be wrong on any number of points
(though I don't think so; I've read this section of the specification
fairly closely) I think that even an implicit accusation of
implementation-bias (before seeing the code!) is a little harsh. 

Cheers,

Christophe

[*] Addressed to anyone, not just Sam

---

From: Sam Steingold <[email-masked]>
Date: 24 Aug 2001 14:33:19 -0400

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] Pathname tests"
> * Sent on Fri, 24 Aug 2001 17:39:40 +0100
> > > <URL:http://www-jcsu.jesus.cam.ac.uk/~csr21/pathname-tests.lisp> 

works now - thanks!

the file is largely useless as it is right now since it prints lots of
junk (like this:

------
A serious error has occurred.
 
This implementation has raised the following error on execution of ANSI-compliant code:
...
Please fix this bug before continuing.
------
which is irrelevant) and does not print the _form_ which resulted in the
error.
it also dies on the first error.

I suggest that you look at the CLOCC/src/tools/ansi-tests and the CLISP
test suite (on which the CLOCC tests are based).
It would be nice if your test could be added to those test suites.

I wish I could investigate the CLISP errors - I cannot now since I do
not have the form(s?) that generate errors (I want to see _all_ of them
before I delve into this) and I do not have time to read your code.
Sorry.

-- 
Sam Steingold (http://www.podval.org/~sds)
Support Israel's right to defend herself! <http://www.i-charity.com/go/israel>
Read what the Arab leaders say to their people on <http://www.memri.org/>
Lottery is a tax on statistics ignorants.  MS is a tax on computer-idiots.

---

From: Christophe Rhodes <[email-masked]>
Date: Sat, 25 Aug 2001 10:57:39 +0100

On Fri, Aug 24, 2001 at 02:33:19PM -0400, Sam Steingold wrote:
> which is irrelevant) 

This is not irrelevant at all, though admittedly it isn't as helpful
as it might be; since the error message (that you snipped) comes from
clisp's condition system perhaps you should treat that as a bug report
too.

There are two issues, only one of which this code is testing; one is
making sure that `answers' from pathname functions are right; the
other is that errors are not thrown when executing compliant code. I
am not testing the latter; if an implementation throws errors when
executing valid code, it's broken beyond the remit of my tests; I'm
testing for mistakes in the interpretation/reading of the standard.

Now, the error message that you snipped above is probably:

---
TRANSLATE-PATHNAME: replacement pieces ((:DIRECTORY "infix") "infix"
"1") do not fit into #P"UNIX:USR;SHARE;COMMON-LISP;SOURCE;**;*.LISP.*"
---

which is telling you that in non-ANSI mode
logical-pathname-translations and friends are not entirely happy (it's
interpreted #P"UNIX:USR;SHARE;COMMON-LISP;SOURCE;**;*.LISP.*" as a
physical pathname (with a very long name). This should have been a
clue; I'm sorry it wasn't obvious enough.

> test suite (on which the CLOCC tests are based).

I suggest that when you test your implementation for ANSI compliance
you run it in ANSI mode; it's not all that surprising that odd effects
occur when you don't; I'm not trying to evaluate the bizarre dialect
that is clisp in non-ANSI mode.

> It would be nice if your test could be added to those test suites.

Yes; the problem lies in constructing things that can lie in an
already-established framework. The problem in doing this is that often
there is no uniquely-determined answer (should the device be nil or
:unspecific? What should the version be in (make-pathname)?) and so
it isn't quite as simple as just copying and pasting into an existing
test framework.

> before I delve into this) and I do not have time to read your code.

I have attached the results of running the code on clisp 2.27 in ansi
mode (with the -ansi command-line option) on my Debian Unix system. I
hope that the output is sufficiently clear; if you're not going to be
reading the code at all, you may need to know that "UNIX" and
"CL-LIBRARY" are defined as logical hosts. It may be that the output
is insufficiently detailed to enable you to pinpoint errors, but I
hope that it will provide sufficient motivation to look at those
routines that seem to be causing problems. 

Cheers,

Christophe

---

From: Sam Steingold <[email-masked]>
Date: 25 Aug 2001 11:34:01 -0400

> * In message <[email-masked]>
> * Sent on Sat, 25 Aug 2001 10:57:39 +0100

In that case it would have been helpful if you added something like
this:

#+clisp
(progn
  (format t "putting CLISP in ANSI compatibility mode~%")
  (setq ext:*merge-pathnames-ansi* t
        ext:*parse-namestring-ansi* t))

> > I suggest that you look at the CLOCC/src/tools/ansi-test and the CLISP
> occur when you don't;

you are right - I forgot to add -ansi.  my fault.
still, the fact that you do not supply the form(s) that generated the
errors makes your test suite much less useful.

> :unspecific? What should the version be in (make-pathname)?) and so it
> isn't quite as simple as just copying and pasting into an existing
> test framework.

this is a very common problem and it has been successfully solved in
both CLISP test suite and the CLOCC one.

> hope that the output is sufficiently clear;

not at all.  good output allows using cut and paste to reproduce errors.
i.e., you should print the entire forms which lead to incorrect output
together with all the init forms (defining logical hosts &c).
CLOCC/src/tools/ansi-tests also prints references to the parts of the
manual on which the particular test is based - which is very good.

again, please see CLISP/tests and CLOCC/src/tools/ansi-test for
examples.

I will look at the problems on Monday.

-- 
Sam Steingold (http://www.podval.org/~sds)
Support Israel's right to defend herself! <http://www.i-charity.com/go/israel>
Read what the Arab leaders say to their people on <http://www.memri.org/>
Why do we want intelligent terminals when there are so many stupid users?