Subject: [clisp-list] special characters in comments

From: Matt Kaufmann <[email-masked]>
Date: Sat Oct 12 20:12:06 2002

Hi --

I've been testing a Lisp application, ACL2, built on CLISP 2.30.  (The home
page for ACL2 is http://www.cs.utexas.edu/users/moore/acl2).  Several input
(.lisp) files supplied to me by ACL2 users have caused problems because of the
use of special characters in comments, as evidenced by these error messages:

*** - invalid byte #xAB in CHARSET:ASCII conversion
*** - invalid byte #xB7 in CHARSET:ASCII conversion
*** - invalid byte #xED in CHARSET:ASCII conversion
*** - invalid byte #xF3 in CHARSET:ASCII conversion
*** - invalid byte #xF1 in CHARSET:ASCII conversion

In all these cases, the characters were to the right of a semicolon on their
lines.  Shouldn't one be able to put any character desired between a semi-colon
(on the left) and a newline (on the right)?

By the way, on the positive side, CLISP found a bug that other Lisps (GCL,
Allegro CL, CMU CL) had missed (use of ",@" on an atom).

Thanks --
-- Matt Kaufmann

---

From: Sam Steingold <[email-masked]>
Date: Sun Oct 13 07:32:02 2002

> * In message <[email-masked]>
> * On the subject of "[clisp-list] special characters in comments"
> * Sent on Sat, 12 Oct 2002 22:11:43 -0500 (CDT)
> * Honorable Matt Kaufmann <[email-masked]> writes:
>

<http://clisp.cons.org/faq.html#enc-err>

"-E utf-8" or an explicit :EXTERNAL-FORMAT argument to OPEN and LOAD
are your friends.

> By the way, on the positive side, CLISP found a bug that other Lisps
> (GCL, Allegro CL, CMU CL) had missed (use of ",@" on an atom).

great!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.3 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
char*a="char*a=%c%s%c;main(){printf(a,34,a,34);}";main(){printf(a,34,a,34);}

---

From: Matt Kaufmann <[email-masked]>
Date: Sun Oct 13 21:10:07 2002

Thank you for the quick reply.  So, I've added the following to the ACL2
sources (the check for unicode is necessary because charset:utf-8 was
introduced after CLISP 2.27):

  #+(and clisp unicode) (setq custom:*default-file-encoding* charset:utf-8)

This eliminated problems with two of the characters (#xAB and #xB7), but some
characters used in comments still cause errors:

*** - invalid byte sequence #xED #x74 #x75 in CHARSET:UTF-8 conversion
*** - invalid byte sequence #xF1 #x61 #x74 #x69 in CHARSET:UTF-8 conversion
*** - invalid byte sequence #xF3 #x20 #x61 #x2F in CHARSET:UTF-8 conversion

By the way, I tried the FAQ page but didn't find the link to #enc-err.  Upon
investigation I noticed the following line in
<http://clisp.cons.org/faq.html#enc-err>:

<h3 id="enc-err">invalid byte #x94 in CHARSET:ASCII conversion</h3>

I'm running Netscape 4.79 under Redhat Linux 7.3, and that browser does not
seem to recognize this tag.  Probably "id=" could be changed to "name=" to fix
this problem, but I'm no expert on web stuff so if this seems wrong, just
ignore that suggestion.

Thanks --
-- Matt
   X-Authentication-Warning: loiso.podval.org: sds set sender to [email-masked] using -f
   Cc: [email-masked], [email-masked]
   Reply-To: [email-masked]
   Mail-Copies-To: never
   From: Sam Steingold <[email-masked]>
   Date: 13 Oct 2002 10:32:26 -0400
   User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50
   Content-Type: text/plain; charset=us-ascii

   great!

---

From: Adam Warner <[email-masked]>
Date: Sun Oct 13 23:20:06 2002

> *** - invalid byte sequence #xF3 #x20 #x61 #x2F in CHARSET:UTF-8 conversion

If you use a faithfully reproducing character set it will be impossible
to use invalid characters in comments. I suggest you try iso-8859-1
decoding instead of utf-8.

You can read about faithful output here:
http://cl-cookbook.sourceforge.net/io.html

Longer term you may wish to convert the code to utf-8 encoding so you
can use that instead of an 8-bit character set.

Regards,
Adam

---

From: Matt Kaufmann <[email-masked]>
Date: Mon Oct 14 06:45:04 2002

Thanks, but changing to iso-8859-1 encoding didn't seem to help:

  [1]> custom:*default-file-encoding*
  #<ENCODING CHARSET:ASCII :UNIX>
  [2]> ; ó

  1. Break [3]> :a

  [4]> (setq custom:*default-file-encoding* charset:iso-8859-1)
  #<ENCODING CHARSET:ISO-8859-1 :UNIX>
  [5]> ; ó

  1. Break [6]> 

The vast majority of my problems were with the two characters that are handled
by utf-8.  I've changed the user-contributed files that use the other three, so
this isn't any longer an issue for me, anyhow.  Thanks for the responses.

-- Matt K.
   From: Adam Warner <[email-masked]>
   Cc: [email-masked]
   Content-Type: text/plain
   Date: 14 Oct 2002 19:19:16 +1300

   Regards,
   Adam