Subject: [clisp-list] Case sensitive reader. A call for action.

From: Marco Antoniotti <[email-masked]>
Date: Mon, 6 Nov 2000 17:10:20 -0500

Hi,

I do not know how many people here have been following the current
debate on C.L.L. regarding the issue of the case sensitive reader.

I believe this is not going away and that it is something we need to
deal with.  Franz has posed a problem and IMHO - in spite of the ANSI
standard - the users and implementors of the (two) major "open source"
implementations need to come up with a decent and well thought-out
response.

I surely do not have the expertise and the knowledge to pinpoint all
the issues in the major "open source" implementations, but I would
like to volunteer to two things:

1 - a straw poll about whether a "case sensitive" reader would be a
    good thing.
2 - what would be the "cost to implementors" (of - at least CMUCL and
    CLisp) -- e.g. what would it take to change all internal symbols
    to lower case.
3 - whether the change could be accommodated without changing too much
    the ANSI standard.  I.e. how the change can be accommodated in
    such a way to allow strict ANSI programs to be accepted.
4 - if the ANSI standard needed to be changed, then how should a
    consensus be reached, so that it somehow binds the implementors to
    the set of agreed upon choices.

I do this because I value the ANSI standard and all the good that has
come out of it, while at the same time I do understand the points made
by the case-sensitivity supporters.

Cheers 

-- 
Marco Antoniotti =============================================================
NYU Bioinformatics Group			 tel. +1 - 212 - 998 3488
719 Broadway 12th Floor                          fax  +1 - 212 - 995 4122
New York, NY 10003, USA				 http://galt.mrl.nyu.edu/valis
             Like DNA, such a language [Lisp] does not go out of style.
			      Paul Graham, ANSI Common Lisp

---

From: Robert Swindells <[email-masked]>
Date: Tue, 7 Nov 2000 02:41:45 GMT

>1 - a straw poll about whether a "case sensitive" reader would be a

I learnt lisp using FranzLisp, so I tend to prefer the idea to case
insensitivity.

I always used normal lower-case-with-hyphens symbol names though, I
just prefer seeing the same symbols that I have typed in.

I can see Kent Pitman's argument that you can hide FFI calls in code
like:

 (defun make-string-i/o-port (...) (jcall "MakeStringIOPort" ...))

but you still have to remember two sets of symbols or invent a strict
conversion algoritm.

I have written quite a lot of Motif code in C, but I haven't used CLM
much since I can't remember how to map from the Motif function names
to the CLM equivalents.

>    CLisp) -- e.g. what would it take to change all internal symbols

I once did this for AKCL - converted all the built in symbols to lower
case and added the extra CLTL2 readtable functionality.

It wasn't all that much work to do the change, I probably spent more
time getting various packages to work.

The kinds of things that break in user code are the ones raised in
c.l.l - packages and code that does (format nil "FOO-~A" bar) to
create symbols.

I think I stopped using it because of problems with Garnet.

>    the set of agreed upon choices.

Robert Swindells

---

From: Raymond Toy <[email-masked]>
Date: 07 Nov 2000 09:10:40 -0500

>>>>> "Robert" == Robert Swindells <[email-masked]> writes:

    Robert> I always used normal lower-case-with-hyphens symbol names though, I
    Robert> just prefer seeing the same symbols that I have typed in.

Same, but I don't care what case the symbols are print in, because I
know Lisp is uppercase.

    Robert> I can see Kent Pitman's argument that you can hide FFI calls in code
    Robert> like:

    Robert>  (defun make-string-i/o-port (...) (jcall "MakeStringIOPort" ...))

    Robert> but you still have to remember two sets of symbols or invent a strict
    Robert> conversion algoritm.

What are you going to do about C++ name mangling?  What about Fortran?
I've used different Fortran compilers whose resulting linker names had
a prepended underscore, an appended underscore, or was in uppercase.
Using case here only solves part of the problem.

Of course, you can always use |MakeStringIOPort| as your Lisp function
if you like.  Then you only have to remember to put || around your
Lisp functions.

I think FFI is a totally separate issue from case.

Ray

---

From: Bruno Haible <[email-masked]>
Date: Tue, 7 Nov 2000 14:57:32 +0100 (CET)

Marco Antoniotti writes:

> deal with.

Yes. Thanks for your initiative.

> Franz has posed a problem

IMO, Franz has given an answer to a problem that has been haunting us
for years.

> 1 - a straw poll about whether a "case sensitive" reader would be a

YES. The debate whether case sensitiveness in languages is over. The
model presented by all other mainstream languages (except Pascal) has
proven to be good.

Three things are involved:

  - How to turn it on for selective packages?
    (There are packages out there, like the Kenzo package, which
    are partially written in upper case and therefore rely on the
    traditional ANSI CL behaviour.) CLISP has been implementing the
    following for three years:

    MAKE-PACKAGE and IN-PACKAGE accept a keyword argument :CASE-SENSITIVE.
    Similarly, DEFPACKAGE accepts an option :CASE-SENSITIVE. When its value is
    non-NIL, the package will be case sensitive, i.e. the reader will not
    case-convert symbol names before looking them up or creating them in this
    package. The package names are still subject to (READTABLE-CASE *READTABLE*),
    though.

  - How to turn it on for the standardized packages (CL, CL-USER etc.)

  - How to turn it on for the package names?

Which *features* should be standardized to indicate one or the other?

>     CLisp) -- e.g. what would it take to change all internal symbols

It depends on whether two distinct images are acceptable, or whether
the switching should be doable at runtime.

>     the set of agreed upon choices.

Ask Rainer Joswig if there is currently a group working on
standardization. If not, it is (IMO) best done by communication among
implementor groups.

Bruno

---

From: Ken Olum <[email-masked]>
Date: Tue, 7 Nov 2000 12:26:09 -0500 (EST)

From: Bruno Haible <[email-masked]>
   Cc: [email-masked], [email-masked]

   > 1 - a straw poll about whether a "case sensitive" reader would be a

   YES. The debate whether case sensitiveness in languages is over.

No, it isn't.  If you say "yes, it is" that will demonstrate that in
fact there is still debate.  I think Lisp should be left alone.  Lisp
does a lot of things right that other languages do wrong.  

Among other things, I use Lisp via dictation, and getting case right in
dictation is a real pain.

				Ken

---

From: William Harold Newman <[email-masked]>
Date: Tue, 7 Nov 2000 17:32:18 -0600

On Tue, Nov 07, 2000 at 08:53:25PM +0100, Pierre R. Mai wrote:
> While the standard does indeed provide for a case-sensitive reader,
> this is pretty useless when the symbol-names of implementation
> packages is specified to be UPPER-CASE.  The :invert readtable-case
> was a half-hearted attempt at mitigating that issue, but it's an ugly
> hack, and doesn't really fill the need.

Is the problem here that the uppercase names are considered ugly, 
so that people want not only case-sensitive input, but also
lowercase names? I.e., they want case-sensitive input and also
want to type
  (cl:in-package "myFavoritePackage")
instead of 
  (CL:IN-PACKAGE "myFavoritePackage")
If people expect this, then what I said about the standard supporting
case sensitivity is off base -- if not technically off base, then at
least politically off base!

Perhaps someone should write down all the things which are expected
in a transition. (Perhaps someone already has and I just didn't
notice?) 
  * All reserved words must be lowercase.
  * Erik doesn't want the system to shout at him.
  * Franz wants their customers to have to keep paying no matter
    what happens.
  * [..]

For that matter, maybe someone could also summarize the nastiest
things which will happen if we don't just do something about this, but
work on other stuff instead.
  * Programmers will have trouble writing identifiers in
    non-English scripts -- Kanji, Arabic, Hangul, I dunno..
    I thought this was supposed to be the real problem, but some
    earlier poster seemed to think it wasn't important. Or perhaps 
    I misunderstood him.
  * People will have to use quoted java::|SymbolsLikeThis| for FFIs,
    or else come up with weird name mangling conventions.
  * People will dislike the syntax of the language (pointing out
    that Perl is much prettier:-) and stop using the language.
  * [..]

> Personally I like the migration path that Erik Naggum has repeatedly
> set out on c.l.l, like e.g. in message <[email-masked]>
> of today:  Separate the move to case-sensitive lower mode into two
> steps, by moving to lower-case symbol-names, and a case-insensitive
> reader (via readtable-case and print-case set to :downcase), which
> will then enable the standard reader controls to just work when
> someone wants to set readtable-case to :preserve, in order to get a
> case-sensitive lower CL.

Would this break old code? Is it worth it to require people to convert
their old
  (export '("A" "B" "C") "D")
code to shiny new
  (export '("a" "b" "c") "d")
code, or to Franz's
  (export '(#:a #:b #:c) #:d)
code?

Does anyone know how Fortran and COBOL have dealt with pressures for
case sensitivity? Maybe they've thought of something. They have a
sufficiently large code base that they should be highly motivated..

-- 
william Harold Newman <[email-masked]>
software consultant
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C

---

From: "Pierre R. Mai" <[email-masked]>
Date: 08 Nov 2000 01:46:04 +0100

> least politically off base!

While the two issues (case-sensitivity and lower-case preference) are
technically disjoint, there is a great overlap between those that
prefer case-sensitivity and those that want to write names with mostly
lower-case characters, i.e. like they write natural language.

There are lots of reasons for this, some of them historic (Unix
tradition), some of them social trends (there has been a long trend of
moving from UNIX-UPPER-CASE through UnixStudlyCaps to alllowercase.com
not only in the computer industry but in society at large), some of
them based on natural language preferences, etc.

Actually I fall into the lower-case preferring, but case-sensitivity
abhorring camp, but I'm a strange person anyway ;).  And really I
care fairly little about the lower-case thing, but very much about the
case-sensitivity thing.

>   * Franz wants their customers to have to keep paying no matter

This would be a good idea, _if_ you could find someone that was
sufficiently detached from the issues to be able to do this fairly.
That's very hard to do, so I won't even try...

> work on other stuff instead.

I think we've ignored the needs of a non-negligible portion of the CL
community for too long, already.  I think this has indirectly lead to
the current situation, and I think we really should now accomodate
those needs, or risk a fractioning of the community.  That's also why
I think this is mostly a social issue, and that the underlying
technical issues are really not at the heart of the problem.  That
doesn't mean we shouldn't care about technical issues as they crop up,
though.

>   (export '(#:a #:b #:c) #:d)

I think that any change to the current situation that will be able to
satisfy the needs of the case-sensitive-lower mode folks will break
some code.  There is just no way around that:  There's legacy code
that breaks even in current ANSI Common Lisp if you just change
*print-case* to :downcase, as Tim Bradshaw pointed out.  Turning on
case-sensitivity will also break much code, since there is lots of
code out there that will use varying case for the same symbols:
There's some code that uses both T and t, or Symbol and symbol and
SYMBOL, ...

That's why I think that both case-sensitivity and lower-caseness
should be separate _options_.  This allows one to fix the easy stuff
first:  Code that makes tacit assumptions about symbol-name case, like
the export/defpackage stuff you mention.  This can be easily fixed
because it can be easily located:  stuff like this almost always
involves symbol-related functions like intern, symbol-name, etc.

Then after that's taken care off, we can switch on case-sensitivity,
and fix the hard stuff, which is finding divergent case-spellings,
upper-case standard symbols, etc.  One are this will almost invariably
involve is interestingly enough #+/#- feature testing, since many
people (me included) seem to have retained ALL-UPPER-CASE symbols in
this context, even if they use all-lower-case symbols otherwise.

At least that's my experience after having gone over several packages
(both in-house developed and foreign code) doing the conversions above.

> sufficiently large code base that they should be highly motivated..

That seems like a good idea.  IIRC then Ada should also be a language
and community to look into.  But we also need to remember that case
issues are more complicated in Common Lisp, since it has the ability
to treat code as data and vice-versa:  Things like macros that create
new symbols and read/write invariants.

Regs, Pierre.

-- 
Pierre R. Mai <[email-masked]>                    http://www.pmsf.de/pmai/
 The most likely way for the world to be destroyed, most experts agree,
 is by accident. That's where we come in; we're computer professionals.
 We cause accidents.                           -- Nathaniel Borenstein