Subject: [clisp-list] Re: *print-pretty*, read-char

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Fri Mar 19 04:50:09 2004

Hi,

my perception of the thread is that what's really lacking is an *efficient* way to build streams of ad-hoc behaviour that interact nicely with all other stream and i/o functions.

Obvious difficulties:
 - interactive stream or not?
 - bidirectional or not (what's the mutual interference then)?
 - (setf stream-element-type)
 - efficiency (if you do read-char in CLISP, then something is wrong, because the code will be orders of magnitude slower than the built-in routines).

 - gray streams has a reputation of being dog slow (as would be anything using only characterwise i/o).
 - nobody implemented simple streams for CLISP
 - CLISP's buffered input or output streams are handy, but don't provide a solution w.r.t. interactiveness and bidrectionality, or (setf stream-element-type)

Yet ext:make-buffered-input-stream may be perfect for Don Cohen's current logfile processing needs.

>Clearly my current solution is exactly what I want.
>I read 8 bit bytes and then call code-char, using an encoding that
>gives me 256 different characters for the 256 bytes.
>It just seems silly that this should be the best (maybe only) way
>to get the desired result.

I recommend READ-BYTE-SEQUENCE and EXT:CONVERT-STRING-FROM-BYTES. That should be orders of magnitude faster and convert CR/LF as is.

If you cannot use built-in array manipulation functions in CLISP, then you loose big in speed (for anything in the area of i/O, FFI, character or number manipulations, etc.). That's what Erik Naggum once criticized, because he assumes this leads to or teaches people to write code that prefers a given solution path (introducing a bias) whereas it would be independent with those Lisp implementations that compile to native code and where almost everything including the i/o is written in Lisp, so no bias is introduced by the implementation and people can code towards the best design instead of the maximum use of built-in functions.

BTW, since you're reading httpd server log files, why is CR and/or LF a problem at all?? If you stick to CLISP's automatic conversion (which is perfect for text procesing), your code will run equally on httpd well on MS-Windows and UNIX machines. What am I missing?

Regards,
	Jorg Hohle.

---

From: Michael Koehne <[email-masked]>
Date: Fri Mar 19 06:50:05 2004

Moin Joerg-Cyril,

> BTW, since you're reading httpd server log files, why is CR and/or LF
> a problem at all?? If you stick to CLISP's automatic conversion (which
> is perfect for text procesing), your code will run equally on httpd well
> on MS-Windows and UNIX machines. What am I missing?

  i can not tell, what ANSI tells about - but autoconversion of \n to
  \r\n is a secure way to shoot your self into your foot, promoted by
  a major software house. The M$CC habbit to expand, will break
  calculations based on tell and seek and many more system calls.

  The problem started, when M$CC deceided to do the expansion based
  breaking the semantics of open, write and read system call
  emulations. A better way would have been to leave those calls as
  they are, and to implement a high level line mode on top.

  A portable application level line mode implementation, would better
  NOT allow any of the \n or \r characters to be part of the string,
  as mainframes either do not use any line break characters at the end
  of the line, but it might have ASA characters at the beginning of the
  line.

  Even Unix systems need \r\n for end of line, when talking interactive
  line mode telnet. Unix does not provide broken open,read and write
  system calls for auto expansion, but this is left to the user application
  for a good reason.

  Interactive line mode is much more difficult than batch line mode,
  if you think about IAC processing. Those commands could come in the
  middle of a line, e.g. the IAC GA command telling that this line is
  a "GO AHEAD" prompt and therefore finished WITHOUT linebreak, as the
  user has to type a line, which will do the linebreak. Now add color
  processing, and processing of double-print-characters (e.g. "^ha for
  &auml;) and double-print-line (using \r instead of \r\n) and you can
  imagine the can of worms.

  The question to the implementors: What are the requirements for an
  object class to integrate itself as a line mode stream into clisp ?
  (besides porting the low level funtions from GCL to FFI and making
  the system usabable at all ;-)

Bye Michael
-- 
  mailto:[email-masked]             UNA:+.? 'CED+2+:::Linux:2.4.22'UNZ+1'
  http://www.xml-edifact.org/           CETERUM CENSEO WINDOWS ESSE DELENDAM