Subject: [clisp-list] next question - socket streams unbuffered?

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Wed Sep 17 00:40:02 2003

Hi,

> #<IO UNBUFFERED SOCKET-STREAM CHARACTER 0.0.0.0:5500>
>What does it mean for the stream to be unbuffered?
>Clearly when the other side sends characters they're stored somewhere
>until I read them from the stream.

It means that CLISP does not use an internal application-level (4K IIRC) buffer. That doesn't prevent the underlying OS from doing some buffering itself. The network always does some buffering :-)

>Impnotes shows a lot of :buffered arguments but doesn't say much about
>what they mean.

Here's a suggestion for adding somewhere in impnotes

:buffered
Have CLISP manage an internal 4KB buffer for input or output. Buffering is a known general technique to speed up i/o by many factors.
CLISP detects i/o to regular files itself and defaults to buffering.
For sockets or special files however, CLISP doesn't default to using an internal buffer. Therefore, for use with e.g. HTTP output to socket streams, you may obtain speed improvements by having CLISP do output in blocks instead of character by character by setting :buffered to true. The size of the internal buffer is currently 4KB (IIRC).

Note that using :buffered on interactive streams (typically network protocols) may require thoughtful addition of calls to FORCE-OUTPUT or FINISH-OUTPUT.

Personal side note:
The actual performance improvements depend a lot on the underlying OS, as the following examples show:

o I used *print-pretty* very early on the Amiga, because the buffering that it causes (using write-string instead of write-char) sped up output to the terminal by a huge factor.

o I reported speed improvements in this group by a factor >10 (and data volume reduce by a factor >200) when using buffering over a samba filesystem connection when using read-char (MS-W2k reading from Linux Samba server). The same code on Linux (using NFS) did not show a difference for :buffered T/NIL, presumably because NFS does i/o in 8KB chunks itself.

Sadly, read-char is very commonly used in CL, and when :buffering is not true, all bets are off as to the performance of the system, as the second example shows. IIRC this was in a thread in this list about who is responsible for buffering: the application programmer or the OS.

I found e-mails of mine from 14th February 2002 and 17th October 2002 on this topic.

Regards,