Subject: [clisp-list] On benchmarking (was: TCP-based Network Server in CL ISP: read-cha r vs. buffering)

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Wed Oct 16 05:54:05 2002

Todd Sabin wrote:
>> Reading 1_000_000 bytes from a larger file (~3MB),
>> CLISP TIME reported real time/run time for one run:
>> from source:    local file      Samba/NFS server
>> unbuffered      161/132 secs    1866/173 seconds!!

>I suspect this has little to do with Windows, per se, and more to do
>with the way clisp does I/O on Windows.  If you look at
>DoInterruptible in win32aux.d, you'll notice that clisp creates (and
>destroys) a thread every time through DoInterruptible.  Combine that
>with an unbuffered stream and the result is that clisp creates and
>destroys an OS thread for every character read.  I'm not sure if that
>would account for all of the difference, but it seems like a very good
>place to start...

Indeed it sounds very convincing and thus a very bad behaviour of CLISP that ought to change in some future, but I now feel that's not enough to acount for the enormous difference between local and remote reading of files.

Invoking task creation overhead for each character (or byte) read should yield the same times in both cases.
So this could possibly explain the 161 seconds of the local file test. But not the 1866 seconds. This is corroborated by the cpu% load graphical display which shows 100% activity in the local case (explained by being constantly busy creating tasks) but almost no activity during the remote file test.

I further assume that because I'm reading from files and not using socket directly in Lisp, that it's the same low-level functionality of CLISP that is invoked, e.g. stream.d: low_read_unbuffered_handle() and win32aux.d: DoInterruptible(&do_full_read,...) -- and not lowlevel_sock_read() for sockets.

Therefore the difference in timing must be accountable to something happening below CLISP: the OS, services or handlers - or interaction between both.

From there on, single bytes (payload) traveling across the network seem a resonable explanation of the measurements. This could be caused by MS-Windows network handlers, the Samba server, some configuration switches or whatever.

The question then remains why Linux doesn't seem to do single byte reads across the network. Maybe Don can shed light on this?

Regards,
	Jorg Hohle.

---

From: Todd Sabin <[email-masked]>
Date: Wed Oct 16 22:07:02 2002

"Hoehle, Joerg-Cyril" <[email-masked]> writes:

> [...]
> Therefore the difference in timing must be accountable to something
> happening below CLISP: the OS, services or handlers - or interaction
> between both.
> 
> From there on, single bytes (payload) traveling across the network
> seem a resonable explanation of the measurements. This could be
> caused by MS-Windows network handlers, the Samba server, some
> configuration switches or whatever.
> 
> The question then remains why Linux doesn't seem to do single byte
> reads across the network. Maybe Don can shed light on this?

I'd guess the difference is between NFS and SMB, then.  The linux NFS
implementation is probably reading entire blocks (instead of single
bytes) across the network, whether you've asked for them or not, but
the windows SMB implementation is only reading single bytes as you ask
for them.  You can try sniffing the network traffic and seeing if
that's the case.

Todd