Subject: [clisp-list] AW: TCP-based Network Server in CLISP: read-char vs. buffering

From: "Hoehle, Joerg-Cyril" <[email-masked]>
Date: Mon Sep 30 08:27:14 2002

Don [Cohen?] writes:
>  I assume the cost of
>read-byte and read-byte-will-hang-p is not greatly different for
>socket streams than for files.

IMHO that's a very wrong assumption. E.g., CLISP will automatically create a #<BUFFERED STREAM> for regular files. This means that already at the application level, read/write() calling behaviour will be different for both.

Then, at OS level, vastly different things happen for sockets, pipes and files.

>  Files are just a lot more convenient to test than sockets.
> [7]> (time (f "/tmp/25MB" 1000000))
> Real time: 2.016425 sec.
> Run time: 2.0 sec.
> Space: 4584 Bytes
That is ~ 500KB/sec if I understood your code well. I believe you can achieve more with CLISP on that hardware - by working on larger entities than characters.

How would that compare to n reads of 1KB or 10KB arrays on your machine? But with buffered streams, doing that would be benchmarking CLISP's bytecode interpreter, not OS i/o speed.

Regards,
	Jorg Hohle.

---

From: [email-masked] (Don Cohen)
Date: Mon Sep 30 09:15:16 2002

Hoehle, Joerg-Cyril writes:
 > IMHO that's a very wrong assumption. E.g., CLISP will automatically
   create a #<BUFFERED STREAM> for regular files. This means that already
   at the application level, read/write() calling behaviour will be
   different for both. 
Isn't it the same thing for tcp streams?
In this case the OS already has a buffered stream so perhaps clisp
doesn't need to do any more.
 > Then, at OS level, vastly different things happen for sockets,
   pipes and files. 
Ok, but we're really only concerned here with what happens in lisp.
If the OS reads files 10 times as fast as tcp streams then I can read
fewer bytes/sec from a tcp stream, but this example at least shows
that clisp is not spending more than 2usec to do both a
read-byte-will-hang-p and a read-byte.

 > That is ~ 500KB/sec if I understood your code well. I believe you
   can achieve more with CLISP on that hardware - by working on larger
   entities than characters. 

 > How would that compare to n reads of 1KB or 10KB arrays on your

data below

   machine? But with buffered streams, doing that would be benchmarking
   CLISP's bytecode interpreter, not OS i/o speed. 
OS IO speed is not the issue here, cause that's buffered in any case.
The difference is whether you do a large IO operation in lisp that
may read or write many bytes "at once" but also might block, or
whether you test for each byte in order to avoid blocking.

 [5]> (compile (defun f (file n &aux (m 0))
  (with-open-file (s file)
  (loop while (> n 0) do (decf n (length (read-line s))) (incf m)) m)))
 F ;
 NIL ;
 NIL
 [6]> (time (f "~/clocc/mail-log" 100000))

 Real time: 0.058673 sec.
 Run time: 0.06 sec.
 Space: 222336 Bytes
 1905
 [7]> (time (f "~/clocc/mail-log" 1000000))

 Real time: 0.646889 sec.
 Run time: 0.63 sec.
 Space: 2192024 Bytes
 GC: 4, GC time: 0.03 sec.
 20076
So for this file the average read-line is about 50 chars and I read
chars a little over 3 times as fast as before.  (I assume that chars
vs bytes don't make a big difference here.)