Subject: Mini benchmark.

From: "John Carter" <[email-masked]>
Date: Thu, 26 Oct 1995 09:14:55 +0200

Greetings All,

I am contemplating writing some largish programs in CLISP. But before 
I commit myself to this I need a very rough idea of performance. I 
have this pet little bench mark which calculates e=2.71828... I like 
this benchmark because :-
  1) It reflects floating point performance, (which I am 
     most concerned with).
  2) It is dead simple. So simple it can be translated in 30 seconds 
     to any language you choose.
  3) It cannot be optimised away entirely.
  4) You can see immediately if you got the right answer.

Here it is in C or C++...
#include <stdio.h>

main( int argc, char * argv[])
{
  int n;
  double x, y;

  sscanf( argv[ 1], "%d", &n);
  x = 1.0;
  y = 1.0 + 1.0 / n;

  for( int i=0; i < n; i++)
    x *= y;

  printf( "y=%lf, x=%lf\n", y, x);
}

Here it is QBasic...
INPUT "n?"; n
x = 1
y = 1 + 1 / n
FOR i = 1 TO n
  x = x * y
NEXT i
PRINT "y="; y; " x="; x

Here it is in AWK...
{
  n=$1
  x = 1.0
  y = 1.0 + 1.0 / n
  for( i=0; i<n; i++)
    x *= y
  print "y=", y, " x=", x
}

Here it is in Actor...
n := 100000L
y := 1.0 + 1.0 / n
x := 1.0
loop
while n > 0
begin 
  x := x * y;
  n := n - 1;
endLoop;
print( "y=");
print( y);
print( " x=");
print( x);

And finally, the point of this whole message, in Lisp...
(defun test (n) 
  (let ((x 1.0) (y (+ 1 (/ 1.0 n))))
    (dotimes (i n)
      (setf x (* x y)))
    (print y)
    (print x)))
  
According to my tests I rank languages like so..
Interpreted Emacs lisp and Clisp
Qbasic.
Awk, Actor and MS-Excel Visual Basic
Compiled CLISP
Compiled EMACS Lisp
GNU C++ (33 times faster than Emacs lisp and 170x faster than Actor)

Now I may be doing something silly in Lisp (I'm not "at home" with 
it), can the readers please rewrite the lisp code to run as fast as 
possible. (All speed optimization flags on).

This may be a VERY VERY crude benchmark, but it gives a suprisingly 
(given the crudeness) accurate reflection of the reality.

Thank you,

John Carter
Institute for Water Quality Studies. Department of Water Affairs.
Internet : [email-masked]  Phone    : 27-12-808-0374x194      
Fax : 27-12-808-0338 [Host for Afwater list server]

Founder of the Council for Unnatural Scientists.

---

From: Giuseppe Attardi <[email-masked]>
Date: Thu, 26 Oct 1995 09:49:38 +0100

I tried your benchmark with ECoLisp (my own implementation of Common Lisp,
which compiles to C) on a Sparcstation10.

I got the following times:

real time : 2.233 secs
run time  : 0.900 secs
GC time   : 0.883 secs

but adding a few declarations like this:

(defun test (n)
  (declare (fixnum n))
  (let ((x 1.0) (y (+ 1.0 (/ 1.0 n))))
    (declare (short-float x y))
    (dotimes (i n)
      (setf x (* x y)))
    (print y)
    (print x)))

I got this result:

real time : 0.017 secs
run time  : 0.000 secs
GC time   : 0.000 secs

-- Beppe

PS
ECoLisp is available via anonymous ftp from:

 - ftp.icsi.berkeley.edu [128.32.201.7], directory /pub/ai/ecl
 - ftp.di.unipi.it [131.114.4.36], directory /pub/lang/lisp

---

From: Marcus Daniels <[email-masked]>
Date: Thu, 26 Oct 1995 10:45:17 GMT

John Carter wrote:

The reason that CLISP isn't faster for this function is
that the multiply isn't special-cased by the compiler for different
kinds of numbers; the multiply function has to figure out what kind of
type it is dealing with, and then do the operation. 

To use CLISP effectively for computationally intensive numeric tasks,
you either will need to make careful use the builtins, hook up foreign
code via the FFI, or use pipes/sockets to C programs.  

[It is worth noting that CL programs that use complex data structures
will likely have completely different performance characteristics
across CL implementations.  One example that stands out for me where
CLISP is as faster or faster than even CMU Lisp is the Koza Genetic
Programing kernel.  Even if you declare the heck out of the Koza
kernel, CLISP will still be as fast.  I think the reason is that CLISP
can ascertain lots of different types just by looking at the address
(at least with most Unix machines).  Other likely factors are the
footprint, the performance of hash-tables, and garbage collecting
performance. ]

The main thing you can be very sure of is that CL is not to blame.
Python (CMU Lisp), will give you performance equivalent to C.

(my adaptation of John's function)

(defun test (n)
  (declare (type (unsigned-byte 29) n))
  (let ((x 1.0d0) 
        (y (+ 1.d0 (/ 1.0d0 n))))
    (declare (type double-float x y))
    (loop
     (when (zerop n) (return x))
     (setq x (the double-float (* x y)))
     (decf n))))

Sure, short floats are faster, but they have the minor disadvantage
of being fairly meaningless for this application. :-)

marcus@sayre[~] $ time a.out 200000000
y=1.000000, x=2.718282

real 12.25
user 11.92
sys  0.04

CMU Lisp:
* (time (test 200000000))

Evaluation took:
  12.1 seconds of real time
  11.982946 seconds of user run time
  0.00745 seconds of system run time
  0 page faults and
  80 bytes consed.
2.718281805141847d0

---

From: [email-masked] (Bruno Haible)
Date: Thu, 26 Oct 95 13:21:12 +0100

>   1) It reflects floating point performance, (which I am 

But you have to be careful to measure the performance with the same
kind of floating-point numbers.

> Here it is in C or C++...

These are IEEE double-precision floats (53 mantissa bits).

>   (let ((x 1.0) (y (+ 1 (/ 1.0 n))))

These are, by default in Common Lisp, single-floats, i.e. (in CLISP)
IEEE single-precision floats (24 mantissa bits).

Giuseppe showed us declarations for short-floats (20 mantissa bits or so?).

You cannot directly compare timings of computations with different
kinds of floats.

> ...

Admittedly, CLISP's floating-point performance for single/double
floats is not good. (The Gabriel FFT benchmark shows this as well.)
But in CLISP, you can increase the precision of your floats as much as
you want, just by changing the *DEFAULT-FLOAT-FORMAT*,
*READ-DEFAULT-FLOAT-FORMAT* and (LONG-FLOAT-DIGITS) variables.

Say you want to compute e by its power series, like this:
(defun e (&optional (x 1.0))
  (do* ((n 0 (1+ n))
        (y (float 1 x) (/ (* y x) n))
        (sum (float 0 x)))
       ((= sum (setf sum (+ sum y))) sum)
) )

In CLISP you don't have to change your program _at_all_ if you want
to increase the precision of the result:
> (e)
2.718282
> (e 1.0L0)
2.7182818284590452354L0
> (e (float-digits 1 200)) 
2.7182818284590452353602874713526624977572470936999595749669676277244L0

(Of course, the built-in function EXP is still twice as fast than the
function E presented here.)

If this feature is not useful to you and you rely heavily on floating-
point performance, then you should try some other Lisps such as GCL or
(as Giuseppe pointed out) Ecolisp.

Bruno Haible                                     email: <[email-masked]>
Software Engineer                                phone: +33-1-49083585