Subject: [clisp-list] CONSing by structure access functions

From: "John K. Hinsdale" <[email-masked]>
Date: Mon Sep 16 18:14:02 2002

After much success tuning up some code under CLISP, I'm still having
trouble understanding why some things cost what they do in terms of
CPU and memory (particularly garbage-collected storage).

By observation, using the consing profiler, noticed that when I call
the struct member access functions generated by DEFSTRUCT that these
functions CONS.  I can't figure out why they would do that given that
they are just accessing fixed pieces of data in a data structre of
known elements.

Below is some code that creates an array of 1000 structures, then
loops over the array, doing nothing more than accessing members of the
strutures.  I don't understand why there should be any consing at all.

The reason this is relevant for me is that I've designed a library w/
heavy use of Lisp's structure constructs, and it tends to loop over
large numbers of them.  Burdening the garbage collector with lots of
CONSing is making it slower, but not terribly so.

I mostly just want to understand.  Is this specific to the CLISP
implementation or part of how Lisp is supposed to work?

; ----------------  CODE HERE

#!/usr/local/bin/clisp -q

(defstruct foo
  mem1 mem2)

; Make a bunch of 1000 "foo" structures
(defun make-stuff ()
  (setq arr (make-array 1000))
  (map-into arr #'(lambda (f) (make-foo :mem1 "blah" :mem2 "blorf")) arr))

; Exercise it
(defun grind ()
  (let ((limit (- (length arr) 1)))
    (loop for i from 1 to limit do
	  (let ((f (aref arr i)))
	    (foo-mem1 f)
	    ))))

(load "/u/pkg/lisp/metering/metering.lisp")
(mon:monitor-form
 (progn
   (make-stuff)
   (grind)))

; ----------------  PROFILER OUTPUT BEGINS HERE

Real time: 0.340808 sec.
Run time: 0.34 sec.
Space: 496344 Bytes
GC: 1, GC time: 0.01 sec.

                                                 Cons    
               %      %                          Per       Total     Total
Function      Time   Cons    Calls  Sec/Call     Call      Time      Cons
-----------------------------------------------------------------------------
MAKE-STUFF:  48.78   49.20        1  0.159994    236144     0.160      236144
GRIND:       39.63   41.63        1  0.129994    199800     0.130      199800
FOO-MEM1:    10.37    1.67      999  0.000034         8     0.034        7992
MAKE-FOO:     1.22    7.50     1000  0.000004        36     0.004       36000
-----------------------------------------------------------------------------
TOTAL:      100.00  100.00     2001                         0.328      479936
Estimated monitoring overhead:  0.01 seconds
Estimated total monitoring overhead:  0.01 seconds

---
John Hinsdale, Alma Mater Software, Inc., Tarrytown, NY 10591-3710 USA
[email-masked] | http://www.alma.com/staff/hin | +1 914 631 4690

---

From: Sam Steingold <[email-masked]>
Date: Tue Sep 17 06:47:03 2002

> * In message <[email-masked]>
> * On the subject of "[clisp-list] CONSing by structure access functions"
> * Sent on Mon, 16 Sep 2002 21:13:16 -0400
> * Honorable "John K. Hinsdale" <[email-masked]> writes:
>
> Real time: 0.340808 sec.
> Run time: 0.34 sec.
> Space: 496344 Bytes

1. you did not compile your code, right?  I did and I got this:

[4]> (space (progn (make-stuff) (grind)))
                                            Permanent            Temporary
Class                                  instances   bytes    instances   bytes
-----                                  --------- ---------  --------- ---------
FOO                                         1000     20000          0         0
SIMPLE-VECTOR                                  1      4008          0         0
-----                                  --------- ---------  --------- ---------
Total                                       1001     24008          0         0

Real time: 0.00291 sec.
Run time: 0.0 sec.
Space: 24008 Bytes

2. You should always take the output of metering.lisp with a grain of
   salt.  It is cross-platform and quite old, so it can have
   unpredictable side effects and overhead.
   
   I did some work this spring (so that 2.30 has it) on CLISP and
   CLOCC/src/tools/metering.lisp to reduce the overhead, but there still
   may be some.

   That said, metering is a great package and I have used it to great
   benefit myself.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.3 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Booze is the answer. I can't remember the question.