[Openmcl-devel] Identity Hashtables

[Sven Van Caekenberghe]

I have a performance problem which I think is related to Identity 
Hashtables (a hashtable where the keys are plain objects like CLOS 
instances, conses, as well as strings, just anything), and I am looking 
for a way to speed things up or for an alternative solution.

The code in question serializes any object structure to XML. While 
doing this, the code keeps a hashtable (currently (make-hash-table 
:test 'eql)) containing every object it has encountered (mapping it to 
its id), so that it can properly handle shared and/or circular 
references (by refering to the known id).

When serializing object structures containing thousands of objects 
(which is not that big) the code becomes so slow that it cannot 
complete (it is still writing XML, but very slowly). I suspect that 
using the hashtable becomes exponentially slower because (1) the 
hashtable keeps growing and everytime it grows it needs to rehash (2) 
the whole process consumes a lot of memory so that GC kicks in a lot, 
which, IIRC from a post by Gary somewhere, results in all hashtables 
being rehashed because the hash is based on the address in memory and 
this can potentially change with each GC.

Can I control this process somehow ? Perhaps by controlling how the 
hashtable grows ? I figure that unless every Lisp object stores its own 
hash, there isn't a lot that can be done. Maybe I can make another 
datastructure for this purpose ?

Sven


_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel