[Openmcl-devel] GC aspects, tuning, behavior

[Jon S. Anthony]

On Mon, 2009-06-29 at 06:49 -0600, Gary Byers wrote:

> There isn't a promotion policy as such (or there's a ridiculously
> simple one): anything that survives a GC of generation N is
> immediately incorporated (promoted, tenured) into generation N+1.

OK, yes I would call that a promotion policy, simple though it may be.
Knowing this also clears up some stuff.

...
> something is tenured to oldspace after it survives a single GC happens
> "a small percentage" (the generation 0 survival rate) of .28% (.0028)
...

Thanks for the analysis - pretty neat.


> It is always the case - after any kind of GC - that all live memory is
> compacted into a contiguous region ("on the left") and therefore free
> memory is a single contiguous region on the right.

So, the I+1 region is compacted as well.  OK.


> > Start with a major/global/full GC.  Save the current value of
> > lisp-gc-heap-threshold. Set the threshold limits of gen-1 and gen-2 (NOT
> > gen 0) to something very small.  Actually how about just 0.  Set
> > lisp-heap-gc-threshold "high" - basically enough to hold all the objects
> > to be created.  Create objects.  There should be no minor GC during this
> > as they should all fit in Gen 0.  Do a full GC.  Set
> > lisp-gc-heap-threshold back to its saved (previous) value.  I guess this
> > part depends on promotion policy: Will all these things be moved to old
> > space at this point?  Or do you need to do a "few" GC's to trigger the
> > promotion?  Or will it all just go "KA-BOOM!"  Or...
> >
> 
> You lost me there.  If you do a full GC, then everything that's been allocated

Well, as I mentioned in another message, what I say there turns out to
makes no sense on a couple of levels.  And even if you "fixed" those, it
still would be kind of bogus.  I'm pretty sure that for this kind of
thing what Matthew suggested (and you echo) is the way to go: disable
the ecg during the load.  Setting the heap threashold to be around what
you expect to need is likely a win as well, but messing with anything
else really doesn't make much sense.


> A bulk-load (something that creates and initializes a lot of
> long-lived objects) does likely violate the "most new objects die
> young" assumption, and Matt's suggestion (turning off the EGC around
> the bulk load) may make that load/initialization run faster (the EGC
> won't be spinning its wheels discovering that most of these new
> objects are long-lived) and will get you to the same place (the
> newly-allocated objects will effectively wind up in old space, where
> they belong.) 

Exactly.  I am now quite sure this is the most sensible route.


> You might want to just time your initialization code with EGC on and
> with it off; I wouldn't be shocked if the EGC spent a lot of time
> doing nothing useful, but it's probably worth trying to verify that:
> even if you're (primarily) creating a lot of long-lived objects, the
> process of doing so may or may not also create a lot of short-lived
> objects that the EGC likes.

Also makes sense.

Many thanks to you (and Matt) for taking the time with this.  I
appreciate it and I think I have a pretty good grasp of what is going on
and the issues now.  More than enough to move on.  Thanks!


/Jon