[Openmcl-devel] Memory managemet Objective C and Clozure

[Gary Byers]

On Sun, 19 Sep 2010, Willem Rein Oudshoorn wrote:

>
>
> Strange behaviour in memory magement
> ------------------------------------
> I am still convinced the following is not right:
>
> 1 - If you create an instance of an subclass of ns:ns-object
>    with lisp slots
> 2 - You should never use (ccl:terminate-when-unreachable)
>    on that instance because:
> 3 - the *objc-object-slot-vectors* keeps a reference to the instance

You're right.  Neither the pointer used as a key in
*OBJC-OBJECT-SLOT-VECTORS* nor the slot-vector's backpointer to the
instance should be EQ to any existing lisp pointer to the instance.

> 4 - This reference only goes away when the #/dealloc is called
> 5 - So terminate will only be called AFTER dealloc.
> 6 - Terminate calls #/release [which is wrong!]
>   (This could potentially
>    be solved by canceling the 'terminate' call in the dealloc call.
>    However I assume that this prevents user code for using the
>    terminate.  Which could be considered bad as well.)

Having the TERMINATE method call #/release is correct and useful
if the object is #/retain'ed when it's registered for termination:
you're basically saying that the ObjC object shouldn't be deallocated
as long as a particular lisp object that references it is reachable.  The
bug above is that the references to that particular lisp pointer from
the slot-vector mechanism mean that that object is unlikely to be GCed
(and the ObjC object would still be retained.)

Just calling #/release from the TERMINATE method - without a matching
#/retain earlier in the lisp object's lifetime - isn't a good idea,
except in the degenerate case where you're somehow sure that the ObjC
object isn't otherwise retained.

>
>
> I naively made the remark that I thought *objc-object-slot-vectors*
> should have weak keys, but as you pointed out that does not work.
>
> Gary Byers <gb at clozure.com> writes:
>
> [About the *objc-object-slot-vectors* hash-table]
>> A weak hash table could tell us when an encapsulating lisp object
>> becomes garbage, but that isn't what we want to know in this and
>> similar cases: the slot-vector should exist as long as the ObjC
>> object exists.  (If all encapsulating lisp pointers to the ObjC
>> object became garbage, we can't safely remove the slot-vector because
>> some  might create a new encapsulating pointer to the object or receive
>> such a pointer as an argument)
>
>
> What would I expect
> -------------------
>
> I have thought a little about it, but what I would expect is that:
>
> 1 - As soon as a lisp reference to an objective-C object 'o' is created
>    'o' is send a #/retain message.   [*]
>
> 2 - As soon as the garbage collector discovers that reference
>    is not longer accessible from the lisp side,
>    it sends a #/release message
>
> 3 - If the #/dealloc is called, everything is cleaned up.
>
>
> However I can see that step 2 conflicts with the
> *objc-object-slot-vectors* hashtable.  Because the garbage collector
> will always see a reference to the object from the hashtable.

That's just a bug, but that bug is related to the reason that schemes
like this aren't viable.  (At the very least, I've never been able to
think of a way to make them viable.)

The function GUI::WINDOWS returns an ordered list of the application's
windows (it's just a wrapper around -[NSApplication orderedWindows].  So:

? (car (gui::windows))
#<HEMLOCK-LISTENER-FRAME <HemlockListenerFrame: 0x585210> (#x585210)>
? (car (gui::windows))
#<HEMLOCK-LISTENER-FRAME <HemlockListenerFrame: 0x585210> (#x585210)>
? (eq * **)
NIL

(Unique, non-EQ) Lisp references to ObjC objects are created all the
time; every time that (for instance) an event handler:

(objc:defmethod (#/mouseMoved: :void)  ((self my-view) event)
  ...)

is called, new dynamically unique lisp objects are created for SELF
and EVENT (and for other values of type :ID that're returned by
methods that this method calls.)  Registering each of those objects
for termination and calling #/retain on the ObjC object that they
refer to would keep the ObjC objects that're referred to from being
deallocated while referenced (and that's certainly desirable), but the
cost of doing this is extremely high.  If the method is called
thousands of times between GCs, then the view will get retained
thousands of times and thousands of lisp pointers to thousands of
(otherwise short-lived) NSEvents will be retained (interfering with
things like #/autorelease).  I don't think that I'd want to ever move
the mouse if that meant going through a largely pointless cycle of
registering for termination, #/retain, GC, TERMINATE, #/release.

All of those lisp pointers getting allocated on method invocation
already cost something (GC-wise), but that cost is fairly small: the
GC -likes- short-lived garbage, it's the non-garbage that causes work
to be done. (Finalization/termination causes work to be done on what
would otherwise be garbage.)  One way of reducing the method
invocation (memory-allocation) overhead is to declare that parameters
(and other pointers) have DYNAMIC-EXTENT.  Though this requires some
care, it can significantly reduce memory-allocation overhead (things
that are declared to have DYNAMIC-EXTENT can be stack-allocated, and
it's meaningless to talk about finalizing something that was
stack-allocated.

The example above (an event handler that may be called very
frequently) may be an extreme case, but I'm pretty much convinced that
there's no way to make the approach that you describe viable.  I'm
fairly sure that it was implemented in CCL - with some obvious
improvements and refinements - several years ago, and that it pretty
clearly wasn't viable.  Machines are bigger and faster now, but the
proportional cost of making great numbers of otherwise short-lived
objects finalizable is likely about the same as it was.  (One of the
refinements involved the notion of "canonicalization" - mapping ObjC
references to the same lisp pointer object - and IIRC it was the
act of introducing a new canonical pointer that did the #/retain and
TERMINATE-WHEN-UNREACHABLE.  There were some problems with that - the
hash table used for canonicalization had some of the same issues as
the slot-vector hash table does - but the real problem was that it
makes much more sense to retain a canonicalized pointer to an NSView
than it does to retain a canonicalized pointer to an autoreleased NSEvent,
and I saw no way of automating the process of distinguishing between those
cases.)

If you conclude that you can't do this in at least some methods
(extreme cases like event handlers), step 1 of your description
changes from "as soon as a lisp reference ..." to "as soon as some
references ...", and the advantages of the scheme are greatly
diminished.  If some (alleged) lisp references to ObjC objects were
guaranteed safe and others weren't (and it was hard to know/remember
which were which), that's not different enough from the status quo
to accept some pervasive overhead.