[Openmcl-devel] Memory managemet Objective C and Clozure

[Gary Byers]

On Sat, 18 Sep 2010, Willem Rein Oudshoorn wrote:
> attempts on integrating (from reading the source.)
>
> This subject needs some contemplation on my part, before I can even
> really formulate my questions accurately.  However in the mean time let
> me offer the followng 'Bug'
>
>
> Inconsistent Memory Management
> ------------------------------
>
> Consider the following two class definitions:
>
> (defclass test-objc-class-a (ns:ns-object)
>  nil
>  (:metaclass ns:+ns-object))
>
> (defclass test-objc-class-b (ns:ns-object)
>  ((test-var :accessor test-var))
>  (:metaclass ns:+ns-object))
>
> Now do the following:
>
> (progn (ccl:terminate-when-unreachable (make-instance 'test-objc-class-a))
>        nil)
>
> (progn (ccl:terminate-when-unreachable (make-instance 'test-objc-class-b))
>         nil)
>
> (progn (make-instance 'test-objc-class-a) nil)
>
> (let* ((instance (make-instance 'test-objc-class-b)))
>      (ccl:terminate-when-unreachable instance)

In other words, "release this thing later, when lisp is about to GC this
pointer object."

>      (#/release instance)

In other words ... release it now.

>      nil)
>
>
> (gc)
>
> It happens that:

The instance gets released twice; the first ("manual") call to #/release
deallocates it, which means that the second #/release in the terminate
method operates on a deallocated object, which means that it's one of:

   - free memory
   - some other, unrelated object that happened to have been allocated at
     the same address
   - a pointer into the middle of some other, unrelated object that spans
     that address.

Whatever happens is likely to be bad.

I don't know whether this is the result of a misunderstanding or just
a simple oversight.  (Ah.  After proper caffeination, it seems that
you're trying to suggest that there's some way that this could possibly
work and that the fact that it doesn't is a bug.  Is that correct ?)

>
> 1 -  the first instance of test-objc-class-a is garbaged collected,
> 2 -  and the instance of test-objc-class-b is NOT garbage collected.
> 3 -  the last instance of test-objc-class-a is garbaged collected, but
>     leaks memory.
> 4 -  This leads to undefined behaviour, most likely crashing in the
>     objective-C runtime.
>
>
> It took me a while to figure this out, but
>
> (a) There is a terminae methd on instances of ns-object
> (b) This terminate message sends a #/release
> (c) Per default the terminate method is not activated.  Hence
>    results 1 and 3
> (d) The fact that test-objc-class-b has lisp slots, means that the
>    instance is a key in the *objc-object-slot-vectors* table
>    and therefore will not be garbage collected.  Explaining 2.
> (e) the fact that the #/dealloc method is implicitly overriden,
>    which removes all the lisp slots from the hashtable makes
>    the object gc-able and therefore a new #/release message
>    will be send to the already 'dealloced' object.

I think that there's a simpler explanation for the failure you
likely see here, but yes: there are all kinds of ways to lose here.
I don't see that there's any practical way for #/release to know
that it should cancel termination, or for the terminate method
to know that the object's already been deallocated.

[Incidentally, one context in which TERMINATE-WHEN-UNREACHABLE
is useful is to keep an object from being deallocated when there's
some canonical lisp reference to it.  An NSWindow is ordinarily
released  when it's closed, so if you do something like:

(defvar *w* (make-instance 'ns:ns-window ...))

and the window's closed, the the pointer that's the value of *W*
will point at a deallocated object.  One way to avoid this is to
do:

(defvar *w* (let* ((w (#/retain (make-instance 'ns:ns-window ...))))
               (terminate-when-unreachable w)
               w))

If the window's closed when the value of *w* is still reachable, it
won't be deallocated (so you you could make it visible again or otherwise
treat it as a real live NSWindow).  If you eventually do something like:

(setq *w* nil)

and there are no other (EQ) references to the particular pointer that's
registered for termination, the terminate method will release the NSWindow.]

Back to your example (which I tend to dismiss as something that can't
work): in the real world, you might decide that you want to manually
release something that's been scheduled for termination (perhaps because
it's desirable to free up some resources as soon as possible, rather than
waiting for the GC to eventually arrange to call TERMINATE.)  The idiom
to use in that case is:

   (ccl:cancel-terminate-when-unreachable *w*)
   (#/release *w*)

>
> To me it seems quite confusing.  I need to think a bit more about
> what I expect and if this is even a bug.
> But it seems to me that the *objc-object-slot-vectors* should
> be a hashtable with weak keys.

A weak hash table can keep track of whether a particular lisp object
is otherwise referenced.  It gets confusing, but it's important to
distinguish between a foreign address and the lisp object (MACPTR)
that encapsulates it.  An arbitrary number of encapsulating lisp
objects can reference the same address; distinct lisp pointer objects
to the same address are EQL but not EQ.

A weak hash table could tell us when an encapsulating lisp object
becomes garbage, but that isn't what we want to know in this and
similar cases: the slot-vector should exist as long as the ObjC
object exists.  (If all encapsulating lisp pointers to the ObjC
object became garbage, we can't safely remove the slot-vector because
some  might create a new encapsulating pointer to the object or receive
such a pointer as an argument)

Even if all pointers to the same address were EQ, that wouldn't solve
the problem.  What we really want is to maintain the mapping between
an ObjC instance and a lisp slot vector as long as that instance
exists.  The address of that instance will exist long after the
instance (and we ...) are gone: it's only meaningful to use the address
of the object to refer to the object until the object is deallocated.

There have been halfhearted schemes to treat
*objc-object-slot-vectors* as some sort of weak (in the lisp sense)
mapping, and they don't work (or aren't viable) for the reasons
outlined above.  What's currently done (and has only been implemented
for the last month or so) is to automatically generate #/dealloc
methods for classes whose instances have lisp slots and to have that
method REMHASH the slot vector associated with the instance.  Unlike
other approaches, that seems to actually work: the mapping between
instances and slot-vectors dissolves at essentially the point where
the instance ceases to exist.  That point isn't the same as (or even
closely related to) the point at which a lisp pointer to the object
is about to become garbage.

>
> Kind regards,
> Wim Oudshoorn.
>
>
>
>
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>