[Openmcl-devel] Objc Mem Management Question

[Gary Byers]

An infinite number of unique (non-EQ) MACPTRs can refer to the same address.

? (let* ((x (%int-to-ptr #x1234))
          (y (%int-ot-ptr #x1234)))
     (values (eq x y) (eql x y)))

That's likely to return (values NIL T).

Like much of the CCL documentation, the section you quote needs to be updated.

The bridge maintains an EQL hash table which maps MACPTRs (presumably the addresses of some
ObjC instances) to data structures (ccl::slot-vectors) that hold the values of the instance's
lisp slots.  When an ObjC class that has (direct) lisp slots is defined, a #/dealloc method
for instances of that class is automatically defined; that #/dealloc method removes the entry
from the hash table and calls the next method.  If you have reason to override this #/dealloc
method, you're responsible for calling the magic function that removes the entry; this is
plainly documented ... in the release notes for some CCL version a few years ago.

As long as that discipline is followed, the lisp-slots-on-foreign-object mechanism works fairly
well.  It doesn't depend on lisp knowing anything about the lifetimes of ObjC objects (and in
general lisp doesn't know anything about ObjC object lifetimes.)

The GC can arrange to call an object's TERMINATE method when the (lisp) object is about to become
garbage.  The lifetime of a particular pointer to an address (potentially one of many EQL pointers
to that address) has nothing to do with the lifetime of an ObjC object at that address.

If we take ObjC out of the picture for a moment:

? (defvar *p* (#_malloc 20))
*P*
? *p*
#<A Foreign Pointer #x7F66F4005DF0>

We can think of *p* as being "a pointer to 20 bytes of allocated memory", but the lisp
doesn't really know anything about "allocated memory", and it's actually very hard to
reliably tell that *p* is pointing at "allocated memory".

If we change that:

? (#_free *p*)
NIL

*P* is pretty much the same thing that it was before:

? *p*
#<A Foreign Pointer #x7F66F4005DF0>

only now it's more dangerous to store into it or even reference memory relative to it.

If we bring ObjC back into the equation, we can tell (based on heuristics that work very well in practice)
that a pointer to a given address is a "pointer to an ObjC instance".  We cache that information (when
a MACPTR references an address that appears to be an ObjC instance, we set some bits in the MACPTR that
caches that information so that we don't have to go through those heuristices again.

We do something similar with pointers created by RLET or MAKE-RECORD:

? (make-record :stat)
#<A Foreign Pointer (:* (:STRUCT :STAT)) #xF7102710>

It's kind of useful to see those bits (bits that say "this MACPTR is a pointer to a (:STRUCT :STAT)"
for debugging.  But:

? (#_free *)
NIL
? **
#<A Foreign Pointer (:* (:STRUCT :STAT)) #xF7102710>

Nothing clears those bits that assert that the pointer's pointing to a particular structure type.

In the ObjC case, it's a little worse than that: if the address that a MACPTR points to seems to
contain an ObjC instance, then we assert that the pointer ... points to an ObjC instance.

? (defvar *p* (make-instance 'ns:ns-view))
*P*
? *p*
#<NS-VIEW <NSView: 0x1c3c60> (#x1C3C60)>
? (defvar *q* (%int-to-ptr #x1C3C60))
*Q*
? *q*
#<NS-VIEW <NSView: 0x1c3c60> (#x1C3C60)>
? (eq *p* *q*)
NIL
? (eql *p* *q*)
T

Notice that *Q* is (at one level) just a MACPTR - created via %INT-TO-PTR - but we can also treat
it as "an instance of the class NS:NS-VIEW", and we can pass *Q* or *P* to any ObjC or CLOS method
specialized on that class.

This all works reasonably well (up to a point): we can treat MACPTRs to addresses that contain ObjC
instances as if they were first-class CLOS objects.  (Kinda. Mostly.)  It breaks down when the ObjC
instance is #/dealloc-ated.  What we might want to happen is for all MACPTRs to the address of the
instance to have their class changed from (e.g.) NS:NS-VIEW to "generic MACPTR", but I'm not sure
that this is practical.  (There are a few other ways of formulating what we might want; all of the
ways that I've thought of seem equally impractical.)  Actually, I just thought of something that
might be practical, maybe.

This is too long already and I'm starting to (uh, maybe more than starting to) ramble.  It's
only partly responsive to your questions, but I think that your model of some of what goes on
isn't quite accurate and if this is at all intelligible hopefully it'll help you change that
model.





On Tue, 5 Feb 2013, Paul Krueger wrote:

> Section 14.5.2 of the CCL doc which talks about defining Objc subclasses with lisp slots says in part:
>
> "As one might expect, this has memory-management implications: we have to maintain an association between a MACPTR and a set of lisp objects (its slots) as long as the Objective-C instance exists, and we have to ensure that the Objective-C instance exists (does not have its -dealloc method called) while lisp is trying to think of it as a first-class object that can't be "deallocated" while it's still possible to reference it. Associating one or more lisp objects with a foreign instance is something that's often very useful; if you were to do this "by hand", you'd have to face many of the same memory-management issues."
>
> I've always depended on this behavior and never had any problems. Recently I ran into a problem when I defined something that was morally equivalent to the following test case:
>
> (defclass thing (ns:ns-view)
>  ((slot :accessor slot
>         :initform nil))
>  (:metaclass ns:+ns-object))
>
> (defmethod initialize-instance :after ((self thing)
>                                       &key
>                                       &allow-other-keys)
>  (ccl:terminate-when-unreachable self))
>
> (defmethod ccl:terminate ((self thing))
>  (format t "terminate called for thing: ~s" self))
>
> (defun test-thing ()
>  (let ((v (make-instance 'ns:ns-view))
>        (th (make-instance 'thing)))
>    (#/addSubview: v th)
>    v))
>
>
> Then in the listener I do the following:
>
>
> ? (setf v (test-thing))
> #<NS-VIEW <NSView: 0x28379140> (#x28379140)>
> ? (gc)
>
> And the message from the terminate method is displayed in the AltConsole app window.
>
> I'm trying to piece together how this all works without spending a lot of time looking through gc code, so I'd appreciate corrections to the following:
>
> 1. When we create an instance of thing a macptr is created that points to the ObjC instance and some sort of association is kept between that macptr and the lisp slot associated with that instance of thing. The reference count of the thing instance at that point is 1.
>
> 2. When the thing view is added to another view, its reference count is incremented by the parent view, so it is now 2. In theory I should be able to safely #/release the thing instance at this point although I did not actually do so in this example.
>
> 3. Since there is no longer any other lisp reference to the macptr it becomes a candidate to be gc'ed.
>
> Now what I sort of expected at that point is that gc code would check the ref count of the object referred to by the macptr and if it is > 0 it would NOT gc the associated lisp slots and NOT call terminate for the thing instance. I've never before encountered any problems with the first assumption (e.g. lisp slots defined for custom Objective-C subclass instances always seem to hang around as long as the object is not dealloc'ed, without any concern to keep another reference to it from the lisp world). So is it possible that terminate is being called prematurely (before the ref count check is made) or is my understanding / expectation flawed and I've just been very lucky?
>
> Note also that if I DO keep around another Lisp reference to that thing instance, then the terminate method is not called, as you would expect.
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>