[Openmcl-devel] Objc Mem Management Question

Thu Feb 7 09:14:17 PST 2013

Thanks for the explanation; it made perfect sense. My takeaways from all this are:

1. The automatically created dealloc method for lisp-defined objective-C subclasses that contain lisp slots works just fine for keeping those slots around for the life of the instance. My experience told me that this had to be occurring and now I understand that mechanism.

2. It's not practical to use the terminate-when-unreachable mechanism on any class instance derived from Objective-C because in fact that applies to the macptr that points to it rather than the allocated instance itself. The obvious work-around is to do anything that you wanted to do in a terminate method in a custom dealloc method instead. At first I was thinking that maybe you could create a reference to that initial macptr (just as you do now for lisp slots in objc subclasses) so that it would only be gc'ed when the underlying instance went away (the idea being to make the terminate method work more like it does for lisp objects). But then I realized that if there happened to be another reference to that macptr around somewhere, you might end up calling the terminate method AFTER the dealloc was completed, which wouldn't generally be very good. That also makes the notion that I had in my original email a pretty stupid idea. So all things considered, I'll start doing anything like this in a dealloc method. It doesn't happen that often anyway.

3. If a dealloc method is created, then to avoid memory leaks any instance with lisp slots must call (objc:remove-lisp-slots self) in addition to call-next-method. For others, this is documented in the 1.6 release notes: http://trac.clozure.com/ccl/wiki/ReleaseNotes/1.6. The argument to use in the call isn't documented anywhere that I saw, but a quick search uncovered the code that generates the default dealloc routine which made it pretty obvious.

This all seems to be workable.

On Feb 7, 2013, at 2:52 AM, Gary Byers <gb at clozure.com> wrote:

> An infinite number of unique (non-EQ) MACPTRs can refer to the same address.
> 
> ? (let* ((x (%int-to-ptr #x1234))
>         (y (%int-ot-ptr #x1234)))
>    (values (eq x y) (eql x y)))
> 
> That's likely to return (values NIL T).
> 
> Like much of the CCL documentation, the section you quote needs to be updated.
> 
> The bridge maintains an EQL hash table which maps MACPTRs (presumably the addresses of some
> ObjC instances) to data structures (ccl::slot-vectors) that hold the values of the instance's
> lisp slots.  When an ObjC class that has (direct) lisp slots is defined, a #/dealloc method
> for instances of that class is automatically defined; that #/dealloc method removes the entry
> from the hash table and calls the next method.  If you have reason to override this #/dealloc
> method, you're responsible for calling the magic function that removes the entry; this is
> plainly documented ... in the release notes for some CCL version a few years ago.
> 
> As long as that discipline is followed, the lisp-slots-on-foreign-object mechanism works fairly
> well.  It doesn't depend on lisp knowing anything about the lifetimes of ObjC objects (and in
> general lisp doesn't know anything about ObjC object lifetimes.)
> 
> The GC can arrange to call an object's TERMINATE method when the (lisp) object is about to become
> garbage.  The lifetime of a particular pointer to an address (potentially one of many EQL pointers
> to that address) has nothing to do with the lifetime of an ObjC object at that address.
> 
> If we take ObjC out of the picture for a moment:
> 
> ? (defvar *p* (#_malloc 20))
> *P*
> ? *p*
> #<A Foreign Pointer #x7F66F4005DF0>
> 
> We can think of *p* as being "a pointer to 20 bytes of allocated memory", but the lisp
> doesn't really know anything about "allocated memory", and it's actually very hard to
> reliably tell that *p* is pointing at "allocated memory".
> 
> If we change that:
> 
> ? (#_free *p*)
> NIL
> 
> *P* is pretty much the same thing that it was before:
> 
> ? *p*
> #<A Foreign Pointer #x7F66F4005DF0>
> 
> only now it's more dangerous to store into it or even reference memory relative to it.
> 
> If we bring ObjC back into the equation, we can tell (based on heuristics that work very well in practice)
> that a pointer to a given address is a "pointer to an ObjC instance".  We cache that information (when
> a MACPTR references an address that appears to be an ObjC instance, we set some bits in the MACPTR that
> caches that information so that we don't have to go through those heuristices again.
> 
> We do something similar with pointers created by RLET or MAKE-RECORD:
> 
> ? (make-record :stat)
> #<A Foreign Pointer (:* (:STRUCT :STAT)) #xF7102710>
> 
> It's kind of useful to see those bits (bits that say "this MACPTR is a pointer to a (:STRUCT :STAT)"
> for debugging.  But:
> 
> ? (#_free *)
> NIL
> ? **
> #<A Foreign Pointer (:* (:STRUCT :STAT)) #xF7102710>
> 
> Nothing clears those bits that assert that the pointer's pointing to a particular structure type.
> 
> In the ObjC case, it's a little worse than that: if the address that a MACPTR points to seems to
> contain an ObjC instance, then we assert that the pointer ... points to an ObjC instance.
> 
> ? (defvar *p* (make-instance 'ns:ns-view))
> *P*
> ? *p*
> #<NS-VIEW <NSView: 0x1c3c60> (#x1C3C60)>
> ? (defvar *q* (%int-to-ptr #x1C3C60))
> *Q*
> ? *q*
> #<NS-VIEW <NSView: 0x1c3c60> (#x1C3C60)>
> ? (eq *p* *q*)
> NIL
> ? (eql *p* *q*)
> T
> 
> Notice that *Q* is (at one level) just a MACPTR - created via %INT-TO-PTR - but we can also treat
> it as "an instance of the class NS:NS-VIEW", and we can pass *Q* or *P* to any ObjC or CLOS method
> specialized on that class.
> 
> This all works reasonably well (up to a point): we can treat MACPTRs to addresses that contain ObjC
> instances as if they were first-class CLOS objects.  (Kinda. Mostly.)  It breaks down when the ObjC
> instance is #/dealloc-ated.  What we might want to happen is for all MACPTRs to the address of the
> instance to have their class changed from (e.g.) NS:NS-VIEW to "generic MACPTR", but I'm not sure
> that this is practical.  (There are a few other ways of formulating what we might want; all of the
> ways that I've thought of seem equally impractical.)  Actually, I just thought of something that
> might be practical, maybe.
> 
> This is too long already and I'm starting to (uh, maybe more than starting to) ramble.  It's
> only partly responsive to your questions, but I think that your model of some of what goes on
> isn't quite accurate and if this is at all intelligible hopefully it'll help you change that
> model.
> 
> 
> 
> 
> 
> On Tue, 5 Feb 2013, Paul Krueger wrote:
> 
>> Section 14.5.2 of the CCL doc which talks about defining Objc subclasses with lisp slots says in part:
>> 
>> "As one might expect, this has memory-management implications: we have to maintain an association between a MACPTR and a set of lisp objects (its slots) as long as the Objective-C instance exists, and we have to ensure that the Objective-C instance exists (does not have its -dealloc method called) while lisp is trying to think of it as a first-class object that can't be "deallocated" while it's still possible to reference it. Associating one or more lisp objects with a foreign instance is something that's often very useful; if you were to do this "by hand", you'd have to face many of the same memory-management issues."
>> 
>> I've always depended on this behavior and never had any problems. Recently I ran into a problem when I defined something that was morally equivalent to the following test case:
>> 
>> (defclass thing (ns:ns-view)
>> ((slot :accessor slot
>>        :initform nil))
>> (:metaclass ns:+ns-object))
>> 
>> (defmethod initialize-instance :after ((self thing)
>>                                      &key
>>                                      &allow-other-keys)
>> (ccl:terminate-when-unreachable self))
>> 
>> (defmethod ccl:terminate ((self thing))
>> (format t "terminate called for thing: ~s" self))
>> 
>> (defun test-thing ()
>> (let ((v (make-instance 'ns:ns-view))
>>       (th (make-instance 'thing)))
>>   (#/addSubview: v th)
>>   v))
>> 
>> 
>> Then in the listener I do the following:
>> 
>> 
>> ? (setf v (test-thing))
>> #<NS-VIEW <NSView: 0x28379140> (#x28379140)>
>> ? (gc)
>> 
>> And the message from the terminate method is displayed in the AltConsole app window.
>> 
>> I'm trying to piece together how this all works without spending a lot of time looking through gc code, so I'd appreciate corrections to the following:
>> 
>> 1. When we create an instance of thing a macptr is created that points to the ObjC instance and some sort of association is kept between that macptr and the lisp slot associated with that instance of thing. The reference count of the thing instance at that point is 1.
>> 
>> 2. When the thing view is added to another view, its reference count is incremented by the parent view, so it is now 2. In theory I should be able to safely #/release the thing instance at this point although I did not actually do so in this example.
>> 
>> 3. Since there is no longer any other lisp reference to the macptr it becomes a candidate to be gc'ed.
>> 
>> Now what I sort of expected at that point is that gc code would check the ref count of the object referred to by the macptr and if it is > 0 it would NOT gc the associated lisp slots and NOT call terminate for the thing instance. I've never before encountered any problems with the first assumption (e.g. lisp slots defined for custom Objective-C subclass instances always seem to hang around as long as the object is not dealloc'ed, without any concern to keep another reference to it from the lisp world). So is it possible that terminate is being called prematurely (before the ref count check is made) or is my understanding / expectation flawed and I've just been very lucky?
>> 
>> Note also that if I DO keep around another Lisp reference to that thing instance, then the terminate method is not called, as you would expect.
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>> 
>> 