[Openmcl-devel] make-record malloc/free question

[Gary Byers]

On Thu, 19 Aug 2004, Cyrus Harmon wrote:

>
> So I see the malloc call in make-record in macros.lisp. Is there a
> corresponding free call somewhere? The reason I ask is that I'm trying
> to (see previous posts) use the MOP to automagically wrap foreign
> structures to objects with slots and I can see where to malloc storage
> for internal buffers and the like, but I can't see the MOP hooks for
> free, which got me wondering how make-record does it. I suppose all of
> this works for stack allocated memory, but I don't see how things that
> get malloc'ed in make-record get freed.
>
> Thanks,
>
> Cyrus
>

In contrast to a Lisp object (with a small "o", so this includes
cons cells, strings, heap-allocated numbers, CLOS instances, functions ...),
a heap-allocated foreign object lives until it's explicitly freed.  (The
lisp objects live until the GC can prove that nothing references them.)

The expectation is that something allocated with MAKE-RECORD (or malloc)
will eventually be freed "when you're done with it and when nothing else
references it."  For a large, complicated system, it can be very difficult
to determine when these conditions hold.

The GC can help a little (sort of ...); you can ask it to tell you when
a particular MACPTR is about to become garbage.  That's not quite as
helpful as it might seem: it's not quite the same as knowing that the
block of memory that the MACPTR encapsulates is unreferenced.

If the address of that block of memory has been passed to foreign code,
it's hard to know in general whether that code has retained that pointer
somehow.  (Most foreign functions don't do this, but it's not hard to
think of exceptions.)

If you have a scheme where a CLOS instance "encapsulates" a foreign
pointer, you don't want those two things (the encapsulating lisp object
and the underlying foreign block) to have different lifetimes.  You
may be able to ensure this if you're willing to enforce a few constraints:

a) foreign code doesn't retain references to the malloc'ed memory object
b) lisp code doesn't reference that malloc'd memory outside of the
   encapsulating class
c) nothing (neither foreign nor lisp code) #_frees the memory behind
   your back.

If all of those things hold, you can at lease ensure that the foreign
pointer is deallocated when the encapsulating object is about to become
garbage, via a scheme like:

(defclass encapsulating-object (whatever)
  ((foreign-pointer ....)))

(defmethod initialize-instance :after ((x encapsulating-object) &rest initargs)
  (ccl:terminate-when-unreachable x))

(defmethod ccl:terminate ((x encapsulating-object))
  (with-slots (foreign-pointer)
    (when foreign-pointer
      (#_free foreign-pointer)
      (setq foreign-pointer nil))))

The CCL:TERMINATE method will be called on some arbitrary thread
sometime (hopefully soon) after the GC has decided that there are no
strong references to an object which has been the argument of a
CCL:TERMINATE-WHEN-UNREACHABLE call.

If it makes sense to say that the foreign object should live as long
as there's lisp code that references it (through the encapsulating
obect) and no longer, this is one way of ensuring that.

(What OpenMCL calls "termination" is essentially the same as what
Java and other languages call "finalization".)