[Openmcl-devel] reloading/unloading shared libraries?

[Gary Byers]

On Mon, 9 Feb 2004, Peter Seibel wrote:

> ISTR seeing something about this on the web somewhere but I couldn't
> find it again. So I'll ask here: what are the semantics of reloading a
> shared library with CCL:OPEN-SHARED-LIBRARY? I.e. compile foo.so, open
> it with o-s-l, recompile, open it again.
>
> -Peter
>
> --
> Peter Seibel                                      peter at javamonkey.com
>
>          Lisp is the red pill. -- John Fraser, comp.lang.lisp
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>

There are two types of shared libraries in common use on OSX:

- "dylibs" are primarily designed to be linked against (so they're
   typically opened by the dynamic linker when an application starts
   up.)  It's possible to load them dynamically into a running image
   (and CCL:OPEN-SHARED-LIBRARY supports that), but the OS provides
   no way to close/unload them.

- "bundles" (not to be confused with OSX directories that contain
   related resources and are also called "bundles") more nearly
   match the notion of a dynamic library found in OSes that have
   not been living in a cave for 15 years or so.

The unix "file" command can distinguish between these two types
of shared libraries (there are different bits in the Mach-O file
header.)

It's possible to call OPEN-SHARED-LIBRARY on the same dylib multiple
times in the same session; that works because of a check that avoids
trying to re-load a library that's already been loaded.  "dylib"s
aren't really designed for dynamic use, so recompiling a dylib that's
already been opened in lisp will not cause the new version to be
loaded.

The OS is more than happy to let you open a bundle multiple times.
It's so happy, in fact, that it allows you to have multiple instances
of the same library loaded at the same time, and it's generally hard
to predict in which of a set of loaded bundles an external symbol
will be found (I -think- that it's the first open library to have
been loaded, but I don't know that one can reliably count on this.)
It's hard in general for the lisp to keep track of which bundles
have already been loaded, since the OS doesn't always tell it
where the bundle was found.

I think that you can do what you want to do by:

- using bundles instead of dylibs.

- try to ensure that a bundle's loaded at most once.

- when it's time to test a new version of a bundle, call
  CLOSE-SHARED-LIBRARY to cause all "external entry points" associated
  with the library to become unresolved.  Loading a new version of the
  library should cause the "external entry points" of those foreign
  symbols to associate themselves with the new library instance.

This should work for things like (external "_foo") and (external-call
"_foo" ...), which are sort of designed around the notion that a
foreign symbol's address can change.