[Openmcl-devel] An unfortunate interaction between CFFI and open-shared-library

[Ron Garret]

Today I tried to load CL-MYSQL which I have done a zillion times before.  To my dismay, it failed with the following error on the altconsole:

> Error: Error opening shared library "libmysqlclient_r.dylib": dlopen(libmysqlclient_r.dylib, 10): image not found
> While executing: (:INTERNAL GUI::|-[LispApplication sendEvent:]|), in process Initial(0).

This despite the fact that I hadn't changed anything since the last time it worked, except for updating CCL itself.

The cause seems to be a recent change in open-shared-library:

(defun open-shared-library (name &optional (process #+darwin-target :initial
                                                    #-darwin-target :current))
  "If the library denoted by name can be loaded by the operating system,
return an object of type SHLIB that describes the library; if the library
is already open, increment a reference count. If the library can't be
loaded, signal a SIMPLE-ERROR which contains an often-cryptic message from
the operating system."
  (if (or (eq process :current)
          (eq process *current-process*)
          (and (eq process :initial)
               (eq *current-process* *initial-process*)))
    (open-shared-library-internal name)
    (call-in-process (lambda () (open-shared-library name))
                     (if (eq process :initial)
                       *initial-process*
                       process))))

It appears that open-shared-library when run on Darwin runs on the main thread rather than the calling thread.  But CFFI wraps this call in a handler-case, which it relies on to find the library:

(defun load-foreign-library-path (name path)
  "Tries to load PATH using %LOAD-FOREIGN-LIBRARY which should try and
find it using the OS's usual methods. If that fails we try to find it
ourselves."
  (handler-case
      (%load-foreign-library name path)
    (error (error)
      (if-let (file (find-file path *foreign-library-directories*))
              (handler-case
                  (%load-foreign-library name (native-namestring file))
                (simple-error (error)
                  (report-simple-error name error)))
              (report-simple-error name error)))))

Now that open-shared-library is running in a different thread it is no longer in the dynamic context of the handler-case, so it dies if it can't find the library on the first try.

So... is there a reason that open-shared-library is dispatching to the initial thread?  If there isn't, it really shouldn't.  And if there is, well, then Houston, we have a problem.

rg