[Openmcl-devel] CCL crash on windows (with multithreading and ffi callbacks)

[Gary Byers]

I'd recommend debugging.

Is the callback (locking-callback) called ?  Do its arguments look plausible ?
Do BT:ACQUIRE-LOCK and BT:RELEASE-LOCK do the right thing(s) ?  If things
seem to work when this callback isn't installed and don't when it is, then one
could suspect either the mechanics of the callback or the code it calls; if
(hypothetically) BT:ACQUIRE-LOCK and BT:RELEASE-LOCK didn't work, then it
wouldn't be too surprising if a multithreaded application that relied on those
things working didn't work.

If the "mechanics of the callback" - receiving arguments from and returning
results from foreign code - were at fault, then that is something that CCL
(and to some extent CFFI) is responsible for and a problem there would 
almost certainly be a bug in CCL.  The callbacks that seem to be involved
don't -look- too unusual, but one never knows.

If someone isolates the problem as a  CCL bug, then I'd certainly be interested
in trying to fix it.  It's potentially a lot of work just to isolate the problem;
I wish that I coul say (well, sort of wish ...) that I had time and interest
in doing that, but I quite frankly have neither.  (I don't even know where
things like QuickLisp put the sources to the systems that it downloads, and I
don't have the attention span or patience or whatever to learn that.)

It sounds like you've already done quite a bit to narrow this down;
adding a few calls to BREAK or PRINT or FORMAT in appropriate places
might do a lot to help isolate the problem to the point where someone
could actually do something about it.


On Mon, 2 May 2011, Anton Vodonosov wrote:

> Hello.
>
> Several people, including me, experience 100% reproducible CCL crash on Windows.
>
> This involves multi-threading and FFI callbacks.
>
> To reproduce this, you may assess hunchentoot via SSL (please use the recent CL+SSL from CVS).
>
> The hunchentoot SSL connector uses OpenSSL via FFI. This includes providing a callback function
> to OpenSSL. See call to the crypto-set-locking-callback in the ffi.lisp in the CL+SSL sources.
>
> The crash happens, doesn't matter what code is inside of the callback. It may even be empty.
> But if we don't pass the callback - the crash does not occur.
>
> On Linux the crash doesn't happen. Also, on Windows, CLISP and Allegro CL doesn't crash.
>
> For your convenience, below is the code to start hunchentoot with SSL. Ready to use certificate and key files
> are also attached.
>
> (pushnew "C:\\Users\\anton\\unpacked\\lisp-libs\\cl+ssl\\"
>          asdf:*central-registry*
>          :test #'equalp)
>
> (ql:quickload :hunchentoot)
>
> (defparameter *keyfile* "C:\\Users\\anton\\unpacked\\lisp-libs\\cl+ssl\\test-cert\\key.pem")
> (defparameter *certfile* "C:\\Users\\anton\\unpacked\\lisp-libs\\cl+ssl\\test-cert\\certificate.pem")
>
>
> (hunchentoot:define-easy-handler (say-yo :uri "/yo") (name)
>  (setf (hunchentoot:content-type*) "text/plain")
>  (format nil "Hey~@[ ~A~]!" name))
>
>
> ;; hunchentoot logging settings
> (setf hunchentoot:*access-log-pathname* "C:/tmp/hunchentoot-access.log"
>      hunchentoot:*message-log-pathname* "C:/tmp/hunchentoot-message.log")
>
> (defparameter *srv*
>  (hunchentoot:start (make-instance 'hunchentoot:ssl-acceptor
>                                    :port 8181
>                                    :ssl-certificate-file *certfile*
>                                    :ssl-privatekey-file *keyfile*
>                                    ;; if you want to test a single-threaded case
>                                    ;;:taskmaster (make-instance 'hunchentoot:single-threaded-taskmaster)
>                                    )))
>
> (format t "Started. Server: ~A~%" *srv*)
>
> This is the command line for the ApacheBench program on windows to query the
> hunchentoot:
>
>   C:\Progra~2\Apache~1\Apache2.2\bin\abs -f SSL2 -n 1000 -c 14 -v 1 https://127.0.0.1:8181/yo?name=Dude
>
> But the crash is quite easy reproduced from browser (I noticed, sometimes it happens not immediately
> upon a request, but when you close the browser after received response).
>
> My platform: Windows7 64bit, ccl-1.6-windowsx86  (also tried with ccl-trunk).
>
> BTW, today I experienced another CCL crash (the same platform). I worked from slime, and run
> a multi threaded test of my application. Probably it is a different problem, because it doesn't involve
> any FFI, while the above test case doesn't crash if we don't pass an FFI callback.
> I can't provide an isolated test case for the second problem. If you are interested to work on this,
> the hunchentoot use case is easier to reproduce.
>
> More links:
> - If you need to know, why OpenSSL requires the callback:
>  http://www.openssl.org/docs/crypto/threads.html
>
> Best regards,
> - Anton