[Openmcl-devel] sharing data between fortran and OpenMCL

[Gary Byers]

On Thu, 30 Sep 2004, Cyrus Harmon wrote:

> Ok, this is probably a mistake, but I've been poking around the
> internals of OpenMCL. I'm trying to port matlisp to OpenMCL and have
> run into a couple issues.
>
> I think the core of what I need to do is make the following work under
> OpenMCL:
>
>        (if (typep vec '(simple-array * (*)))
> 	  (sb-sys:vector-sap vec)
> 	  (let ((base-address
> 		 (the (unsigned-byte 32)
> 		   (logandc1 7 (sb-kernel:get-lisp-obj-address vec)))))
> 	    (declare (type (unsigned-byte 32) base-address))
> 	    (sb-sys:int-sap
> 	     (etypecase vec
> 	       ((complex double-float)
> 		(the (unsigned-byte 32) (+ 8 base-address)))
> 	       ((complex single-float)
> 		(the (unsigned-byte 32) (+ 4 base-address)))
> 	       ((simple-array * *)
> 		;; A multidimensional simple-array
> 		(let ((data-vector
> 		       (logandc1 7 (sb-sys:sap-ref-32
> 				    (sb-sys:int-sap (+ base-address 16))
> 				    0))))
> 		  (the (unsigned-byte 32) (+ data-vector 8))))))))))
>
>
> Obviously, the sb-sys stuff doesn't directly apply. The main issues I
> think I'm facing are figuring out how to get the address of single and
> double complex's and to get access to the data in an array. I think I
> can do this last part with
>
> %vect-data-to-macptr


Lisp data can be moved around in memory by the GC.
Because lisp threads are preemptively scheduled, a GC can occur
between any two instructions.
It is therefore meaningless to talk about taking the address of
a lisp object (in general; it would work for things that were
stack-allocated, or allocated via %MAKE-HEAP-IVECTOR, and it might
work a very high percentage of the time otherwise - the GC doesn't,
after all, run on -most- instruction boundaries.)

Suppose that you said "Aha! all that we need to do to avoid crashing
mysteriously is to surround some of this code with a WITHOUT-GCING.
If the GC can't run, the lisp objects can't move around, and it'd
be safe to pass their addresses to foreign code."  That's true; if
you know exactly how long foreign code is going to hang on to those
addresses, you know where the close-paren for the WITHOUT-GCING form
should go; if you know what other threads in your application are
doing, you know enough to evaluate the cost of disabling the GC
so that this could work reliably.

The reasons that I'm always the bad cop about this sort of stuff
are historical: early versions of MCL were very casual about allowing
the equivalent of SB-VECTOR-SAP/%VECT-DATA-TO-MACPTR, and a lot of
system code dependend on this functionality.  Very early versions of
MCL were single-threaded and callbacks could only run if certain
well-known foriegn functions were called, so this was generally
safe.  As things got more complicated - even before MCL had threads,
it had an interrupt-driven event system - weird, irreproducible GC
related crashes started to occur.  Very often - after hours or days
of debugging - it was revealed that it was our own code's dependence
on things like %VECT-DATA-TO-MACPTR that led to the problem.

After many years of being the bad cop, I'm willing to make a deal:
I'll add support for "casually passing lisp data to foreign code",
as long as someone else - preferably people who really want this
functionality - volunteer to debug the subtle and infrequent GC
problems that'd arise.

>
> but I can't figure out how to get access to the single-floats or
> double-floats in a complex. Also, are any of these internal guts
> documented? I've just been poking around in level-0/PPC and
> compiler/PPC trying to find bits that look like they might be useful.

It's over 4 years old (and therefore inaccurate in many details), but
I think that anyone who wanted to try to understand some of the code
in the */PPC directories would find Chapter 11 of the documentation a
helpful starting point.  Most of the actual magic numbers that define
things like "how many bytes away from a tagged pointer to a
DOUBLE-FLOAT does the data start?" are defined in
"ccl:compiler;PPC;arch.lisp" and in "ccl:compiler;PPC;PPC32;arch.lisp".

> Any further suggestions?
>
> Thanks,
>
> Cyrus
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>