[Openmcl-devel] sharing data between fortran and OpenMCL

[Cyrus Harmon]

Ok, this was a while back, but I'm trying to pick up where I left off. 
I'll leave the whole thread attached below, but basically you said:

> After many years of being the bad cop, I'm willing to make a deal:
> I'll add support for "casually passing lisp data to foreign code",
> as long as someone else - preferably people who really want this
> functionality - volunteer to debug the subtle and infrequent GC
> problems that'd arise.

Ok, I'll do my best to debug this. What do I need to do? I know that I 
can use the make-heap-ivector, but for arbitrary lisp objects, what 
needs to happen? Clearly this will happen in some sort of without-gcing 
block.

On a similar note, forgive me if we've discussed this before, but could 
the without-gcing block take a list of lisp objects that would 
basically be "locked" allowing other gc actions to take place or is 
this just a bad idea?

Thanks again,

Cyrus

On Sep 30, 2004, at 2:06 PM, Gary Byers wrote:

>
>
> On Thu, 30 Sep 2004, Cyrus Harmon wrote:
>
>> Ok, this is probably a mistake, but I've been poking around the
>> internals of OpenMCL. I'm trying to port matlisp to OpenMCL and have
>> run into a couple issues.
>>
>> I think the core of what I need to do is make the following work under
>> OpenMCL:
>>
>>        (if (typep vec '(simple-array * (*)))
>> 	  (sb-sys:vector-sap vec)
>> 	  (let ((base-address
>> 		 (the (unsigned-byte 32)
>> 		   (logandc1 7 (sb-kernel:get-lisp-obj-address vec)))))
>> 	    (declare (type (unsigned-byte 32) base-address))
>> 	    (sb-sys:int-sap
>> 	     (etypecase vec
>> 	       ((complex double-float)
>> 		(the (unsigned-byte 32) (+ 8 base-address)))
>> 	       ((complex single-float)
>> 		(the (unsigned-byte 32) (+ 4 base-address)))
>> 	       ((simple-array * *)
>> 		;; A multidimensional simple-array
>> 		(let ((data-vector
>> 		       (logandc1 7 (sb-sys:sap-ref-32
>> 				    (sb-sys:int-sap (+ base-address 16))
>> 				    0))))
>> 		  (the (unsigned-byte 32) (+ data-vector 8))))))))))
>>
>>
>> Obviously, the sb-sys stuff doesn't directly apply. The main issues I
>> think I'm facing are figuring out how to get the address of single and
>> double complex's and to get access to the data in an array. I think I
>> can do this last part with
>>
>> %vect-data-to-macptr
>
>
> Lisp data can be moved around in memory by the GC.
> Because lisp threads are preemptively scheduled, a GC can occur
> between any two instructions.
> It is therefore meaningless to talk about taking the address of
> a lisp object (in general; it would work for things that were
> stack-allocated, or allocated via %MAKE-HEAP-IVECTOR, and it might
> work a very high percentage of the time otherwise - the GC doesn't,
> after all, run on -most- instruction boundaries.)
>
> Suppose that you said "Aha! all that we need to do to avoid crashing
> mysteriously is to surround some of this code with a WITHOUT-GCING.
> If the GC can't run, the lisp objects can't move around, and it'd
> be safe to pass their addresses to foreign code."  That's true; if
> you know exactly how long foreign code is going to hang on to those
> addresses, you know where the close-paren for the WITHOUT-GCING form
> should go; if you know what other threads in your application are
> doing, you know enough to evaluate the cost of disabling the GC
> so that this could work reliably.
>
> The reasons that I'm always the bad cop about this sort of stuff
> are historical: early versions of MCL were very casual about allowing
> the equivalent of SB-VECTOR-SAP/%VECT-DATA-TO-MACPTR, and a lot of
> system code dependend on this functionality.  Very early versions of
> MCL were single-threaded and callbacks could only run if certain
> well-known foriegn functions were called, so this was generally
> safe.  As things got more complicated - even before MCL had threads,
> it had an interrupt-driven event system - weird, irreproducible GC
> related crashes started to occur.  Very often - after hours or days
> of debugging - it was revealed that it was our own code's dependence
> on things like %VECT-DATA-TO-MACPTR that led to the problem.
>
> After many years of being the bad cop, I'm willing to make a deal:
> I'll add support for "casually passing lisp data to foreign code",
> as long as someone else - preferably people who really want this
> functionality - volunteer to debug the subtle and infrequent GC
> problems that'd arise.
>
>>
>> but I can't figure out how to get access to the single-floats or
>> double-floats in a complex. Also, are any of these internal guts
>> documented? I've just been poking around in level-0/PPC and
>> compiler/PPC trying to find bits that look like they might be useful.
>
> It's over 4 years old (and therefore inaccurate in many details), but
> I think that anyone who wanted to try to understand some of the code
> in the */PPC directories would find Chapter 11 of the documentation a
> helpful starting point.  Most of the actual magic numbers that define
> things like "how many bytes away from a tagged pointer to a
> DOUBLE-FLOAT does the data start?" are defined in
> "ccl:compiler;PPC;arch.lisp" and in "ccl:compiler;PPC;PPC32;arch.lisp".
>
>> Any further suggestions?
>>
>> Thanks,
>>
>> Cyrus
>>
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>>
>>