[Openmcl-devel] sharing data between fortran and OpenMCL

[Cyrus Harmon]

On Sep 30, 2004, at 2:06 PM, Gary Byers wrote:

> Lisp data can be moved around in memory by the GC.
> Because lisp threads are preemptively scheduled, a GC can occur
> between any two instructions.
> It is therefore meaningless to talk about taking the address of
> a lisp object (in general; it would work for things that were
> stack-allocated, or allocated via %MAKE-HEAP-IVECTOR, and it might
> work a very high percentage of the time otherwise - the GC doesn't,
> after all, run on -most- instruction boundaries.)

So %make-heap-ivector allocated blocks are guaranteed not to move? 
Perhaps for my particular problem I can ensure that the arrays I need 
to reference are created via %make-heap-ivector or a similar mechanism.

> Suppose that you said "Aha! all that we need to do to avoid crashing
> mysteriously is to surround some of this code with a WITHOUT-GCING.
> If the GC can't run, the lisp objects can't move around, and it'd
> be safe to pass their addresses to foreign code."  That's true; if
> you know exactly how long foreign code is going to hang on to those
> addresses, you know where the close-paren for the WITHOUT-GCING form
> should go; if you know what other threads in your application are
> doing, you know enough to evaluate the cost of disabling the GC
> so that this could work reliably.

Yup. I think I'm OK with disabling GC while I call out to this 
particular fortran stuff.

> The reasons that I'm always the bad cop about this sort of stuff
> are historical: early versions of MCL were very casual about allowing
> the equivalent of SB-VECTOR-SAP/%VECT-DATA-TO-MACPTR, and a lot of
> system code dependend on this functionality.  Very early versions of
> MCL were single-threaded and callbacks could only run if certain
> well-known foriegn functions were called, so this was generally
> safe.  As things got more complicated - even before MCL had threads,
> it had an interrupt-driven event system - weird, irreproducible GC
> related crashes started to occur.  Very often - after hours or days
> of debugging - it was revealed that it was our own code's dependence
> on things like %VECT-DATA-TO-MACPTR that led to the problem.
>
> After many years of being the bad cop, I'm willing to make a deal:
> I'll add support for "casually passing lisp data to foreign code",
> as long as someone else - preferably people who really want this
> functionality - volunteer to debug the subtle and infrequent GC
> problems that'd arise.

I _think_ I really want this. I'm willing to be dissuaded otherwise but 
until then, can't me as a volunteer.

>> but I can't figure out how to get access to the single-floats or
>> double-floats in a complex. Also, are any of these internal guts
>> documented? I've just been poking around in level-0/PPC and
>> compiler/PPC trying to find bits that look like they might be useful.
>
> It's over 4 years old (and therefore inaccurate in many details), but
> I think that anyone who wanted to try to understand some of the code
> in the */PPC directories would find Chapter 11 of the documentation a
> helpful starting point.  Most of the actual magic numbers that define
> things like "how many bytes away from a tagged pointer to a
> DOUBLE-FLOAT does the data start?" are defined in
> "ccl:compiler;PPC;arch.lisp" and in "ccl:compiler;PPC;PPC32;arch.lisp".

That's what I'm looking for. I'll check it out.

Thanks again,

Cyrus