[Openmcl-devel] sharing data between fortran and OpenMCL

[Cyrus Harmon]

On Oct 29, 2004, at 1:37 AM, Gary Byers wrote:

> It's admittedly a little hard to tell, but I think that I was being
> facetious.  If you were to (hypothetically) try to combine:
>
>  - a fully relocating GC
>  - preemptively-scheduled threads, which make it difficult to predict
>    when a GC might happen
>  - the ability to pass the (fleeting, transitory) address of some lisp
>    object(s) to foreign code
>
> you'd likely find that things worked fine a high percentage of the time
> and would fail (possibly with bizarre symptoms, possibly with 
> spectacular
> symptoms) some small percentage of the time.

I get all of this, yet I still think I want the ability to pass the 
address of some lisp object to foreign code. Forgive me if I'm 
repeating myself and please forgive my general cluelessness on lisp 
internals.

> [The point of the FOO example is that it and the practice of passing
> the address of relocatable lisp objects to foreign code can both
> lead to memory-corruption scenarios that're very hard to debug.]

Yeah, but it's not like the relocatable lisp objects are necessary for 
memory-corruption scenarios that are very hard to debug. Sure, the may 
make the problem more likely, but this, is part of life. I agree that 
trying to minimize these scenarios is a good thing.

> Saying "I'll implement this, but someone else gets to debug the 
> problems
> that it'll cause" is intended to mean "it wouldn't be a good idea to
> implement this, and I'm glad that it isn't implemented because 
> debugging
> those problems is often virtually impossible."

Hmm... Ok, I'm willing to consider alternatives. For better or worse, 
I'm attempting to try to port matlisp, which is basically a wrapper 
around the BLAS and LAPACK, and maybe a few other, fortran libraries. I 
don't really want to re-implement all of this nice matrix math stuff in 
lisp. Clearly, there are off-the-shelf packages like MATLAB to build 
their own high level language around this stuff, but I, for hopefully 
obvious reasons, would like to use lisp as that higher-level 
(higher-level than Fortran) language. The matlisp guys have made this 
work for sbcl, cmucl, acl, etc... And I think I'd like to make this 
work for openmcl.

There is another approach, which is a copy from lisp to/from non-lisp 
data structures on every round trip to the fortran code. This would 
presumably avoid (or minimize) the problems we've been discussing, but 
I think this would incur a big performance overhead.

I hate to keep flogging this same horse, but I really would like some 
nice matrix package to use from OpenMCL and I get the feeling that the 
one I've cobbled together on my own is suboptimal.

> If you remember the bouncer-in-the-diner analogy, the compaction
> algorithm (the bouncer shoving diner customers around to make all free
> space contiguous) is simple and linear in the number of active
> customers, and it makes subsequent seating (allocation) trivial.
> Partial compaction/relocation algorithms are certainly possible (see
> the traditional MacOS memory manager), but they're generally slower,
> more complicated, less effective, and complicate allocation (see the
> traditional MacOS memory manager.)
>
> I think that the schemes that would work better are:
>
>  - inside a certain syntactic construct , it'd be legal to use some
>    primitive to obtain the address of (at least certain types of) lisp
>    objects and to pass those addresses to foreign code, with the
>    understanding that those addresses have "dynamic extent" and cease
>    to be valid when the construct exits.

Sounds like what I need.

>  - (harder, but more general): provide for the allocation of lisp
>    objects in one or more "static" memory areas.  In general (there
>    may be room for exceptions), the GC might reclaim the memory used
>    by a statically-allocated object, but it would never move such
>    objects around (and it'd be legal/safe for foreign code to point
>    at static objects, and doing so wouldn't constrain the GC from
>    doing whatever it wants to do with "dynamic" objects.)

I like this approach. Does it entail a ton of work?

Thanks again,

Cyrus