[Openmcl-devel] (how) is ffigen to be made to work (?)

Tue Feb 24 19:03:00 PST 2009

This is a mess; there should be a wiki page explaining this.

The easiest way to obtain a version of ffigen that works (on most platforms;
see below)  is to build it yourself from svn.

For most platforms, you can do:

shell> svn co http://svn.clozure.com/publicsvn/ffigen4/trunk/ffigen4

You'll also need to obtain the GCC 4.0 "core" sources and the GCC 4.0
ObjC frontend from gcc.gnu.org or a mirror site; the files that you
need are "gcc-core-4.0.0.tar.bz2" and "gcc-objc-4.0.0.tar.bz2"; be
sure to get the .bz2 versions.  Copy these archives to the ffigen4
directory created by the svn checkout above; don't uncompress/untar
them.  Then:

shell> cd ffigen4
shell> make

That should:

- un-tar/uncompress the archives
- apply some patches to the GCC sources
- run configure/make to produce a modified version of the ObjC frontend.
   (ObjC is s superset of C; there's no real harm in having the frontend
   understand ObjC extensions even on platfroms where they're rarely used.
- create a compressed tar file that contains that binary, an "h-to-ffi.sh"
   shell script, and (in some cases) some compiler-specific .h files.  That
   archive is functionally equivalen to the ffigen archives that have been
   distributed in the past.

That should "work" for Linux, FreeBSD, Solaris, Win32, and older/32-bit
Darwin code that doesn't use ObjC 2.0 extensions.  For more modern 
Darwin versions (where the .h files may contain those ObjC extensions),
there's a separate branch that patches Apple's "gcc-5465":

shell> svn co http://svn.clozure.com/publicsvn/ffigen4/branches/ffigen-apple-gcc-5465/ffigen4

That svn tree contains Apple's 'gcc-5465.tar.gz', so the "svn co" will pause
for however long it takes to download that.

Then:

shell> cd ffigen4
shell> make

and things should work as they do in the trunk.

For Win64 ... (sound of crickets chirping).  Win64 support is being added
on the gcc-4.4 branch; when we first started working on the Win64 port,
the Win64 GCC couldn't reliably compile itself natively.  (I may be
misremembering some of that; gcc-4.4 depends on several additional
libraries -gmp, libdecnumber, ...- beyond what 4.0 did, and the problem
may have had to do with building those libraries natively on Win64.)
I wound up building a cross-translator on Linux, but (since that's
always involved copying lots of .h and .ffi files back and forth between
Linux and Win64) it's suboptimal.  It's likely that whatever problems
prevented doing a native Win64 build a year ago are long gone and it's
worth trying to do a native Win64 build again; I think that there are
also some issues having to do with the fact that gcc 4.4 recognizes
some things ("decimal numbers") as primitive types and we need to
sort of recognize them even though there's no obvious way to map them
to a primitive Lisp foreign type.

I think that Apple has contributed their ObjC 2.0 extensions back into
the GCC sources, so at some point it should be possible to build an
ffigen from the same set of gcc sources and expect that to work on
all platfroms that the lisp runs on.

I don't think that it's practical for us to distribute ffigen binaries
as we have in the past: there are too many variables (word size, ABI
and library version differences.)  It's hard to do this just for the
CCL lisp kernel, which (compared to gcc) is a relatively tiny/simple C
program with relatively few and relatively simple external
dependencies. It's hard to build C programs on Leopard that'll run on
Tiger (or at the very least it's hard to modify GCC's autobuild setup
to ensure that the right compiler options are passed around, and the
same sorts of concerns apply to varying degrees on other platforms as
well.

Building ffigen from source takes a few (for some value of "a few")
minutes, as does doing the svn checkout and the gcc download. An
ffigen binary that you build yourself on a particular system is much
more likely to work on that system than one that I build on a
generally similar system, so this seems like a reasonable tradeoff.

On Tue, 24 Feb 2009, james anderson wrote:

> hello;
>
> i would like to generate interface databases for fftw and open-mpi.
> i have tried to follow the several instructions
>
>  - the trac wiki [ http://trac.clozure.com/openmcl/wiki/OpenMclFfi ]
>  - the ccl-documentation page [ http://ccl.clozure.com/ccl-
> documentation.html#The-Foreign-Function-Interface ]
>  - the doc wiki [ http://doc.clozure.com/doku.php/doc/openmcl/
> ffi#creating_new_interface_directories ]
> (whereby the links in the latter appear most reliable)
>
> but have yet to succeed. it may just be, i've not yet found the right
> combination, but i also wonder whether there is a working combination
> for my particular environment.
>
> i observe that,
>
> a, there are several darwin ppc ffigen versions:
> two distinct ones in pub
> -rw-r--r--    1 501      0         2224311 May 17  2005 ffigen-bin-
> darwinppc-gcc-4.0.0-2005-05-17-11
> -rw-r--r--    1 501      0         2241140 Oct 24  2003 ffigen-
> darwin-0.1.tar.gz
> a very similar one in pub/FFIGEN
> -rw-r--r--    1 0        0         2224311 Apr 21  2008 ffigen-bin-
> darwinppc-gcc-4.0.0-2005-05-17-11
> as well as a dozen others which appear contraindicated by name.
>
> b, there are (at least) two active ccl versions
> - i had been using 1.2
> - i've now tried also 1.3
>
> c, there are (more than) two active os x versions
> - i use 10.4.11
> - it would be inconvenient to switch to 10.5 - which several
> populate.sh scripts stipulate
>
> d, there are two interface versions
> - 32 bit
> - 64 bit
>
>
> while i've yet to exhaust the variations at my disposal, i'm also yet
> to observe any populate to run without issuing some error message.
> for my new cases (open-mpi and fftw), this may well involve incorrect
> c-flags, but i've tried also to use the newer ffigen with various of
> the interfaces included in the two ccl releases, and see error
> messages then as well. true, for the 32-bit headers, this is limited
> to missing .h files.
>
> could someone who has ffigen experience, please advise me
>
> ? which combinations are known to work?
>
> ? whether i should expect the 17.may ffigen, and os x 10.4.11, and
> either ccl 1.2, or 1.3 to work? for the 64-bit interfaces, or just
> the 32-bit.
>
> ? if i have built and installed the two new libraries (fftw and open-
> mpi) which of the c-flags from the build transcripts will be material
> to ffigen?
>   if there are no general principles, and one needs the command
> lines, i can send them along.
>
> thanks,
>
>
>
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>