[Openmcl-devel] Solaris x86-64?

[Gary Byers]

On Wed, 2 Jan 2008, Chris Curtis wrote:

>
> On Jan 2, 2008, at 11:39 AM, Gary Byers wrote:

>> The x86-64 port needs (or at least really, really wants)
>> some way to set the gs segment register so that it points
>> to per-thread lisp data.  I don't remember whether Solaris
>> offers a way to do this; if not, it might be hard to work
>> around that.

>
> That one I'd have to spend a little more time digging into... nothing off the 
> top of my head, since I've tried really hard to forget everything I ever knew 
> about segmented memory.

In ccl/lisp-kernel/thread-manager.c, there's a function called
'setup_tcr_extra_segment' that's called on thread startup on x86-64;
it tries to do something OS-dependent to make the %gs register
point at lisp per-thread data (the thread's "thread context record",
or TCR.)

Darwin ("the world's most advanced operating system!") doesn't offer
that, so we do something horrible and ugly: the pthreads library
uses %gs on Darwin (less advanced OSes that're compliant with
the amd64 ABI use %fs ...) and we switch %gs to point to either
pthreads data or lisp data every time we switch between running
lisp code and running foreign code; see code conditionalized on
DARWIN_GS_HACK.  If there isn't an advertised way to make %gs
available to user code on Solaris, there might be a way to do
SOLARIS_GS_HACK.

>
> So it at least looks like it's at least somewhat possible. I know it's 
> definitely not available for normal user consumption, but is there any guide 
> somewhere as to how I'd go about starting to build the codebase from scratch? 
Matt Emerson's been working on an IA-32 port; he has some notes about
that at <http://trac.clozure.com/openmcl/wiki/PortToIA-32>.  Cross-compiling
from Linux or Darwin x86-64 to Solaris x64 is probably a bit simpler;
the general idea is:

- You may find that this stuff works better if you're in the CCL
package and redefinition warnings are turned off.  One way to get
into such an environment is to do

(in-package "CCL")
(set-development-environment)

in your ~/ccl-init.lisp.

- the signal-handling stuff in the lisp kernel that's of immediate
concern involves recognizing the various flavors of traps and other
exceptions that occur during (more-or-less normal) lisp execution.
The lisp will use various kinds of UUOs (trap instructions) to
signal exceptional situations; most of these are of the form
"int $n" (possibly followed by a byte or two of data), and different
OSes map the exceptions that occur when such instructions are
executed to different synchronous signals.  It's fairly common
for "int $n" to generate a SIGSEGV; it may be necessary to look
at other information to distinguish between this and other causes
of SIGSEGV.

- there's probably other OS-specific stuff in the kernel that
may not be defined correctly for Solaris (e.g., how do we
determine the bounds of a thread's stack ?), depending on
how far I got a couple of years ago.

- look at ccl/compiler/X86/X8664/x8664-backend.lisp; it defines "backend" 
structures for the currently supported platforms.  Make one for
Solaris/x64, customizing at least the obvious stuff.  Push this
structure on the lists *known-backends* and *known-x8664-backends*,
and generally do the stuff that's done for the other platforms.

- look at ccl/library/*syscalls.lisp, which define system calls (the
trap numbers and argument/return-value types.)  Make a solaris-x64
version; system call numbers are often defined in unistd.h.
(For extra credit, figure out a way to generate these files
automatically; the manual definitions have been error-prone and
error-riddled.)

- look at the definition of SETUP-X8664-FTD in backend.lisp; this
is supposed to initialize a backend structure's "foreign type data"
(ftd). Add a CASE clause for the Solaris backend (for extra credit ...);
a lot of the structure options have to do with ABI details and should
be the same for Solaris as for other x86-64 platforms.  The value
of the :interface-db-directory is set differently according to
whether or not we're cross-compiling; this was intended to make
it easier to share an NFS-mounted "ccl" directory between the
host system and the target.  (The data in the interface .cdb
files is host-endian, and it's desirable to keep the host and
target versions of the interface files separate.)

Call SETUP-X8664-FTD on the Solaris backend.

- generate interface files for Solaris (at least the standard
C library).  You can possibly build FFIGEN on Solaris (if
gcc 4.0 works there) using <ftp://ftp.clozure.com/pub/ffigen-src.tar.gz>,
which may need some tweaks to its Makefile to recognize Solaris.
It should also work to copy the Solaris headers to a Linux or
Darwin or FreeBSD box and run the interface translator there
(with -isysroot set approprately).

- once you have a directory full of Solaris .ffi files on the
host system, setup the Solaris backend and do:

? (require "PARSE-FFI")

? (ccl::parse-standard-ffi-files :libc :solarisx64)

Run that twice to resolve forward-refrenced types and constants.

You should then have a bunch of shiny new .cdb files in
ccl/cross-solarisx64-headers/libc.

- look at the "xload backends" defined in ccl/xload/xx8664-fasload.lisp;
make one or Solaris.  You may want to change the image-base-address
field to match the address where Solaris will let you map a large
chunk of memory.

- load the xloader (the code that makes the initial bootstrapping
image out of the level-0 fasls).

? (ccl::require-update-modules* ccl::*x8664-xload-modules* t)

That'll recompile and load 3 fairly small files; if their fasls
are already up-to-date, you can use CCL::REQUIRE-MODULES instead
of CCL::REQUIRE-UPDATE-MODULES.

- try to build a bootstrapping image for Solaris:

? (cross-xload-level-0 :solaris864 :force)

should compile the level-0 sources and try to write a bootstrapping
image for Solaris (the bootstrapping image pathname is set in
the xload backend).

Some functions in level-0 do system calls; most of them aren't
too OS-dependent, so this could/should work fairly easily.

- create a file named "ccl/lib/ffi-solarisx64.lisp"; it needs
to define a few functions whose names are in the "interface"
package for Solarisx64 (set in the ftd for the backend); if
the x86-64 C calling conventions for Solaris are identical
to those used on other x86-64 platforms, the "Solaris-specific"
ffi hooks can just use things defined in the x8664 package
(which is what the linux/darwin/freebsd versions of that file
do.)  Add this file to ccl/lib/systems.lisp and ccl/lib/compile-ccl.lisp;
conditionalize compile-ccl for Solaris (some functions use the name
of the target to decide which architecture-specific files to include.)

- with updated versions of those files loaded, try to cross-compile
the rest of the lisp:

? (cross-compile-ccl :solarisx64 t)

Some files have a lot of OS-conditionalized code (level-1/linux-files.lisp,
level-1/l1-sockets.lisp in particular) and may need some work before
they'll compile cleanly.  level-1/x86-trap-support.lisp defines some
stuff which knows how to interpret the fields of a signal context;
if you compare the Solaris definitions of struct ucontext and
the mcontext structure that it contains/references with the Linux
version, that may help you to understand what the Linux code
in that file does and what the Solaris code needs to do.

Once you're as satisfied as you can be that things are compiling
cleanly, see what happens.  On the target (Solaris) system, ensure
that you have access to the (possibly NFS-shared) ccl directory,
and try:

./sx86cl64 sx86-boot64  # or whatever the bootstrapping image was called

If this doesn't crash spectacularly the first few times that you
try it, you are extremely lucky and/or good.

I actually think that if you can get to this point, you're probably
not too far from being able to save a native Solaris image and 
have it compile itself, and I'd guess that most of the problems
that you'd find are in the kernel's exception-handling code.  In
theory at least, non-OS-dependent Lisp code should behave exactly
the same way under Solaris as it does under Linux/Darwin/FreeBSD;
most of the OS-dependent code (filesystem, etc.) involves the
same sort of conditionalization that'd be involved in porting
a typical application between OSes, and it's the exception-handling
stuff that can be a little tricky and that absolutely, positively
has to work right.

(One thing that depends on a lot of things working right is the
GC, so it's sometimes helpful to turn off the EGC and defer GC
as long as possible, so that you can get things "working, modulo
GC issues.")

> It seems somehow a waste to run lx-branded zones on Solaris just to get CCL 
> running. (Plus, it'd be really nifty to add the dtrace hooks....)

How easy this is to do probably depends on how easy it is to understand
what the existing platform-specific code does and how easy it is to provide
Solaris-specific code; the code in question often deals with very
low-level things that aren't well documented (both in CCL and in the
OS.)

>
> --chris
>