[Openmcl-devel] More Intel: Rosetta & Universal Binaries

[Gary Byers]

On Fri, 10 Jun 2005, David Steuber wrote:

> Hi Gary & Co,
>
> I just watched the 2005 Keynote on Apple's site.  I have a few OpenMCL 
> questions (I know you must be sick of this by now, but please bear with me) 
> that don't have to be answered straight away since there is a one to two year 
> transition time from the PPC switch to Intel plus additional time in which 
> legacy hardware needs to be supported.
>
> The first question is, will OpenMCL work under Rosetta?  From the talk on 
> Xcode 2.1, the "tweaks" needed for source code for Cocoa require are fairly 
> minor (presumably for Objective-C code) while the tweaks for Carbon are not 
> much worse.  OpenMCL has a fairly decent FFI, so I expect to possibly still 
> have access to the APIs even with PPC code generation if Rosetta doesn't 
> break the way Lisp works.

I can't tell anything about how Rosetta works from the information that's
available on Apple's web site, so it's very difficult to predict whether
OpenMCL will run under Rosetta (or what would prevent it from doing so
if it doesn't.)  I'll be out of town (at ILC and on vacation) the last
10 days or so of this month; I don't want to order a development system
until I get back, and Rosetta is one of the first things that I look at
when I have that system in hand.

>
> The second question is how complicated will Universal Binaries be?  The C 
> portion of OpenMCL has never given me any compile troubles.  So it seems fair 
> to speculate that with Xcode 2.1 or later, the kernel portion may build with 
> the flick of a switch, so to speak, so that part is a Universal Binary.  As 
> for the image file, could there perhaps be one image each for PPC and Intel? 
> Or would it be better if they were Universal as well?  And what about the 
> .dfsl files produced?

OpenMCL's kernel (the C part of it) mostly sits there and handles exceptions
that occur during the execution of compiled lisp code (including the
"exception" of not being able to allocate memory until the GC's had a
chance to run.)  It knows a lot about the state (register contents) of
running threads, and that code's currently very PPC-specific.

The idea behind Universal Binaries seems to be that a single executable
file contains X86 and PPC code and shared data; that's complicated by
the fact that the X86 is little-endian and the PPC is natively big-endian;
the saving grace is that (as a broad generalization) most C programs
contain relatively little static, initialized data, and it's practical
(if a bit Byzantine) to define "byte-swapping callbacks" as the UB
Porting Guide describes.  A typical lisp image contains about as
much data as it does code, and I have some difficulty convincing myself
that it'd be a good idea to page in a few megs of data a byte-swap it;
I get a bad headache when I start thinking about incrementally compiling
code in a Universal Binary version of a lisp development environment (do
we compile FACTORIAL for both targets ?), ad infinitum.

I wind up thinking that there are lots of other straightforward ways
to deliver bundled applications that'll run on both platforms, and
find myself strongly tempted (at this point) to dismiss Universal
Binary technology as being an irrelevant distraction.  (Unfortunately,
it's one of the few things for which any concrete information is
currently available.)

Exactly what an X86 kernel would look like depends a lot on what we
mean by "X86"; and X86-64 kernel would probably be significantly
different from an X86-32 kernel (and probably significantly better.)
The current development machines are apparently capable of running in
64-bit mode, but the OS is limited ("for now") to running 32-bit
(IA32) programs.  If we knew for sure that every interesting machine
that Apple sold to end-users over the next few years was
hardware/software capable of running X86-64 code, and X86-32 support
was only there as a transitional technology (largely to support
Universal Binaries), then I think that it'd make sense to concentrate
on an X86-64 port that supports native threads and precise GC and a
good FFI and - maybe - some smarter floating-point compilation, while
we're at it.


> I was surprised to hear in the keynote that the $999 development
> machines have to be given back.  Will that $999 be refunded?  I
> don't know how much of that is a simple security deposit and how
> much is a lease.

The day after Apple's announcement (Tuesday), I got a phone call from
Apple.  Unfortunately, it wasn't Steve Jobs asking me for my opinion
and wondering whether he'd made the right decision; it was someone
calling to remind me that the iMac G5 I'd bought a few months ago
was no longer under warranty.  Thinking quickly, I said "Huh.  You
mean I'm stuck with a piece of obsolete hardware ?", and felt a moment's
satisfaction as I slammed the phone down.

A moment later, I thought "Huh.  I'm stuck with a piece of obsolete
hardware.  Several pieces, in fact ..", and the sense of satisfaction
started to fade.  This transition won't be cheap for anyone, and it
seems to me that if Apple was concerned about not alienating
small/medium developers they wouldn't be squeezing the nickel quite so
hard on the development system issue.


> I've been thinking about this transition Apple is making.  I've had
> a lot of emotion about it.  I had been planning at some point to
> look into OpenMCL's LAP to take advantage of PPC special features
> like Altivec.  Now I don't see myself doing that.  I would rather
> just wait for Intel before doing LAP.  More of an immediate concern
> to me is the fact that OpenMCL, imho, is the best free Lisp going
> for OS X.  SBCL is nice and fast for floating point math, but
> OpenMCL seems to be faster with generic function calls.  Then of
> course there is the native threading and C callbacks that make using
> Carbon possible.  If I ever become a fan of Cocoa, there is that
> also.  In short, I would like to see OpenMCL continue on over the
> years.

So would I (and so would my colleagues at Clozure.)

>
> Gary, if there is a road plan for OpenMCL, will you post it to this list?

Yes, and though I may not always express this clearly, I hope that
other people will express their thoughts, needs, ideas, and concerns
here (and provide concrete information as it becomes available.)

If I'm stopping a little short of offering a road plan at the moment,
please forgive me and please be patient (at least a little patient.)
Based on the (incomplete and vague) information available at this
point, I'd be tempted to say that the right strategy is to concentrate
on a native X86-64 port and ignore both X86-32 and "Universal
Binaries" as being irrelevant (or at least of much lower priority.)
That's a reasonably significant effort, and it's likely that that
effort will have to be funded somehow.  Before making an investment in
time or asking anyone else to make an investment in time or money, I'd
want to be able to offer a plan that's based on more than me reading
between the lines and trying to make reasonable guesses about where
things are going.