[Openmcl-devel] A Debian packaging update and an outstanding issue

[Gary Byers]

On Mon, 13 Aug 2012, Pascal J. Bourguignon wrote:

> Faheem Mitha <faheem at faheem.info> writes:
>>>> The phrase "slightly older and compatible" is vague.
>>

Tough.

If you think that it's somehow possible to describe all of the dependencies
and issues concisely (and that whoever wrote that phrase was just too lazy
to do so),  you probably should think about that harder.

>>> Try to compile gcc 4.5 with gcc 1.0.
>>
>> I really don't see how this is relevant. I'm not suggesting that the
>> CCL developers support compilation of CCL by an ancient version of
>> CCL.
>
> There's also a vagueness in the path to upgrade gcc compilers.  You may
> compile version X of gcc with some version X-? of gcc, as long as ? is
> small enough.  It's the same with ccl.
>

If anyone was paying attention to this conversation, they'd probably find
it hard to reconcile this claim with the information in (e.g.)
<http://gcc.gnu.org/install/prerequisites.html>.

It'd indeed be hard to compile gcc 4.5 with gcc 1.0, but this is because
  - it's unlikely that anyone still has a copy of gcc 1.0
  - it's almost as unlikely that anyone still has a working machine
    that gcc 1.0 (which was released in 1987) would run on.

GCC (the C compiler proper) is written in fairly portable C and for much
of its lifetime it'd been critical to GCC's success that it be possible
to build GCC from sources using the vendor's C compiler.  ("The vendor"
might be a hardware vendor - Dec or Sun or ... - or the vendor of a
Unix distribution - AT&T, BSD, or some commercial entity.)

It used to be fairly easy to run into a case where an OS (Solaris
comes to mind) "unbundled" the vendor-provided C compiler (didn't
distribute it for "free"), and having GCC sources freely available was
a necessary but not sufficient step towards getting GCC running on
such a platform.  In those days (in which long walks to and from
school or work - uphill in both directions - were also common), it
was sometimes difficult to solve the chicken-and-egg bootstrapping
problem that can arise from not having a compiler for the language
that GCC is written in (even when that was K&R C for a fairly long
time and where GCC goes to great lengths to be portable and easily
bootstrapped.)  It's not that easy to run into such a scenario
anymore ("Can't you just do 'apt-get install gcc', or Google for
a site that contains binaries ?"), but I think that anyone who doesn't
have a feel for this issue would benefit from a little time travel
and a lot of walking uphill (which allegedly builds character, too.)


I don't know with absolute certainty that the oldest extant
running/runnable version of GCC will reliably build the current
version, but I suspect that that's much more nearly true than is
claimed above (and that if anyone was paying attention to this and
wanted to understand the issue that the claim above wouldn't help them
to do so.)

CCL's written in itself (some mixture of more-or-less portable ANSI
CL, implementation-dependent idioms and extensions, and bits and
pieces of assembly language/LAP code, all of which is to some degree
tied to a kernel/runtime that's written in a mixture of C and assembly
language.)  That implementation layer changes over time (and it's
generally a good thing that it does.)  The changes can be fairly near
the surface (a change to the implementation of one or more functions),
fairly deep (a change to runtime conventions), or anywhere in between.
Things that're near the surface can almost always be bootstrapped by
simply recompiling (via REBUILD-CCL); deeper changes are almost always
harder (sometimes much harder) to bootstrap.  (In other words, the
language that CCL's written in changes over time.)

In the trunk, we make "deep" (for some value of "deep") changes fairly
often, and such changes are usually accompanied by updated binaries.
(Those updated binaries are generally produced by adhoc means.  Sometimes
that's simply a matter of compiling and loading files in a slightly
different order than REBUILD-CCL does; sometimes it's more complicated
than that.)

In the current release (e.g., 1.8), we generally just try to avoid making
"deep" changes that would be hard to bootstrap via REBUILD-CCL. Sources
in the release tree are updated with bug fixes, but the general idea is
that it should always be possible to build an up-to-date copy of the current
release using the binaries distributed with the release and REBUILD-CCL.
(If fixing a bug in the release required a "deep" change, we've generally
refrained from making such a change.)

We nominally try to do two CCL releases each year.  (We're often late, so
the number of releases per year is probably a bit under 2.)  The code that
goes into a release is roughly "whatever's in the trunk shortly before the
release is built, with some speculative stuff removed", so the difference
between "the language that release N is written in" and "the language that
release N+1 is written in" is roughly equivalent to the differences in the
trunk over a 6+ month period.  If it worked to use the binaries from release
N to compile release N+1 it would do so by sheer luck and there'd be no
reason to believe that that would hold for later releases.

So: the language that CCL is written in changes over time, and it's necessary
to use a compiler that recognizes that language (a sufficiently recent heap
image) in order to compile its sources and produce an image that implements
the functionality described in those sources.  There are certainly other
bootstrapping strategies that don't depend on this chicken-and-egg cycle,
but it's not clear that those strategies would enable it to build itself
as quickly as it does, and that has real value.