[Openmcl-devel] Release co, update, build - again...
Gary Byers
gb at clozure.com
Sat Oct 24 13:50:08 PDT 2009
The general release scenario is something like:
- we decide that the state of things in svn at some point in time
is a "release", create a "release" branch in svn, and create
.tar/.zip/.dmg files that represent a snapshot of what's in svn at
that point in time. That snapshot includes sources and "binaries"
(generally platform-specific kernel and heap image files.)
- over the lifetime of a "release", bugs are found and fixed. We
usually fix them in the release tree by just checking in sources
and expect people to use "svn update"/REBUILD-CCL to update their
binaries (kernel/heap image.) One exception to this policy involves
lisp kernel changes on Windows: it's somewhat awkward to install
a C toolchain on Windows, and we've tried to remember that and
check in the .exe file when its sources change.
- the changes that we make to a release are intentionally conservative:
a slightly older set of binaries from the same release should be
able to bootstrap an up-to-date set of binaries from updated sources.
(In the trunk - where day-to-day changes are made - some changes
are deeper and harder to bootstrap, and are usually accompanied by
updated binaries.)
So: if you do an "svn co", the sources that you get will be up-to-date;
the kernel and image should be able to compile those sources but
generally may missing some bug fixes, and (REBUILD-CCL :FULL T) should
uneventfully (in a release branch) or "with possible warnings and CERRORs"
(in the case of the trunk) make your local binaries match the sources.
The svn revision number that shows up in (LISP-IMPLEMENTATION-VERSION)
and in the "Welcome" banner indicates the svn revision as of the time
that the image was built.
In general, what you were trying to do here (REBUILD-CCL :FULL T) is
correct. The error(s) have to do with the fact that header files on
your system describe a low-level data structure as having different
fields than we expect them to. (I believe that the file in question
is /usr/include/asm/ldt.h and structure in question is called 'user_desc';
we expect it to look like:
struct user_desc {
unsigned int entry_number;
unsigned int base_addr;
unsigned int limit;
unsigned int seg_32bit:1;
unsigned int contents:2;
unsigned int read_exec_only:1;
unsigned int limit_in_pages:1;
unsigned int seg_not_present:1;
unsigned int useable:1;
#ifdef __x86_64__
unsigned int lm:1;
#endif
};
and I'd guess that your version of that file describes the structure
as having similar fields (IIRC, the layout of the structure is largely
or entirely dictated by the x86 hardware) but refers to those fields
with different names.
This kind of gratuitous renaming between OS releases happens from time
to time (on OSX, a lot of it happened between 10.4 and 10.5 and we jump
through a lot of hoops to work around it.)
On Linux, the situation's a bit different: this sort of "let's rename
everything since we haven't done so in a few years" change probably
happens less frequently, but the fact that there are dozens of Linux
distros in widespread use makes tracking this sort of thing difficult.
(This particular change likely happened in version X of Fedora and version
Y of Ubuntu and version Z of Debian, but I don't know what the values of
X, Y, and Z are. On the SystemRequirements page of the Wiki, we just
say something vague like "more recent Linux releases are more likely
to work than older ones".
I'd guess that if you do:
$ cat /etc/issue
on your Fedora system, you'll find that you have a fairly old release
(where "old" might mean "a few years old".) The current Fedora release
is 11, Fedora 12 is supposed to be out in a few weeks, and they generally
do ~2 releases each year.
If you don't want to do a full OS update (for such a silly reason), you
might find that updating a "kernel headers" package (by some name) gives
you a more up-to-date version of the 'ldt.h' header file.
It might be possible for us to avoid this issue by defining the structure
ourselves and not depending on the header file. We do that in a few cases,
but that kind of defeats the purpose of using header files ...
On Sat, 24 Oct 2009, Jon S. Anthony wrote:
> Hi,
>
> I'm trying to make sure I'm squared up on getting the most recent
> release for a given version and having it actually running.
>
> I'm fine with just checking out the latest (not need for update), for
> example:
>
> svn co http://svn.clozure.com/publicsvn/openmcl/release/1.3/linuxx86/ccl
>> svn.out
>
> My understanding is that this then has the latest kernel binaries and
> the latest source for the release. But to get the latest version
> actually running, you need to rebuild ccl via (rebuild-ccl :full t).
>
> This seems to be confirmed by the output in svn.out (from the co
> command), which states that everything came over fine (all files notated
> with A or U) and is at r13098 as evidenced by these statements:
>
> "Checked out external at revision 13098."
>
> And when ccl is run, it still says it is 1.3-r11936:
>
> $ ./lx86cl
> Welcome to Clozure Common Lisp Version 1.3-r11936 (LinuxX8632)!
> ? (quit)
>
> or
>
> $ ccl
> Welcome to Clozure Common Lisp Version 1.3-r11936 (LinuxX8632)!
> ?
>
> OK, so this is where/when you are supposed to evaluate
> (rebuild-ccl :full t) to ensure the binaries (fasls) reflect the latest
> source that came over in the CO. Also, since this is a fresh complete
> co, the latest kernel binaries and image to be able to do this are
> supposed to be there, right? Or is this the goof up - the latest
> required binaries aren't really here??
>
> OTOH, I can't think of a reason why I actually need to rebuild the
> kernel, so maybe I don't need the :full t?
>
> This is on Fedora lx8632 box. It has (I believe) the full gcc
> functionality available, including gcc 4.1, m4 and as (certainly, I've
> done all sorts of other C compilation and builds for all sorts of other
> things with no problems).
>
> I know you've been waiting for the but, and here it is:
>
> $ ./lx86cl
> Welcome to Clozure Common Lisp Version 1.3-r11936 (LinuxX8632)!
> ? (rebuild-ccl :full t)
> Rebuilding Clozure Common Lisp using Version 1.3-r11936 (LinuxX8632)
> ;Building lisp-kernel ...
>> Error: Error(s) during kernel compilation.
>> make: Entering directory
> `/opt/CCL/ccl-1.3-r13098/ccl/lisp-kernel/linuxx8632'
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../pad.s | as --32
> -o pad.o
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../x86-spjump32.s |
> as --32 -o x86-spjump32.o
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../x86-spentry32.s |
> as --32 -o x86-spentry32.o
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../x86-subprims32.s |
> as --32 -o x86-subprims32.o
>> cc -c ../pmcl-kernel.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o pmcl-kernel.o
>> cc -c ../gc-common.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o gc-common.o
>> cc -c ../x86-gc.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o x86-gc.o
>> cc -c ../bits.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o bits.o
>> cc -c ../x86-exceptions.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o x86-exceptions.o
>> cc -c ../image.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o image.o
>> cc -c ../thread_manager.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o thread_manager.o
>> cc -c ../lisp-debug.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o lisp-debug.o
>> ../thread_manager.c: In function ‘setup_tcr_extra_segment’:
>> ../thread_manager.c:966: error: variable ‘u’ has initializer
> but incomplete type
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: warning: excess elements in struct
> initializer
>> ../thread_manager.c:966: warning: (near initialization for ‘u’)
>> ../thread_manager.c:966: error: storage size of ‘u’ isn’t known
>> ../thread_manager.c:985: error: invalid application of ‘sizeof’
> to incomplete type ‘struct user_desc’
>> ../thread_manager.c: In function ‘free_tcr_extra_segment’:
>> ../thread_manager.c:998: error: variable ‘u’ has initializer
> but incomplete type
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: warning: excess elements in struct
> initializer
>> ../thread_manager.c:998: warning: (near initialization for ‘u’)
>> ../thread_manager.c:998: error: storage size of ‘u’ isn’t known
>> ../thread_manager.c:1006: error: invalid application of
> ‘sizeof’ to incomplete type ‘struct user_desc’
>> make: *** [thread_manager.o] Error 1
>> cc -c ../memory.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o memory.o
>> cc -c ../unix-calls.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o unix-calls.o
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../x86-asmutils32.s |
> as --32 -o x86-asmutils32.o
>> m4 -DLINUX -DX86 -DX8632 -DHAVE_TLS -I../ ../imports.s | as
> --32 -o imports.o
>> cc -c ../lispdcmd.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o lispdcmd.o
>> cc -c ../plprint.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o plprint.o
>> cc -c ../plsym.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o plsym.o
>> cc -c ../xlbt.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o xlbt.o
>> cc -c ../x86_print.c -DLINUX -D_REENTRANT -DX86 -DX8632
> -D_GNU_SOURCE -DHAVE_TLS -DUSE_FUTEX -g -m32 -o x86_print.o
>> make: Target `all' not remade because of errors.
>> make: Leaving directory
> `/opt/CCL/ccl-1.3-r13098/ccl/lisp-kernel/linuxx8632'
>>
>> While executing: REBUILD-CCL, in process listener(1).
>> Type :POP to abort, :R for a list of available restarts.
>> Type :? for other options.
> 1 >
>
> It seems that somewhere along the way, I'm missing the full definition
> for user_desc and possibly some other stuff.
>
> I also tried the "manual" approach:
>
> shell> cd ccl/lisp-kernel/PLATFORM
> shell> make
>
> Exact same results (not surprising...).
>
> I've gone back over
>
> <http://clozure.com/pipermail/openmcl-devel/2009-September/010361.html>
>
> and went over
>
> <http://ccl.clozure.com/manual/chapter2.2.html#obtaining-via-svn>
>
> and
>
> <http://trac.clozure.com/openmcl/wiki/UpdatingFromSource>
>
> I don't think the updating one actually applies here as I am going
> against a completely fresh latest release check out. But I've been
> wrong before...
>
> Clearly I'm missing something, but I haven't noticed what it is. But,
> as I mentioned before, maybe there is no need to rebuild the kernel in
> this case anyway. If not, what keys should I give to REBUILD-CCL? Just
> take the defaults: (rebuild-ccl)? If I do that, it does complete and
> then says it is indeed r13098, so maybe that's just fine?
>
> Any help (including, or even especially, "you missed this obvious bit:
> '.....'") appreciated.
>
>
> /Jon
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
More information about the Openmcl-devel
mailing list