[Openmcl-devel] 3 questions - TLS the default? global thread-shared vars? .CDB gen broken?
Gary Byers
gb at clozure.com
Sun Mar 20 16:49:17 PDT 2011
The bug caused an error to occur while the .cdb files were being rebuilt,
so you likely have a bunch of empty or partial .cdb files and things
like #$ can't find constant/type/function definitions in them.
There are two ways to recover working copies:
1) before trying to create new .cdb files, the interface parser made
backup copies (with extensions like .cdb-BAK or something like that;
I don't remember off the top of my head.) The backup copies are
in (e.g.) .../x86-headers64/libc, right next to the new/empty .cdb
files. It'd be nice if the interface parser did so for you if an
error occurs, but you can manually just recover the .cdb files from
the backup copies.
2) svn essentially maintains a backup copy of the entire CCL tree, so
you can do:
$ cd ccl/x86-headers64/libc
$ svn revert *.cdb
On Sun, 20 Mar 2011, Jason E. Aten wrote:
> Arg. I'm so close to getting this working! ? Here I did an svn update to
> r14684 / r14469, and get an error in lib/compile-ccl.lisp about a 'Error:
> Foreign variable "EX_OSERR" not found'. ?I'll attach a full log with svn
> update details and a backtrace in case that helps.
>
> ?- Jason
>
> p.s. a couple of small things came up while I was trying to figure this
> out...
>
> a. is there a way to show the source for a function already defined? ?In
> googling I found some notes about *recor\
> d-source-file*, and CCL:*SAVE-SOURCE-LOCATIONS*, but I haven't figured out
> how to display the source for a functi\
> on. Is that possible?
A lot of information about source locations of function definitions and other
things is maintained for the benefit of development environments (like SLIME
under Emacs and the Cocoa IDE on OSX and Windows.) In those environments,
the M-. ("meta-dot" or "meta-point") editor command can be used to find the
source to a definition, and other things ("go to source location where error
occurred") also use this information.
>
> b. is there an option to automatically, upon error condition, go to the
> toplevel as if typing ctrl-d or :q, instead o\
> f to the nested repl every time. I make alot of mistakes(!) and just want to
> try again immediately, without the e\
> xtra keystroke of having to get out of the condition restart nested repl.
C-d or :POP should be equivalent and exit the innnermost break loop; :Q
is supposed to exit all break loops and get back to the toplevel REPL.
>
> Thank you, thank you!
>
> c. summary of attached log of svn update attempt:
> ?
> jaten at afarm:~/pkg/ccl$ ccl --no-init
> trying: /home/jaten/pkg/ccl/lx86cl64
> Welcome to Clozure Common Lisp Version 1.6 ?(LinuxX8664)!
> ? (ccl:rebuild-ccl :full t)
> Rebuilding Clozure Common Lisp using Version 1.6 ?(LinuxX8664)
> ;Building lisp-kernel ...
> ;Kernel built successfully.
> ;Compiling "/home/jaten/pkg/ccl/lib/systems.lisp"...
> ;Loading #P"/home/jaten/pkg/ccl/bin/systems.lx64fsl"...
> ;Compiling "/home/jaten/pkg/ccl/lib/compile-ccl.lisp"...
> Read error between positions 17010 and 17742 in
> /home/jaten/pkg/ccl/lib/compile-ccl.lisp.
> > Error: Foreign variable "EX_OSERR" not found
> > While executing: %LOAD-VAR, in process listener(1).
> > Type :POP to abort, :R for a list of available restarts.
> > Type :? for other options.
> 1 >?
>
> #hmm... do I have the fix to lib/db-io.lisp ? ?It looks like it should be
> there from the svn log...
>
> jaten at afarm:~/pkg/ccl/lib$ svn log db-io.lisp
> ------------------------------------------------------------------------
> r14684 | gb | 2011-03-19 17:00:03 -0500 (Sat, 19 Mar 2011) | 2 lines
>
> Propagate r14525 to 1.6 branch.
>
> # umph... I don't know what do try next!?!
>
>
> On Sun, Mar 20, 2011 at 4:57 AM, Gary Byers <gb at clozure.com> wrote:
> That was fixed in the trunk a few months ago but the change
> hadn't
> been propagated to the 1.6 tree. ?It's there now.
>
>
> On Sat, 19 Mar 2011, Jason E. Aten wrote:
>
> Thank you for the fix on this, Gary.
>
> I tested it and it works. ?I can confirm that svn r72 of ffigen4
> now works
> for me on?ccl/x86-headers64/libc/C/populate.sh ?under Ubunutu
> 10.04 on
> x86_64.
>
> I am still getting some kind of issue with the final assembly of
> .cdb files
> (they are not produced) during ENCODE-FFI-TYPE. ?I try to run
> cc::parse-standard-ffi-files on the resulting :libc, and I get
> this error:
>
> ? (require "PARSE-FFI")
> "PARSE-FFI"
> ("PARSE-FFI")
> ? (ccl::parse-standard-ffi-files :libc)
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/_G_config.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/a\\.out.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/aio.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/aliases.ffi"
> ...
> ...
> ...(all the .ffi files are listed and seem to load okay, and
> then)....
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/wchar.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/wctype.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/wordexp.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/xlocale.ffi"
> ...
> #P"/home/jaten/pkg/ccl/x86-headers64/libc/C/usr/include/zlib.ffi"
> ...
> > Error: value #<FFI-TRANSPARENT-UNION 286_/usr/include/wait.h
> #x302002F0059D> is not of the expected type FFI-UNION.
> > While executing: ENCODE-FFI-TYPE, in process listener(1).
> > Type :POP to abort, :R for a list of available restarts.
> > Type :? for other options.
> 1 >?
>
> In case it helps diagnose this, I've attached copies of
> /usr/include/wait.h
> ?and the ?~/pkg/ccl/x86-headers64/libc/C/usr/include/wait.ffi
> ?file
> generated by h-to-ffi.sh run under populate.sh with CFLAGS="-m64
> -D_GNU_SOURCE";export CFLAGS.
>
> Let me know if I can provide anything more.
>
> ?- Jason
>
>
> On Sat, Mar 19, 2011 at 4:01 PM, Gary Byers <gb at clozure.com>
> wrote:
> ? ? ?gcc-4.0.0 is several years old, but we (usually) don't care
> ? ? ?about that;
> ? ? ?what generally matters is that the gcc version in question
> ? ? ?follows the
> ? ? ?platform's ABI's rules about how structures are laid out
> and how
> ? ? ?their
> ? ? ?fields are aligned. ?Older GCC versions are generally
> easier to
> ? ? ?build
> ? ? ?(have fewer dependencies) than newer versions are, and on
> many
> ? ? ?platforms
> ? ? ?we can patch 4.0.0 and get a working translator; that's
> what the
> ? ? ?code
> ? ? ?in the ffigen4 svn trunk tries to do.
>
> ? ? ?The construct from /usr/include/bits/link.h that it's
> choking on
> ? ? ?(and
> ? ? ?it chokes for me as well, though I didn't notice) is the
> ? ? ?description
> ? ? ?of the (relatively new) 256-bit "ymm" vector register type.
> ? ? ??ffigen
> ? ? ?doesn't believe that such a thing exists. ?It happens to be
> sort
> ? ? ?of
> ? ? ?right about that (it was supposed to have been introduced
> in
> ? ? ?Intel's
> ? ? ?"Sandy Bridge" chips, which got delayed/recalled and may or
> may
> ? ? ?not
> ? ? ?exist in shipping systems; I haven't followed it too
> closely)
> ? ? ?but the
> ? ? ?concept of 256-bit (and possibly larger) SIMD registers has
> been
> ? ? ?in
> ? ? ?some Linux headers (and kernels/libraries) for the last few
> ? ? ?years, and
> ? ? ?we need to support that.
>
> ? ? ?I checked in a change to ffigen that seems to fix this.
>
> ? ? ?On Sat, 19 Mar 2011, Jason E. Aten wrote:
>
> ? ? ?Hmm... that does indeed look like the wiki page I was
> ? ? ?referencing. ?Is
> ? ? ?gcc-4.0.0 not the current version?
> ? ? ?Here's the exact script (with sequence of commands) that I
> ? ? ?wrote to preserve
> ? ? ?exactly how I went about acquiring and installing ffigen
> ? ? ?and h-to-ffi.sh. If
> ? ? ?you have trouble reproducing it, let me know. It's just a
> ? ? ?stock Ubuntu 10.04
> ? ? ?on x86_64, but I can give you a login on a remote box if
> ? ? ?need be, or get you
> ? ? ?a virtualbox image.
> ? ? ?Thanks,
> ? ? ?Jason
>
> ? ? ?jaten at AtenMBP-2:~/research/djitia$ cat install-ffigen.sh?
> ? ? ?#!/bin/bash
> ? ? ?svn co
> ? ? ?http://svn.clozure.com/publicsvn/ffigen4/trunk/ffigen4
> ? ? ?cd ffigen4/
> ? ? ?wget
> ? ? ?http://ftp.gnu.org/gnu/gcc/gcc-4.0.0/gcc-core-4.0.0.tar.bz2
> ? ? ?wget
> ? ? ?http://ftp.gnu.org/gnu/gcc/gcc-4.0.0/gcc-objc-4.0.0.tar.bz2
> ? ? ?make
> ? ? ?# example sudotarxfz/home/jaten/pkg/llvm28/llvm-2.8/tools/ldc2/ffigen/ffigen4/ffigen-bin-
> lin
>
> ? ? ?uxx
> ? ? ?8664-gcc-4.0.0-2011-03-18-23-40-54.tar.gz -C /usr/local
> ? ? ?export TARBALL=`ls -1 | grep ffigen-bin`
> ? ? ?export HERE=`pwd`
>
> ? ? ?echo "installing now to: ? ?/usr/local/ffigen ? and ?
> ? ? ?/usr/local/bin/h-to-ffi.sh ? ...with sudo"
> ? ? ?echo "sudo tar xfz `pwd`/`ls -1 | grep ffigen-bin` -C
> ? ? ?/usr/local"
> ? ? ?sudo tar xfz `pwd`/`ls -1 | grep ffigen-bin` -C /usr/local
>
>
> ? ? ?On Sat, Mar 19, 2011 at 11:37 AM, Gary Byers
> ? ? ?<gb at clozure.com> wrote:
> ? ? ?? ? ?Hmm. ?The Wiki page at
> ? ? ?? ? ?<http://trac.clozure.com/ccl/wiki/BuildFFIGEN> is
> ? ? ?? ? ?supposed to be the canonical reference that describes
> ? ? ?how to
> ? ? ?? ? ?obtain/build/install
> ? ? ?? ? ?the FFI translator. ?In the past, we've distributed
> ? ? ?sources and
> ? ? ?? ? ?binaries via
> ? ? ?? ? ?other means, but there aren't supposed to be any
> ? ? ?links to those
> ? ? ?? ? ?older versions
> ? ? ?? ? ?and I can't find any links in the documentation.
>
> ? ? ?? ? ?The current version doesn't have any problem
> ? ? ?translating
> ? ? ?? ? ?/usr/include/link.h
> ? ? ?? ? ?(or with the SIMD vector type referenced in the
> ? ? ?included file),
> ? ? ?? ? ?but I'd certainly
> ? ? ?? ? ?believe that older versions did. ?If you got an older
> ? ? ?version,
> ? ? ?? ? ?that'd explain
> ? ? ?? ? ?it; if you remember how you got that version, we
> ? ? ?should fix any
> ? ? ?? ? ?links to it
> ? ? ?? ? ?so that they reference the Wiki page instead.
>
> ? ? ?? ? ?On Sat, 19 Mar 2011, Jason E. Aten wrote:
>
> ? ? ?? ? ?Thank you Raffael. ?I appreciate the pointer. I guess
> ? ? ?that
> ? ? ?? ? ?confirms #1, and
> ? ? ?? ? ?answers #2. :-)
>
> ? ? ?? ? ?For #3, I've managed to locate the source code for
> ? ? ?the
> ? ? ?? ? ?ffigen loader, and
> ? ? ?? ? ?figured out that :SUBDIR is supposed to be :libc or
> ? ? ?:elf
> ? ? ?? ? ??or similar.
>
> ? ? ?? ? ?However I do manage to have populate.sh run into an
> ? ? ?issue:
>
> ? ? ?? ? ?+++ /usr/include/link.h
> ? ? ?? ? ?Need to create info for type:
> ? ? ?? ? ??<vector_type 0x7fab52eacb60
> ? ? ?? ? ??? ?type <real_type 0x7fab53619000 float SF
> ? ? ?? ? ??? ? ? ?size <integer_cst 0x7fab535fba80 constant
> ? ? ?? ? ?invariant 32>
> ? ? ?? ? ??? ? ? ?unit size <integer_cst 0x7fab535fb5a0
> ? ? ?constant
> ? ? ?? ? ?invariant 4>
> ? ? ?? ? ??? ? ? ?align 32 symtab 18 alias set -1 precision 32
> ? ? ?? ? ??? ? ? ?pointer_to_this <pointer_type
> ? ? ?0x7fab53619270>>
> ? ? ?? ? ??? ?BLK
> ? ? ?? ? ??? ?size <integer_cst 0x7fab536146f0 type
> ? ? ?<integer_type
> ? ? ?? ? ?0x7fab536015b0
> ? ? ?? ? ?bit_size_type> constant invariant 256>
> ? ? ?? ? ??? ?unit size <integer_cst 0x7fab535fb570 type
> ? ? ?? ? ?<integer_type 0x7fab536014e0
> ? ? ?? ? ?long unsigned int> constant invariant 32>
> ? ? ?? ? ??? ?align 256 symtab 341 alias set -1 nunits 8>
> ? ? ?? ? ?In file included from /usr/include/link.h:36:
> ? ? ?? ? ?/usr/include/bits/link.h:68: internal compiler error:
> ? ? ?in
> ? ? ?? ? ?ffi_create_type_info, at ffi.c:428
> ? ? ?? ? ?Please submit a full bug report,
> ? ? ?? ? ?with preprocessed source if appropriate.
> ? ? ?? ? ?See <URL:http://gcc.gnu.org/bugs.html> for
> ? ? ?instructions.
>
> ? ? ?? ? ?where the relevant section from
> ? ? ?/usr/include/bits/link.h,
> ? ? ?? ? ?including line 68
> ? ? ?? ? ?is this:
>
> ? ? ?? ? ?/* Registers for entry into PLT on x86-64. ?*/
> ? ? ?? ? ?# if __GNUC_PREREQ (4,0)
> ? ? ?? ? ?typedef float La_x86_64_xmm __attribute__
> ? ? ?? ? ?((__vector_size__ (16)));
> ? ? ?? ? ?typedef float La_x86_64_ymm __attribute__
> ? ? ?? ? ?((__vector_size__ (32))); ?/* line
> ? ? ?? ? ?68 */
> ? ? ?? ? ?# else
> ? ? ?? ? ?typedef float La_x86_64_xmm __attribute__ ((__mode__
> ? ? ?? ? ?(__V4SF__)));
> ? ? ?? ? ?# endif
>
> ? ? ?? ? ?Arg. What should I try? ?Is this the right place to
> ? ? ?report
> ? ? ?? ? ?this?
>
> ? ? ?? ? ?Thank you.
>
> ? ? ?? ? ?Jason
>
>
> ? ? ?? ? ?On Sat, Mar 19, 2011 at 12:21 AM, Raffael Cavallaro
> ? ? ?? ? ?<raffaelcavallaro at mac.com> wrote:
>
> ? ? ?? ? ?? ? ?On Mar 19, 2011, at 1:09 AM, Jason E. Aten
> ? ? ?wrote:
>
> ? ? ?? ? ?? ? ?> In order words, when two different threads
> ? ? ?need to
> ? ? ?? ? ?be able to
> ? ? ?? ? ?? ? ?name the same variable (after locking the thread
> ? ? ?? ? ?inspects the
> ? ? ?? ? ?? ? ?shared variable, possibly updates it, then
> ? ? ?releases
> ? ? ?? ? ?the lock),
> ? ? ?? ? ?? ? ?how do I tell lisp this is what I'd like?
>
> ? ? ?? ? ?See section 4.8 of the documentation "Static
> ? ? ?Variables"
>
> ? ? ?? ? ?e.g., (defstatic *foo* 3)
>
> ? ? ?? ? ?It's an error to dynamically rebind a static
> ? ? ?variable; its
> ? ? ?? ? ?value is
> ? ? ?? ? ?always the same across all threads. Naturally, as you
> ? ? ?? ? ?suggest, you'll
> ? ? ?? ? ?want to avoid race conditions by using a lock or
> ? ? ?? ? ?semaphore.
>
> ? ? ?? ? ?warmest regards,
>
> ? ? ?? ? ?Ralph
>
>
> ? ? ?? ? ?Raffael Cavallaro
> ? ? ?? ? ?raffaelcavallaro at me.com
>
>
>
>
>
>
>
> ? ? ?--
> ? ? ?Jason E. Aten, Ph.D.
>
>
>
>
>
>
>
> --
> Jason E. Aten, Ph.D.
>
>
>
>
>
>
>
> --
> Jason E. Aten, Ph.D.
>
>
>
>
More information about the Openmcl-devel
mailing list