[Openmcl-devel] New snapshots available

Gary Byers gb at clozure.com
Wed Feb 14 23:31:00 UTC 2007


That looks like an old (061231) lisp image trying to compile the
newer (071214) sources.  I'm 99.9% that the archives I uploaded
last night have the new images/kernels in them.

I just downloaded the archive and checked; the
"openmcl-darwinx8664-snapshot-070214.tar.gz" tarball does indeed
contain the 070214 image, and it rebuilt itself without problems.

On Wed, 14 Feb 2007, Sunil Mishra wrote:

> Hi Gary,
>
> Thanks for the new snapshot. The latest CVS isn't compiling for me. I
> started up openmcl and ran (ccl:rebuild-ccl :full t). I am running Mac
> OS X 10.4.8 on an intel macbook pro.
>
> Here's a snippet of the trace:
>
> ;Compiling "/usr/local/src/ccl/compiler/X86/x86-backend.lisp"...
> ;Loading #P"/usr/local/src/ccl/bin/x86-backend.dx64fsl"...
> ;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp"...
> ;Compiler warnings for
> "/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
> ;   Function call arguments don't match current definition of MAKE-FTD
> (3 references), in SETUP-X8664-FTD.
> ;Compiler warnings for
> "/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
> ;   Undefined function REQUIRE-FOREIGN-TYPE-BITS, in
> X8664::GENERATE-CALLBACK-BINDINGS.
> ;   Undefined function NULL-COERCE-FOREIGN-RESULT, in X8664::EXPAND-FF-CALL.
> ;   Undefined function NULL-COERCE-FOREIGN-ARG, in X8664::EXPAND-FF-CALL.
> ;Loading #P"/usr/local/src/ccl/bin/x8664-backend.dx64fsl"...
> ;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"...
>> Error: subprim named .SPFFCALL-RETURN-REGISTERS not found.
>> While executing: %SUBPRIM-NAME->OFFSET, in process listener(1).
>> Type :POP to abort, :R for a list of available restarts.
>> Type :? for other options.
> 1 > :b
> (C26E28) : 0 (%SUBPRIM-NAME->OFFSET '.SPFFCALL-RETURN-REGISTERS
> #(#<SUBPRIMITIVE-INFO .SPJMPSYM @ #x5000> #<SUBPRIMITIVE-INFO .SPJMPNFN
> @ #x5008> #<SUBPRIMITIVE-INFO .SPFUNCALL @ #x5010> #<SUBPRIMITIVE-INFO
> .SPMKCATCH1V @ #x5018> #<SUBPRIMITIVE-INFO .SPMKUNWIND @ #x5020> ...)) 261
> (C26E50) : 1 (FUNCALL #'#<FIND-NAME> '.SPFFCALL-RETURN-REGISTERS) 125
> (C26E80) : 2 (FUNCALL #'#<SIMPLIFY-MEMORY-OPERAND>
> '(.SPFFCALL-RETURN-REGISTERS)) 1061
> (C26EF0) : 3 (FUNCALL #'#<SIMPLIFY-OPERAND> '(:@
> .SPFFCALL-RETURN-REGISTERS)) 197
> (C26F28) : 4 (FUNCALL #'#<SIMPLIFY-FORM> '(JMP #)) 1461
> (C26FE8) : 5 (DEFINE-X86-VINSN #<BACKEND DARWINX8664 #x30004076ECCD>
> 'FF-CALL-RETURN-REGISTERS NIL NIL NIL '(# # # # :BACK)) 4037
> (C27160) : 6 (FUNCALL #'#<DEFINE-X8664-VINSN> ???) 509
> (C271D0) : 7 (FUNCALL #<Compiled-function DEFINE-X8664-VINSN
> (Non-Global)  #x300041B1B54F> [...]) 157
> (C27200) : 8 (MACROEXPAND-1 '(DEFINE-X8664-VINSN # # # # ...) [...]) 301
> (C27230) : 9 (FCOMP-FORM-1 '(DEFINE-X8664-VINSN # # # # ...)
> #<LEXICAL-ENVIRONMENT  #x300041B0F1BD> :NOT-COMPILE-TIME) 661
> (C27270) : 10 (FCOMP-FORM '(DEFINE-X8664-VINSN # # # # ...)
> #<LEXICAL-ENVIRONMENT  #x300041B0F1BD> :NOT-COMPILE-TIME) 2117
> (C272F8) : 11 (FCOMP-FORM '(DEFINE-X8664-SUBPRIM-CALL-VINSN #
> .SPFFCALL-RETURN-REGISTERS) #<LEXICAL-ENVIRONMENT  #x300041B0F1BD>
> :NOT-COMPILE-TIME) 2117
> (C27380) : 12 (FCOMP-READ-LOOP
> "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
> #P"ccl:compiler;X86;X8664;x8664-vinsns.lisp.newest"
> #<LEXICAL-ENVIRONMENT  #x300041B0F1BD> :NOT-COMPILE-TIME) 1821
> (C27498) : 13 (FCOMP-FILE
> "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
> #P"ccl:compiler;X86;X8664;x8664-vinsns.lisp.newest"
> #<LEXICAL-ENVIRONMENT  #x300041B0F1BD>) 821
> (C27630) : 14 (%COMPILE-FILE
> "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
> "/usr/local/src/ccl/bin/x8664-vinsns.dx64fsl" T NIL T NIL T T NIL NIL
> #<BACKEND DARWINX8664 #x30004076ECCD> :DEFAULT) 2525
> (C278B8) : 15 (COMPILE-FILE "ccl:compiler;X86;X8664;x8664-vinsns.lisp"
> [...]) 1237
> (C279C8) : 16 (UPDATE-MODULES '(X86-ARCH X86-ASM X86-LAP X8664-ARCH
> X8664ENV ...) [...]) 597
> (C27A60) : 17 (COMPILE-CCL [...]) 253
> (C27A80) : 18 (REBUILD-CCL [...]) 1941
> (C27B68) : 19 (CALL-CHECK-REGS 'REBUILD-CCL [...]) 229
> (C27BA0) : 20 (TOPLEVEL-EVAL '(REBUILD-CCL :FULL T) [...]) 429
> (C27BF0) : 21 (READ-LOOP [...]) 1845
> (C27DF8) : 22 (TOPLEVEL-LOOP) 197
> (C27E28) : 23 (FUNCALL #'#<Anonymous Function #x30004042932F>) 133
> (C27E40) : 24 (FUNCALL #'#<Anonymous Function #x300040575AEF>) 773
> (C27EC8) : 25 (RUN-PROCESS-INITIAL-FORM #<PROCESS listener(1) [Active]
> #x300040D43B7D> '(#)) 701
> (C27F48) : 26 (FUNCALL #'#<Anonymous Function #x30004044088F> #<PROCESS
> listener(1) [Active] #x300040D43B7D> '(#)) 397
> (C27F98) : 27 (FUNCALL #'#<Anonymous Function #x3000403F56CF>) 365
>
>
> Gary Byers wrote:
>> There are now new (070214) self-contained archives containing OpenMCL
>> source, binaries, and interfaces for DarwinPPC32/64, LinuxPPC32/64,
>> DarwinX8664, LinuxX8664, and FreeBSDX8664 available in
>> <ftp://clozure.com/pub/testing>.  The release notes entry says:
>>
>> OpenMCL 1.1-pre-070214
>> - The FASL version changed (old FASL files won't work with this
>>    lisp version), as did the version information which tries to
>>    keep the kernel in sync with heap images.
>> - There are new interface files for all platforms.  These files
>>    encode some foreign type information a little differently
>>    than older ones did (notably information about foreign functions
>>    that return structures or accept structure args by value.)  The
>>    new .cdb files can't be used by older versions of OpenMCL; using
>>    older .cdb files with this version is "allowed, but not supported
>>    or recommended."
>> - Almost all of the changes in functionality since the last (061231)
>>    snapshots and since the CVS freeze on 070117 have to do with
>>    relatively obscure issues having to do with passing structures
>>    to foreign functions by value and/or returning structures from foreign
>>    function calls.
>>
>>    These idioms are fairly rare in traditional C code (though it's
>>    fairly common to pass -pointers- to structures by reference
>>    and sometimes to return pointers to structures.  There are
>>    a few C compiler runtime routines that perform some flavor
>>    of integer division and return a two-element structure that
>>    contains "quotient" and "remainder" fields, but that's typically
>>    about the extent of the use of this idiom.)  The idioms are used
>>    much more often in Apple's Carbon and Cooca libraries and in
>>    some of the frameworks (CoreGraphics, CoreFoundation) that those
>>    libraries are based on.
>>
>>    OpenMCL's FFI has provided some support for this in the past;
>>    notably, it's provided support for (most of the) structure-returning
>>    and struct-by-value conventions used on 32-bit PPC Darwin.  In these
>>    conventions, a foreign function that returned a structure received
>>    a pointer to an instance of that structure type as a first argument,
>>    and a function that received a structure argument by value received
>>    the structure's contents in 32-bit word-size integer chunks (regardless
>>    of the types or sizes of the structure's fields.)  Knowledge of these
>>    conventions was hardwired into various parts of the system (e.g.,
>>    the interface database), so that it was not generally possible to
>>    tell whether a given foreign function returned a structure type
>>    (or just happened to take an extra pointer argument.)
>>
>>    Unfortunately, there are at least 4 other sets of conventions for
>>    dealing with structure arguments/return values on the platforms
>>    that OpenMCL runs on (and even the DarwinPPC32 conventions weren't
>>    fully/correctly implemented.)  OpenMCL's FFI is generally pretty
>>    low-level, but to the extent that it's reasonable to talk about
>>    "higher level" constructs (EXTERNAL-CALL, SEND, FF-CALL, #_), those
>>    higher-level constructs try to enforce uniform syntax and try
>>    to hide the platform-specific details in backend-specific functions.
>>
>>    The impact of these changes should generally be pretty minimal.
>>    In a "higher-level" construct used to call a foreign function that
>>    returns a structure type, the first parameter in the call should
>>    be a pointer to an instance of that structure type.
>>
>>    For example, if a :rect structure is defined as:
>>
>>    (def-foreign-type nil
>>      (:struct :rect
>>        (:width :int)
>>        (:height :int)
>>        (:x :int)  ; x coordinate of origin
>>        (:y :int)))
>>
>>    and a foreign function named "inset_rect" takes a rect and an integer
>>    delta and returns a new :rect "inset" by that delta, a call to that
>>    foreign function might look like:
>>
>>    (rlet ((result :rect))
>>      (ff-call *address-of-inset-rect* result (:struct :rect) r :int delta :(:struct rect))
>>      ;; or, if "inset_rect" was declared in the interface database:
>>      (#_inset_rect result r delta))
>>
>>
>>    A callback that returns a :rect likewise should accept a pointer
>>    to an instance of the :rect type as a first (unqualified) argument
>>    and explicitly declare that it returns a (:STRUCT :RECT).
>>
>>    (defcallback *address-of-inset-rect (result (:struct :rect) r :int delta (:struct :rect))
>>      (setf (pref result :rect.x) (+ (pref r :rect.x) delta)
>>            (pref result :rect.y) (+ (pref r :rect.y) delta)
>>            (pref result :rect.width) (- (pref r :rect.width) (* 2 delta))
>>            (pref result :rect.height) (- (pref r :rect.height) (* 2 delta))))
>>
>>    Note that this is very similar to what's been (implicitly) supported
>>    on DarwinPPC32; the basic difference is that the return type
>>    ("(:STRUCT :RECT)") is explicitly specified (or, in the case of #_,
>>    specified in the interface database).  Whether the "result" pointer
>>    is actually passed as an argument or not is platform-dependent (on
>>    DarwinPPC64, the :rect structure would be "returned" by returning
>>    4 :int values in 4 different machine registers), but the same syntax
>>    can be used (and hides those details) on all platforms.
>>
>>    In the examples above, we said that the (presumed source) rectangle
>>    was passed by value as a value of type (:struct :rect), and we let
>>    the FFI deal with the details.  Historically, this parameter could
>>    have been specified as a small unsigned integer N (denoting the
>>    DarwinPPC32 convention of passing the structure value a N
>>    native-word-size integer arguments.)  Again, there are several
>>    different conventions for passing and receiving structure values,
>>    and it's best to let the FFI decide how to follow those conventions.
>>    (Some of those conventions are quite complicated, and depend on
>>    the size of the structure as well as the types of its fields.)
>>
>>    In all cases, a callback which declares a parameter to be of a
>>    structure type can treat that parameter as a pointer an instance of
>>    that structure type with fields initialized by the caller (as in
>>    the case of "r" in the example above.)
>>
>>    In the ObjC bridge, the DEFINE-OBJC-METHOD macro has always provided
>>    syntax for specifiying that the method "returns" a structure. (That
>>    syntax is (:struct <struct-type> <parameter-name>). That continues
>>    to be supported.
>>
>>    Apple's ObjC runtime provides different functions (#_objc_msgSend and
>>    #_objc_msgSend_stret) to handle the cases of sending messages which
>>    return non-structure and structure results.  These low-level functions
>>    are very sensitive to whether the structure is actually returned via
>>    an "invisible" first argument or not (this is only one of a few different
>>    conventions on some platforms.)  OpenMCL's ObjC bridge makes similar
>>    distinctions, but uses simple, consistent rules: a message that returns
>>    a structure should always be sent via SEND/STRET (or some variant of
>>    SEND/STRET) and should have a first parameter of type "pointer to
>>    returned structure type", regardless of whether or not that pointer
>>    is actually passed to the method implementation or just used as by
>>    some platform-specific code to transfer register values.)
>>
>>    The end result of all of this (several weeks of bootstrapping) is
>>    that most things are pretty much the same, at least on DarwinPPC32;
>>    only foreign function calls/callbacks that involve passing structures
>>    by value or returning structures need change at all, and the changes
>>    generally involve being more explicit/declarative about what's going
>>    on.  These changes -do- allow these idioms to be used on other
>>    (64-bit) platforms, and since they're heavily used in Apple GUI
>>    libraries and since 64-bit versions of Carbon and Cocoa are announced
>>    features of Leopard, it seemed appropriate to get support for this
>>    stuff into the FFI on those platforms and to try to do it in a way
>>    that hid the platform-dependent details.  (I didn't expect all of
>>    this to take so long.)
>>
>> - The initial listener PROCESS now persists across SAVE-APPLICATION.
>>    This means that (for instance):
>>
>>    ? (defvar *listener-process* (current-process))
>>    *LISTENER-PROCESS*
>>    ? (save-application "new.image")
>>    shell> openmcl new.image
>>    ? (eq (current-process) *listener-process*)
>>    T
>>    ;; though of course the underlying OS thread, stacks, etc are unlikely
>>    ;; to be "equal" in any sense.
>>
>>    The current process is sometimes used to mark "ownership" of thread-private
>>    hash-tables and streams.  (Even though it doesn't make much sense for
>>    STREAMs to persist across SAVE-APPLICATION, it does make sense for
>>    HASH-TABLEs to do so; HASH-TABLES created with the :PRIVATE T option
>>    and "owned" by the initial listener process continue to be owned by
>>    that the current listener process in the new image.)
>>
>> - All of the FFI changes above do seem to allow the Cocoa IDE example
>>    to run on ppc64/x86-64 (as well as ppc32) under Leopard, and
>>    hopefully that'll soon be true of applications generated via Mikel
>>    Evins' Bosco system as well.  The bridge and demo code have been
>>    conditionalized to support ObjC 2.0 on 64-bit systems, to avoid
>>    deprecated functions and methods, and to support 64-bit Cocoa
>>    changes.  Hopefully, this has been done in a way that doesn't break
>>    PPC32 Cocoa under Tiger (he said, quickly rushing to the nearest
>>    PPC32 Tiger machine and breathing a sigh of relief when the Cocoa
>>    listener appeared ..)  64-bit Cocoa sometimes used 64-bit signed and
>>    unsigned integers in place of 32-bit integers; accordingly, the
>>    foreign types :<NSI>nteger and :<NSUI>nteger are defined (as 32-bit
>>    signed/unsigned integers) on 32-bit platforms, and these types are
>>    used in some method and type definitions.  (Those integer types are
>>    predefined in Objc 2.0, and are 64 bits wide on 64-bit platforms.)
>>
>>    More pervasively (and a little more problematically), CoreGraphics
>>    (and things built on top of it, including Cocoa) uses double-floats
>>    instead of single-floats for many things on 64-bit hardware; the
>>    difference is abstracted (a little) via the new CGFloat type.
>>    This means that (for instance) code which initializes a constant-sized
>>    NSRect on a 32-bit machines and has traditionally done so via
>>    something like:
>>
>>    (ns-make-rect 0.0 0.0 500.0 200.0)
>>
>>    now needs to do something like:
>>
>>    (ns-make-rect (float 0.0 ccl::+cgfloat-zero+) ..)
>>
>>    in order to compile and run on both 32-bit and 64-bit platforms.
>>
>>    where ccl::+cgfloat-zero+ is defined as 1.0f0 on 32-bit platforms
>>    and as 1.0d0 on 64-bit machines.  Cases involving constants won't
>>    incur any runtime overhead and the occasional runtime overhead in
>>    other cases -probably- isn't that great in context (compared to
>>    initializing a view hierarchy ...)  but it's certainly ugly to
>>    look at.  It's possible that some of this ugliness could be
>>    hidden in the bridge/FFI (by making them do the necessary coercions
>>    for you), but there are tradeoffs there.
>>
>> - The ObjC bridge has had a long-standing bug whereby a standalone
>>    Cocoa application may have needed to find the interface databases
>>    at runtime in order for MAKE-OBJC-INSTANCE and MAKE-INSTANCE of
>>    an ObjC class to work.  (These functions needed to be able to
>>    send an "init" message to the newly-allocated instance, and needed
>>    to know the type signature of that init message in order to do that.)
>>    The current scheme tries to avoid this by pre-compiling helper
>>    functions to enable calling all known "init" message signatures.
>>    (More accurately, all fixed-argument "init" message signatures.)
>>    This scheme avoids the need to send messages whose argument
>>    and result types are computed at runtime (via %SEND), and %SEND
>>    (a) was known to be inefficient and (b) would have a lot of
>>    difficulty handling all known structure return/passing conventions
>>    on supported platforms.  Accordingly, %SEND has been deprecated
>>    (with extreme prejudice, e.g., removed.)
>>
>> - a couple of little functions are defined (but their names are
>>    not yet exported) on x86-64: ccl::rdtsc and ccl::rdtsc64 provide
>>    access to the values returned by on-chip cycle counting instructions.
>>    For instance:
>>
>> ? (let* ((start (ccl::rdtsc)))
>>      (sleep 1)
>>      (- (ccl::rdtsc) start))
>> 1995065244
>>
>>    Hmm.  Apparently, the 2.0GHz MacBook I tried that on is actually
>>    a 1.995GHz MacBook.
>>
>>    There are all kinds of ways for rdtsc to lose (and return
>>    inaccurate or misleading results): the cycle counters for
>>    each CPU core in a multi-core system aren't necessarily
>>    kept in sync, and many modern systems allow CPU clock rates
>>    to vary (for power-management reasons) and/or allow the CPU
>>    to sleep/hibernate.  OSes seem to offer some support for
>>    compensating for these effects, and it seems like ccl::rdtsc
>>    and ccl::rdtsc64 can be used to obtain interesting results.
>>
>>    The RDTSC instruction actually returns an unsigned 64-bit
>>    result; apparently, some Intel documentation claims that this
>>    value will not "wrap around" to 0 at contemporary clock rates
>>    for at least 10 years after the system was booted.  (If you can
>>    keep an Intel system running for 9 years between reboots, you
>>    might consider telling Intel that the RDTSC counter wrapped around
>>    a year early; they might give you a refund.  Or maybe not.)
>>    A non-negative OpenMCL64 fixnum is limited to 60 bits; the
>>    ccl::rdtsc function truncates the 64-bit counter value so
>>    that it fits in a non-negative fixnum; if the 10 year limit
>>    for the 64-bit value is accurate, the 60-bit value would
>>    wrap around after about 223 days of uptime.
>>
>>    ccl::rdtsc64 returns the full 64-bit counter value, but
>>    may return a bignum after 223 days of uptime.
>>
>> - lots of bug fixes (not all of which involved the FFI or ObjC
>>    bridge.)
>> ----------------------------------------------------------------
>>
>> Needless to say, I'm sorry that this took several times longer than
>> the "few days" that I thoght it would.
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>
>



More information about the Openmcl-devel mailing list