[Openmcl-devel] New snapshots available

Sunil Mishra smishra at sfmishras.com
Wed Feb 14 16:41:50 UTC 2007

Hi Gary,

Thanks for the new snapshot. The latest CVS isn't compiling for me. I
started up openmcl and ran (ccl:rebuild-ccl :full t). I am running Mac
OS X 10.4.8 on an intel macbook pro.

Here's a snippet of the trace:

;Compiling "/usr/local/src/ccl/compiler/X86/x86-backend.lisp"...
;Loading #P"/usr/local/src/ccl/bin/x86-backend.dx64fsl"...
;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp"...
;Compiler warnings for
"/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
;   Function call arguments don't match current definition of MAKE-FTD
(3 references), in SETUP-X8664-FTD.
;Compiler warnings for
"/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
;   Undefined function REQUIRE-FOREIGN-TYPE-BITS, in
;   Undefined function NULL-COERCE-FOREIGN-RESULT, in X8664::EXPAND-FF-CALL.
;   Undefined function NULL-COERCE-FOREIGN-ARG, in X8664::EXPAND-FF-CALL.
;Loading #P"/usr/local/src/ccl/bin/x8664-backend.dx64fsl"...
;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"...
> Error: subprim named .SPFFCALL-RETURN-REGISTERS not found.
> While executing: %SUBPRIM-NAME->OFFSET, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
1 > :b
.SPMKCATCH1V @ #x5018> #<SUBPRIMITIVE-INFO .SPMKUNWIND @ #x5020> ...)) 261
(C26F28) : 4 (FUNCALL #'#<SIMPLIFY-FORM> '(JMP #)) 1461
(C26FE8) : 5 (DEFINE-X86-VINSN #<BACKEND DARWINX8664 #x30004076ECCD>
(C27160) : 6 (FUNCALL #'#<DEFINE-X8664-VINSN> ???) 509
(C271D0) : 7 (FUNCALL #<Compiled-function DEFINE-X8664-VINSN
(Non-Global)  #x300041B1B54F> [...]) 157
(C27200) : 8 (MACROEXPAND-1 '(DEFINE-X8664-VINSN # # # # ...) [...]) 301
(C27230) : 9 (FCOMP-FORM-1 '(DEFINE-X8664-VINSN # # # # ...)
(C27270) : 10 (FCOMP-FORM '(DEFINE-X8664-VINSN # # # # ...)
(C27380) : 12 (FCOMP-READ-LOOP
(C27498) : 13 (FCOMP-FILE
(C27630) : 14 (%COMPILE-FILE
"/usr/local/src/ccl/bin/x8664-vinsns.dx64fsl" T NIL T NIL T T NIL NIL
#<BACKEND DARWINX8664 #x30004076ECCD> :DEFAULT) 2525
(C278B8) : 15 (COMPILE-FILE "ccl:compiler;X86;X8664;x8664-vinsns.lisp"
[...]) 1237
(C279C8) : 16 (UPDATE-MODULES '(X86-ARCH X86-ASM X86-LAP X8664-ARCH
X8664ENV ...) [...]) 597
(C27A60) : 17 (COMPILE-CCL [...]) 253
(C27A80) : 18 (REBUILD-CCL [...]) 1941
(C27B68) : 19 (CALL-CHECK-REGS 'REBUILD-CCL [...]) 229
(C27BA0) : 20 (TOPLEVEL-EVAL '(REBUILD-CCL :FULL T) [...]) 429
(C27BF0) : 21 (READ-LOOP [...]) 1845
(C27DF8) : 22 (TOPLEVEL-LOOP) 197
(C27E28) : 23 (FUNCALL #'#<Anonymous Function #x30004042932F>) 133
(C27E40) : 24 (FUNCALL #'#<Anonymous Function #x300040575AEF>) 773
(C27EC8) : 25 (RUN-PROCESS-INITIAL-FORM #<PROCESS listener(1) [Active]
#x300040D43B7D> '(#)) 701
(C27F48) : 26 (FUNCALL #'#<Anonymous Function #x30004044088F> #<PROCESS
listener(1) [Active] #x300040D43B7D> '(#)) 397
(C27F98) : 27 (FUNCALL #'#<Anonymous Function #x3000403F56CF>) 365

Gary Byers wrote:
> There are now new (070214) self-contained archives containing OpenMCL
> source, binaries, and interfaces for DarwinPPC32/64, LinuxPPC32/64,
> DarwinX8664, LinuxX8664, and FreeBSDX8664 available in
> <ftp://clozure.com/pub/testing>.  The release notes entry says:
> OpenMCL 1.1-pre-070214
> - The FASL version changed (old FASL files won't work with this
>    lisp version), as did the version information which tries to
>    keep the kernel in sync with heap images.
> - There are new interface files for all platforms.  These files
>    encode some foreign type information a little differently
>    than older ones did (notably information about foreign functions
>    that return structures or accept structure args by value.)  The
>    new .cdb files can't be used by older versions of OpenMCL; using
>    older .cdb files with this version is "allowed, but not supported
>    or recommended."
> - Almost all of the changes in functionality since the last (061231)
>    snapshots and since the CVS freeze on 070117 have to do with
>    relatively obscure issues having to do with passing structures
>    to foreign functions by value and/or returning structures from foreign
>    function calls.
>    These idioms are fairly rare in traditional C code (though it's
>    fairly common to pass -pointers- to structures by reference
>    and sometimes to return pointers to structures.  There are
>    a few C compiler runtime routines that perform some flavor
>    of integer division and return a two-element structure that
>    contains "quotient" and "remainder" fields, but that's typically
>    about the extent of the use of this idiom.)  The idioms are used
>    much more often in Apple's Carbon and Cooca libraries and in
>    some of the frameworks (CoreGraphics, CoreFoundation) that those
>    libraries are based on.
>    OpenMCL's FFI has provided some support for this in the past;
>    notably, it's provided support for (most of the) structure-returning
>    and struct-by-value conventions used on 32-bit PPC Darwin.  In these
>    conventions, a foreign function that returned a structure received
>    a pointer to an instance of that structure type as a first argument,
>    and a function that received a structure argument by value received
>    the structure's contents in 32-bit word-size integer chunks (regardless
>    of the types or sizes of the structure's fields.)  Knowledge of these
>    conventions was hardwired into various parts of the system (e.g.,
>    the interface database), so that it was not generally possible to
>    tell whether a given foreign function returned a structure type
>    (or just happened to take an extra pointer argument.)
>    Unfortunately, there are at least 4 other sets of conventions for
>    dealing with structure arguments/return values on the platforms
>    that OpenMCL runs on (and even the DarwinPPC32 conventions weren't
>    fully/correctly implemented.)  OpenMCL's FFI is generally pretty
>    low-level, but to the extent that it's reasonable to talk about
>    "higher level" constructs (EXTERNAL-CALL, SEND, FF-CALL, #_), those
>    higher-level constructs try to enforce uniform syntax and try
>    to hide the platform-specific details in backend-specific functions.
>    The impact of these changes should generally be pretty minimal.
>    In a "higher-level" construct used to call a foreign function that
>    returns a structure type, the first parameter in the call should
>    be a pointer to an instance of that structure type.
>    For example, if a :rect structure is defined as:
>    (def-foreign-type nil
>      (:struct :rect
>        (:width :int)
>        (:height :int)
>        (:x :int)  ; x coordinate of origin
>        (:y :int)))
>    and a foreign function named "inset_rect" takes a rect and an integer
>    delta and returns a new :rect "inset" by that delta, a call to that
>    foreign function might look like:
>    (rlet ((result :rect))
>      (ff-call *address-of-inset-rect* result (:struct :rect) r :int delta :(:struct rect))
>      ;; or, if "inset_rect" was declared in the interface database:
>      (#_inset_rect result r delta))
>    A callback that returns a :rect likewise should accept a pointer
>    to an instance of the :rect type as a first (unqualified) argument
>    and explicitly declare that it returns a (:STRUCT :RECT).
>    (defcallback *address-of-inset-rect (result (:struct :rect) r :int delta (:struct :rect))
>      (setf (pref result :rect.x) (+ (pref r :rect.x) delta)
>            (pref result :rect.y) (+ (pref r :rect.y) delta)
>            (pref result :rect.width) (- (pref r :rect.width) (* 2 delta))
>            (pref result :rect.height) (- (pref r :rect.height) (* 2 delta))))
>    Note that this is very similar to what's been (implicitly) supported
>    on DarwinPPC32; the basic difference is that the return type
>    ("(:STRUCT :RECT)") is explicitly specified (or, in the case of #_,
>    specified in the interface database).  Whether the "result" pointer
>    is actually passed as an argument or not is platform-dependent (on
>    DarwinPPC64, the :rect structure would be "returned" by returning
>    4 :int values in 4 different machine registers), but the same syntax
>    can be used (and hides those details) on all platforms.
>    In the examples above, we said that the (presumed source) rectangle
>    was passed by value as a value of type (:struct :rect), and we let
>    the FFI deal with the details.  Historically, this parameter could
>    have been specified as a small unsigned integer N (denoting the
>    DarwinPPC32 convention of passing the structure value a N
>    native-word-size integer arguments.)  Again, there are several
>    different conventions for passing and receiving structure values,
>    and it's best to let the FFI decide how to follow those conventions.
>    (Some of those conventions are quite complicated, and depend on
>    the size of the structure as well as the types of its fields.)
>    In all cases, a callback which declares a parameter to be of a
>    structure type can treat that parameter as a pointer an instance of
>    that structure type with fields initialized by the caller (as in
>    the case of "r" in the example above.)
>    In the ObjC bridge, the DEFINE-OBJC-METHOD macro has always provided
>    syntax for specifiying that the method "returns" a structure. (That
>    syntax is (:struct <struct-type> <parameter-name>). That continues
>    to be supported.
>    Apple's ObjC runtime provides different functions (#_objc_msgSend and
>    #_objc_msgSend_stret) to handle the cases of sending messages which
>    return non-structure and structure results.  These low-level functions
>    are very sensitive to whether the structure is actually returned via
>    an "invisible" first argument or not (this is only one of a few different
>    conventions on some platforms.)  OpenMCL's ObjC bridge makes similar
>    distinctions, but uses simple, consistent rules: a message that returns
>    a structure should always be sent via SEND/STRET (or some variant of
>    SEND/STRET) and should have a first parameter of type "pointer to
>    returned structure type", regardless of whether or not that pointer
>    is actually passed to the method implementation or just used as by
>    some platform-specific code to transfer register values.)
>    The end result of all of this (several weeks of bootstrapping) is
>    that most things are pretty much the same, at least on DarwinPPC32;
>    only foreign function calls/callbacks that involve passing structures
>    by value or returning structures need change at all, and the changes
>    generally involve being more explicit/declarative about what's going
>    on.  These changes -do- allow these idioms to be used on other
>    (64-bit) platforms, and since they're heavily used in Apple GUI
>    libraries and since 64-bit versions of Carbon and Cocoa are announced
>    features of Leopard, it seemed appropriate to get support for this
>    stuff into the FFI on those platforms and to try to do it in a way
>    that hid the platform-dependent details.  (I didn't expect all of
>    this to take so long.)
> - The initial listener PROCESS now persists across SAVE-APPLICATION.
>    This means that (for instance):
>    ? (defvar *listener-process* (current-process))
>    ? (save-application "new.image")
>    shell> openmcl new.image
>    ? (eq (current-process) *listener-process*)
>    T
>    ;; though of course the underlying OS thread, stacks, etc are unlikely
>    ;; to be "equal" in any sense.
>    The current process is sometimes used to mark "ownership" of thread-private
>    hash-tables and streams.  (Even though it doesn't make much sense for
>    STREAMs to persist across SAVE-APPLICATION, it does make sense for
>    HASH-TABLEs to do so; HASH-TABLES created with the :PRIVATE T option
>    and "owned" by the initial listener process continue to be owned by
>    that the current listener process in the new image.)
> - All of the FFI changes above do seem to allow the Cocoa IDE example
>    to run on ppc64/x86-64 (as well as ppc32) under Leopard, and
>    hopefully that'll soon be true of applications generated via Mikel
>    Evins' Bosco system as well.  The bridge and demo code have been
>    conditionalized to support ObjC 2.0 on 64-bit systems, to avoid
>    deprecated functions and methods, and to support 64-bit Cocoa
>    changes.  Hopefully, this has been done in a way that doesn't break
>    PPC32 Cocoa under Tiger (he said, quickly rushing to the nearest
>    PPC32 Tiger machine and breathing a sigh of relief when the Cocoa
>    listener appeared ..)  64-bit Cocoa sometimes used 64-bit signed and
>    unsigned integers in place of 32-bit integers; accordingly, the
>    foreign types :<NSI>nteger and :<NSUI>nteger are defined (as 32-bit
>    signed/unsigned integers) on 32-bit platforms, and these types are
>    used in some method and type definitions.  (Those integer types are
>    predefined in Objc 2.0, and are 64 bits wide on 64-bit platforms.)
>    More pervasively (and a little more problematically), CoreGraphics
>    (and things built on top of it, including Cocoa) uses double-floats
>    instead of single-floats for many things on 64-bit hardware; the
>    difference is abstracted (a little) via the new CGFloat type.
>    This means that (for instance) code which initializes a constant-sized
>    NSRect on a 32-bit machines and has traditionally done so via
>    something like:
>    (ns-make-rect 0.0 0.0 500.0 200.0)
>    now needs to do something like:
>    (ns-make-rect (float 0.0 ccl::+cgfloat-zero+) ..)
>    in order to compile and run on both 32-bit and 64-bit platforms.
>    where ccl::+cgfloat-zero+ is defined as 1.0f0 on 32-bit platforms
>    and as 1.0d0 on 64-bit machines.  Cases involving constants won't
>    incur any runtime overhead and the occasional runtime overhead in
>    other cases -probably- isn't that great in context (compared to
>    initializing a view hierarchy ...)  but it's certainly ugly to
>    look at.  It's possible that some of this ugliness could be
>    hidden in the bridge/FFI (by making them do the necessary coercions
>    for you), but there are tradeoffs there.
> - The ObjC bridge has had a long-standing bug whereby a standalone
>    Cocoa application may have needed to find the interface databases
>    at runtime in order for MAKE-OBJC-INSTANCE and MAKE-INSTANCE of
>    an ObjC class to work.  (These functions needed to be able to
>    send an "init" message to the newly-allocated instance, and needed
>    to know the type signature of that init message in order to do that.)
>    The current scheme tries to avoid this by pre-compiling helper
>    functions to enable calling all known "init" message signatures.
>    (More accurately, all fixed-argument "init" message signatures.)
>    This scheme avoids the need to send messages whose argument
>    and result types are computed at runtime (via %SEND), and %SEND
>    (a) was known to be inefficient and (b) would have a lot of
>    difficulty handling all known structure return/passing conventions
>    on supported platforms.  Accordingly, %SEND has been deprecated
>    (with extreme prejudice, e.g., removed.)
> - a couple of little functions are defined (but their names are
>    not yet exported) on x86-64: ccl::rdtsc and ccl::rdtsc64 provide
>    access to the values returned by on-chip cycle counting instructions.
>    For instance:
> ? (let* ((start (ccl::rdtsc)))
>      (sleep 1)
>      (- (ccl::rdtsc) start))
> 1995065244
>    Hmm.  Apparently, the 2.0GHz MacBook I tried that on is actually
>    a 1.995GHz MacBook.
>    There are all kinds of ways for rdtsc to lose (and return
>    inaccurate or misleading results): the cycle counters for
>    each CPU core in a multi-core system aren't necessarily
>    kept in sync, and many modern systems allow CPU clock rates
>    to vary (for power-management reasons) and/or allow the CPU
>    to sleep/hibernate.  OSes seem to offer some support for
>    compensating for these effects, and it seems like ccl::rdtsc
>    and ccl::rdtsc64 can be used to obtain interesting results.
>    The RDTSC instruction actually returns an unsigned 64-bit
>    result; apparently, some Intel documentation claims that this
>    value will not "wrap around" to 0 at contemporary clock rates
>    for at least 10 years after the system was booted.  (If you can
>    keep an Intel system running for 9 years between reboots, you
>    might consider telling Intel that the RDTSC counter wrapped around
>    a year early; they might give you a refund.  Or maybe not.)
>    A non-negative OpenMCL64 fixnum is limited to 60 bits; the
>    ccl::rdtsc function truncates the 64-bit counter value so
>    that it fits in a non-negative fixnum; if the 10 year limit
>    for the 64-bit value is accurate, the 60-bit value would
>    wrap around after about 223 days of uptime.
>    ccl::rdtsc64 returns the full 64-bit counter value, but
>    may return a bignum after 223 days of uptime.
> - lots of bug fixes (not all of which involved the FFI or ObjC
>    bridge.) 
> ----------------------------------------------------------------
> Needless to say, I'm sorry that this took several times longer than
> the "few days" that I thoght it would.
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel

More information about the Openmcl-devel mailing list