[Openmcl-devel] New snapshots available
Sunil Mishra
smishra at sfmishras.com
Wed Feb 14 08:41:50 PST 2007
Hi Gary,
Thanks for the new snapshot. The latest CVS isn't compiling for me. I
started up openmcl and ran (ccl:rebuild-ccl :full t). I am running Mac
OS X 10.4.8 on an intel macbook pro.
Here's a snippet of the trace:
;Compiling "/usr/local/src/ccl/compiler/X86/x86-backend.lisp"...
;Loading #P"/usr/local/src/ccl/bin/x86-backend.dx64fsl"...
;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp"...
;Compiler warnings for
"/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
; Function call arguments don't match current definition of MAKE-FTD
(3 references), in SETUP-X8664-FTD.
;Compiler warnings for
"/usr/local/src/ccl/compiler/X86/X8664/x8664-backend.lisp" :
; Undefined function REQUIRE-FOREIGN-TYPE-BITS, in
X8664::GENERATE-CALLBACK-BINDINGS.
; Undefined function NULL-COERCE-FOREIGN-RESULT, in X8664::EXPAND-FF-CALL.
; Undefined function NULL-COERCE-FOREIGN-ARG, in X8664::EXPAND-FF-CALL.
;Loading #P"/usr/local/src/ccl/bin/x8664-backend.dx64fsl"...
;Compiling "/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"...
> Error: subprim named .SPFFCALL-RETURN-REGISTERS not found.
> While executing: %SUBPRIM-NAME->OFFSET, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
1 > :b
(C26E28) : 0 (%SUBPRIM-NAME->OFFSET '.SPFFCALL-RETURN-REGISTERS
#(#<SUBPRIMITIVE-INFO .SPJMPSYM @ #x5000> #<SUBPRIMITIVE-INFO .SPJMPNFN
@ #x5008> #<SUBPRIMITIVE-INFO .SPFUNCALL @ #x5010> #<SUBPRIMITIVE-INFO
.SPMKCATCH1V @ #x5018> #<SUBPRIMITIVE-INFO .SPMKUNWIND @ #x5020> ...)) 261
(C26E50) : 1 (FUNCALL #'#<FIND-NAME> '.SPFFCALL-RETURN-REGISTERS) 125
(C26E80) : 2 (FUNCALL #'#<SIMPLIFY-MEMORY-OPERAND>
'(.SPFFCALL-RETURN-REGISTERS)) 1061
(C26EF0) : 3 (FUNCALL #'#<SIMPLIFY-OPERAND> '(:@
.SPFFCALL-RETURN-REGISTERS)) 197
(C26F28) : 4 (FUNCALL #'#<SIMPLIFY-FORM> '(JMP #)) 1461
(C26FE8) : 5 (DEFINE-X86-VINSN #<BACKEND DARWINX8664 #x30004076ECCD>
'FF-CALL-RETURN-REGISTERS NIL NIL NIL '(# # # # :BACK)) 4037
(C27160) : 6 (FUNCALL #'#<DEFINE-X8664-VINSN> ???) 509
(C271D0) : 7 (FUNCALL #<Compiled-function DEFINE-X8664-VINSN
(Non-Global) #x300041B1B54F> [...]) 157
(C27200) : 8 (MACROEXPAND-1 '(DEFINE-X8664-VINSN # # # # ...) [...]) 301
(C27230) : 9 (FCOMP-FORM-1 '(DEFINE-X8664-VINSN # # # # ...)
#<LEXICAL-ENVIRONMENT #x300041B0F1BD> :NOT-COMPILE-TIME) 661
(C27270) : 10 (FCOMP-FORM '(DEFINE-X8664-VINSN # # # # ...)
#<LEXICAL-ENVIRONMENT #x300041B0F1BD> :NOT-COMPILE-TIME) 2117
(C272F8) : 11 (FCOMP-FORM '(DEFINE-X8664-SUBPRIM-CALL-VINSN #
.SPFFCALL-RETURN-REGISTERS) #<LEXICAL-ENVIRONMENT #x300041B0F1BD>
:NOT-COMPILE-TIME) 2117
(C27380) : 12 (FCOMP-READ-LOOP
"/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
#P"ccl:compiler;X86;X8664;x8664-vinsns.lisp.newest"
#<LEXICAL-ENVIRONMENT #x300041B0F1BD> :NOT-COMPILE-TIME) 1821
(C27498) : 13 (FCOMP-FILE
"/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
#P"ccl:compiler;X86;X8664;x8664-vinsns.lisp.newest"
#<LEXICAL-ENVIRONMENT #x300041B0F1BD>) 821
(C27630) : 14 (%COMPILE-FILE
"/usr/local/src/ccl/compiler/X86/X8664/x8664-vinsns.lisp"
"/usr/local/src/ccl/bin/x8664-vinsns.dx64fsl" T NIL T NIL T T NIL NIL
#<BACKEND DARWINX8664 #x30004076ECCD> :DEFAULT) 2525
(C278B8) : 15 (COMPILE-FILE "ccl:compiler;X86;X8664;x8664-vinsns.lisp"
[...]) 1237
(C279C8) : 16 (UPDATE-MODULES '(X86-ARCH X86-ASM X86-LAP X8664-ARCH
X8664ENV ...) [...]) 597
(C27A60) : 17 (COMPILE-CCL [...]) 253
(C27A80) : 18 (REBUILD-CCL [...]) 1941
(C27B68) : 19 (CALL-CHECK-REGS 'REBUILD-CCL [...]) 229
(C27BA0) : 20 (TOPLEVEL-EVAL '(REBUILD-CCL :FULL T) [...]) 429
(C27BF0) : 21 (READ-LOOP [...]) 1845
(C27DF8) : 22 (TOPLEVEL-LOOP) 197
(C27E28) : 23 (FUNCALL #'#<Anonymous Function #x30004042932F>) 133
(C27E40) : 24 (FUNCALL #'#<Anonymous Function #x300040575AEF>) 773
(C27EC8) : 25 (RUN-PROCESS-INITIAL-FORM #<PROCESS listener(1) [Active]
#x300040D43B7D> '(#)) 701
(C27F48) : 26 (FUNCALL #'#<Anonymous Function #x30004044088F> #<PROCESS
listener(1) [Active] #x300040D43B7D> '(#)) 397
(C27F98) : 27 (FUNCALL #'#<Anonymous Function #x3000403F56CF>) 365
Gary Byers wrote:
> There are now new (070214) self-contained archives containing OpenMCL
> source, binaries, and interfaces for DarwinPPC32/64, LinuxPPC32/64,
> DarwinX8664, LinuxX8664, and FreeBSDX8664 available in
> <ftp://clozure.com/pub/testing>. The release notes entry says:
>
> OpenMCL 1.1-pre-070214
> - The FASL version changed (old FASL files won't work with this
> lisp version), as did the version information which tries to
> keep the kernel in sync with heap images.
> - There are new interface files for all platforms. These files
> encode some foreign type information a little differently
> than older ones did (notably information about foreign functions
> that return structures or accept structure args by value.) The
> new .cdb files can't be used by older versions of OpenMCL; using
> older .cdb files with this version is "allowed, but not supported
> or recommended."
> - Almost all of the changes in functionality since the last (061231)
> snapshots and since the CVS freeze on 070117 have to do with
> relatively obscure issues having to do with passing structures
> to foreign functions by value and/or returning structures from foreign
> function calls.
>
> These idioms are fairly rare in traditional C code (though it's
> fairly common to pass -pointers- to structures by reference
> and sometimes to return pointers to structures. There are
> a few C compiler runtime routines that perform some flavor
> of integer division and return a two-element structure that
> contains "quotient" and "remainder" fields, but that's typically
> about the extent of the use of this idiom.) The idioms are used
> much more often in Apple's Carbon and Cooca libraries and in
> some of the frameworks (CoreGraphics, CoreFoundation) that those
> libraries are based on.
>
> OpenMCL's FFI has provided some support for this in the past;
> notably, it's provided support for (most of the) structure-returning
> and struct-by-value conventions used on 32-bit PPC Darwin. In these
> conventions, a foreign function that returned a structure received
> a pointer to an instance of that structure type as a first argument,
> and a function that received a structure argument by value received
> the structure's contents in 32-bit word-size integer chunks (regardless
> of the types or sizes of the structure's fields.) Knowledge of these
> conventions was hardwired into various parts of the system (e.g.,
> the interface database), so that it was not generally possible to
> tell whether a given foreign function returned a structure type
> (or just happened to take an extra pointer argument.)
>
> Unfortunately, there are at least 4 other sets of conventions for
> dealing with structure arguments/return values on the platforms
> that OpenMCL runs on (and even the DarwinPPC32 conventions weren't
> fully/correctly implemented.) OpenMCL's FFI is generally pretty
> low-level, but to the extent that it's reasonable to talk about
> "higher level" constructs (EXTERNAL-CALL, SEND, FF-CALL, #_), those
> higher-level constructs try to enforce uniform syntax and try
> to hide the platform-specific details in backend-specific functions.
>
> The impact of these changes should generally be pretty minimal.
> In a "higher-level" construct used to call a foreign function that
> returns a structure type, the first parameter in the call should
> be a pointer to an instance of that structure type.
>
> For example, if a :rect structure is defined as:
>
> (def-foreign-type nil
> (:struct :rect
> (:width :int)
> (:height :int)
> (:x :int) ; x coordinate of origin
> (:y :int)))
>
> and a foreign function named "inset_rect" takes a rect and an integer
> delta and returns a new :rect "inset" by that delta, a call to that
> foreign function might look like:
>
> (rlet ((result :rect))
> (ff-call *address-of-inset-rect* result (:struct :rect) r :int delta :(:struct rect))
> ;; or, if "inset_rect" was declared in the interface database:
> (#_inset_rect result r delta))
>
>
> A callback that returns a :rect likewise should accept a pointer
> to an instance of the :rect type as a first (unqualified) argument
> and explicitly declare that it returns a (:STRUCT :RECT).
>
> (defcallback *address-of-inset-rect (result (:struct :rect) r :int delta (:struct :rect))
> (setf (pref result :rect.x) (+ (pref r :rect.x) delta)
> (pref result :rect.y) (+ (pref r :rect.y) delta)
> (pref result :rect.width) (- (pref r :rect.width) (* 2 delta))
> (pref result :rect.height) (- (pref r :rect.height) (* 2 delta))))
>
> Note that this is very similar to what's been (implicitly) supported
> on DarwinPPC32; the basic difference is that the return type
> ("(:STRUCT :RECT)") is explicitly specified (or, in the case of #_,
> specified in the interface database). Whether the "result" pointer
> is actually passed as an argument or not is platform-dependent (on
> DarwinPPC64, the :rect structure would be "returned" by returning
> 4 :int values in 4 different machine registers), but the same syntax
> can be used (and hides those details) on all platforms.
>
> In the examples above, we said that the (presumed source) rectangle
> was passed by value as a value of type (:struct :rect), and we let
> the FFI deal with the details. Historically, this parameter could
> have been specified as a small unsigned integer N (denoting the
> DarwinPPC32 convention of passing the structure value a N
> native-word-size integer arguments.) Again, there are several
> different conventions for passing and receiving structure values,
> and it's best to let the FFI decide how to follow those conventions.
> (Some of those conventions are quite complicated, and depend on
> the size of the structure as well as the types of its fields.)
>
> In all cases, a callback which declares a parameter to be of a
> structure type can treat that parameter as a pointer an instance of
> that structure type with fields initialized by the caller (as in
> the case of "r" in the example above.)
>
> In the ObjC bridge, the DEFINE-OBJC-METHOD macro has always provided
> syntax for specifiying that the method "returns" a structure. (That
> syntax is (:struct <struct-type> <parameter-name>). That continues
> to be supported.
>
> Apple's ObjC runtime provides different functions (#_objc_msgSend and
> #_objc_msgSend_stret) to handle the cases of sending messages which
> return non-structure and structure results. These low-level functions
> are very sensitive to whether the structure is actually returned via
> an "invisible" first argument or not (this is only one of a few different
> conventions on some platforms.) OpenMCL's ObjC bridge makes similar
> distinctions, but uses simple, consistent rules: a message that returns
> a structure should always be sent via SEND/STRET (or some variant of
> SEND/STRET) and should have a first parameter of type "pointer to
> returned structure type", regardless of whether or not that pointer
> is actually passed to the method implementation or just used as by
> some platform-specific code to transfer register values.)
>
> The end result of all of this (several weeks of bootstrapping) is
> that most things are pretty much the same, at least on DarwinPPC32;
> only foreign function calls/callbacks that involve passing structures
> by value or returning structures need change at all, and the changes
> generally involve being more explicit/declarative about what's going
> on. These changes -do- allow these idioms to be used on other
> (64-bit) platforms, and since they're heavily used in Apple GUI
> libraries and since 64-bit versions of Carbon and Cocoa are announced
> features of Leopard, it seemed appropriate to get support for this
> stuff into the FFI on those platforms and to try to do it in a way
> that hid the platform-dependent details. (I didn't expect all of
> this to take so long.)
>
> - The initial listener PROCESS now persists across SAVE-APPLICATION.
> This means that (for instance):
>
> ? (defvar *listener-process* (current-process))
> *LISTENER-PROCESS*
> ? (save-application "new.image")
> shell> openmcl new.image
> ? (eq (current-process) *listener-process*)
> T
> ;; though of course the underlying OS thread, stacks, etc are unlikely
> ;; to be "equal" in any sense.
>
> The current process is sometimes used to mark "ownership" of thread-private
> hash-tables and streams. (Even though it doesn't make much sense for
> STREAMs to persist across SAVE-APPLICATION, it does make sense for
> HASH-TABLEs to do so; HASH-TABLES created with the :PRIVATE T option
> and "owned" by the initial listener process continue to be owned by
> that the current listener process in the new image.)
>
> - All of the FFI changes above do seem to allow the Cocoa IDE example
> to run on ppc64/x86-64 (as well as ppc32) under Leopard, and
> hopefully that'll soon be true of applications generated via Mikel
> Evins' Bosco system as well. The bridge and demo code have been
> conditionalized to support ObjC 2.0 on 64-bit systems, to avoid
> deprecated functions and methods, and to support 64-bit Cocoa
> changes. Hopefully, this has been done in a way that doesn't break
> PPC32 Cocoa under Tiger (he said, quickly rushing to the nearest
> PPC32 Tiger machine and breathing a sigh of relief when the Cocoa
> listener appeared ..) 64-bit Cocoa sometimes used 64-bit signed and
> unsigned integers in place of 32-bit integers; accordingly, the
> foreign types :<NSI>nteger and :<NSUI>nteger are defined (as 32-bit
> signed/unsigned integers) on 32-bit platforms, and these types are
> used in some method and type definitions. (Those integer types are
> predefined in Objc 2.0, and are 64 bits wide on 64-bit platforms.)
>
> More pervasively (and a little more problematically), CoreGraphics
> (and things built on top of it, including Cocoa) uses double-floats
> instead of single-floats for many things on 64-bit hardware; the
> difference is abstracted (a little) via the new CGFloat type.
> This means that (for instance) code which initializes a constant-sized
> NSRect on a 32-bit machines and has traditionally done so via
> something like:
>
> (ns-make-rect 0.0 0.0 500.0 200.0)
>
> now needs to do something like:
>
> (ns-make-rect (float 0.0 ccl::+cgfloat-zero+) ..)
>
> in order to compile and run on both 32-bit and 64-bit platforms.
>
> where ccl::+cgfloat-zero+ is defined as 1.0f0 on 32-bit platforms
> and as 1.0d0 on 64-bit machines. Cases involving constants won't
> incur any runtime overhead and the occasional runtime overhead in
> other cases -probably- isn't that great in context (compared to
> initializing a view hierarchy ...) but it's certainly ugly to
> look at. It's possible that some of this ugliness could be
> hidden in the bridge/FFI (by making them do the necessary coercions
> for you), but there are tradeoffs there.
>
> - The ObjC bridge has had a long-standing bug whereby a standalone
> Cocoa application may have needed to find the interface databases
> at runtime in order for MAKE-OBJC-INSTANCE and MAKE-INSTANCE of
> an ObjC class to work. (These functions needed to be able to
> send an "init" message to the newly-allocated instance, and needed
> to know the type signature of that init message in order to do that.)
> The current scheme tries to avoid this by pre-compiling helper
> functions to enable calling all known "init" message signatures.
> (More accurately, all fixed-argument "init" message signatures.)
> This scheme avoids the need to send messages whose argument
> and result types are computed at runtime (via %SEND), and %SEND
> (a) was known to be inefficient and (b) would have a lot of
> difficulty handling all known structure return/passing conventions
> on supported platforms. Accordingly, %SEND has been deprecated
> (with extreme prejudice, e.g., removed.)
>
> - a couple of little functions are defined (but their names are
> not yet exported) on x86-64: ccl::rdtsc and ccl::rdtsc64 provide
> access to the values returned by on-chip cycle counting instructions.
> For instance:
>
> ? (let* ((start (ccl::rdtsc)))
> (sleep 1)
> (- (ccl::rdtsc) start))
> 1995065244
>
> Hmm. Apparently, the 2.0GHz MacBook I tried that on is actually
> a 1.995GHz MacBook.
>
> There are all kinds of ways for rdtsc to lose (and return
> inaccurate or misleading results): the cycle counters for
> each CPU core in a multi-core system aren't necessarily
> kept in sync, and many modern systems allow CPU clock rates
> to vary (for power-management reasons) and/or allow the CPU
> to sleep/hibernate. OSes seem to offer some support for
> compensating for these effects, and it seems like ccl::rdtsc
> and ccl::rdtsc64 can be used to obtain interesting results.
>
> The RDTSC instruction actually returns an unsigned 64-bit
> result; apparently, some Intel documentation claims that this
> value will not "wrap around" to 0 at contemporary clock rates
> for at least 10 years after the system was booted. (If you can
> keep an Intel system running for 9 years between reboots, you
> might consider telling Intel that the RDTSC counter wrapped around
> a year early; they might give you a refund. Or maybe not.)
> A non-negative OpenMCL64 fixnum is limited to 60 bits; the
> ccl::rdtsc function truncates the 64-bit counter value so
> that it fits in a non-negative fixnum; if the 10 year limit
> for the 64-bit value is accurate, the 60-bit value would
> wrap around after about 223 days of uptime.
>
> ccl::rdtsc64 returns the full 64-bit counter value, but
> may return a bignum after 223 days of uptime.
>
> - lots of bug fixes (not all of which involved the FFI or ObjC
> bridge.)
> ----------------------------------------------------------------
>
> Needless to say, I'm sorry that this took several times longer than
> the "few days" that I thoght it would.
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
More information about the Openmcl-devel
mailing list