[Openmcl-devel] dfsl/foreign function oddities

[Gary Byers]

One potential source of confusion that sometimes leads to symptoms
like those that you describe stems from the fact that #_ is a reader
macro (and therefore does most of its work at read-time.).  The act of
reading #_foo has the (important!) side-effect of defining a global
macro function on the symbol OS::|foo| as well as the effect of
returning that symbol.

If you compile a source file that does something like:

(defun allocate-initialized-pointer (n ch)
   "Allocate foreign memory, fill it with the specified character value"
   (let* ((p (#_malloc n)))
     (unless (%null-ptr-p p)
       (#_memset p (char-code ch) n)
        p)))

then the side-effects of defining the macros OS::|malloc| and OS::|memset|
take place just prior to the time that the definition is compiled.  READ
will return something like:

(DEFUN ALLOCATE-INITIALIZED-POINTER (N CH)
   "Allocate foreign memory, fill it with the specified character value"
   (LET* ((P (OS::|malloc| N)))
     (UNLESS (%NULL-PTR-P P)
       (OS::|memset| P (CHAR-CODE CH) N)
        P)))

(this is what the compiler sees), and the macros OS::|malloc| and
OS::|memset| will have just been defined and things should work as 
expected.

It's sometimes tempting to define a little utility like the example
as a macro:

(defmacro allocate-initialized-pointer (n ch)
   "Allocate foreign memory, fill it with the specified character value"
   (let* ((p (gensym)))
     `(let* ((,p (#_malloc ,n)))
        (unless (%null-ptr-p ,p)
          (#_memset ,p (char-code ,ch) ,n)
          ,p))))

If you store that macro definition if file A.lisp, then the read-time
side-effects of defining global macro functions on OS::|malloc| and
OS:|memset| will occur -when A.lisp is compiled- (or, slightly more
accurately, when A.lisp is read.)  The FASL file will store the 
compiled equivalent of:

(DEFMACRO ALLOCATE-INITIALIZED-POINTER (N CH)
   "Allocate foreign memory, fill it with the specified character value"
   (LET* ((P (GENSYM)))
     `(LET* ((,P (OS::|malloc| ,N)))
        (UNLESS (%NULL-PTR-P ,P)
          (OS:|memset| ,P (CHAR-CODE ,CH) ,N)
          ,P))))

but nothing in the FASL file does anything to ensure that those
symbols (OS::|malloc|/OS::|memset|) have global macro definitions (that
step happens as a side-effect of the #_ reader macro.)

If B.lisp contains something like:

(let* ((block-of-x (allocate-initialized-pointer 256 #\x)))
   ...)

then whether or not that works depends on whether or not A.lisp and B.lisp
were compiled in the same session; if not, then it's likely that the OS
package symbols won't have any kind of global definitions and the compiler
will compile calls to unknown (and nonexistent) functions.

Like I said, that scenario would explain the symptoms that you describe;
I think that the manual actually tries to explain the issue, but the prose
may be a little terse and cryptic.

An example of a workaround is the macro MAKE-RECORD (defined in
"ccl:lib;macros.lisp"); it goes to great and somewhat ugly lengths to
ensure that read-time side-effects take place:

(defmacro make-record (record-name &rest initforms)
   (let* ((ftype (%foreign-type-or-record record-name))
          (bits (ensure-foreign-type-bits ftype))
 	 (bytes (if bits
 		  (ceiling bits 8)
 		  (error "Unknown size for foreign type ~S."
 			 (unparse-foreign-type ftype))))
 	 (p (gensym))
 	 (bzero (read-from-string "#_bzero")))
     `(let* ((,p (malloc ,bytes)))
       (,bzero ,p ,bytes)
       ,@(%foreign-record-field-forms p ftype record-name initforms)
       ,p)))

An alternative to that sort of READ-FROM-STRING nonsense is to avoid
the use of #_  in macro expansions, e.g.:


(defmacro allocate-initialized-pointer (n ch)
   "Allocate foreign memory, fill it with the specified character value"
   (let* ((p (gensym)))
     `(let* ((,p (external-call "_malloc" :unsigned-fullword ,n :address)))
        (unless (%null-ptr-p ,p)
          (external-call "_memset" :address ,p
                                   :unsigned-byte (char-code ,ch)
                                   :unsigned-fullword ,n
                                   :void)
          ,p))))


On Fri, 13 May 2005, Chris Curtis wrote:

> I'm having some dfsl weirdness with foreign functions.
>
> I have one asdf system with code that calls #___CFStringMakeConstantString. 
> That system is then loaded by another asdf system (via :depends-on). If I 
> clean out all the dfsls in both systems and build, everything works great. If 
> I build again, I get compiler warnings:
>
> ;   Undefined function OS::|__CFStringMakeConstantString| (2 references)
>
> and, unsurprisingly, at runtime I get:
>
>    Error: Undefined function OS::|__CFStringMakeConstantString| called with 
> arguments (#<A Mac Pointer #xCAF40>) .
>
> This can be easily corrected by deleting *just* the dfsl that references 
> #___CFStringMakeConstantString (or any dfsl that it depends on and thus 
> triggers a recompile) -- voilà, it works again.
>
> I suspect that this might have something to do with the fact that 
> __CFStringMakeConstantString is not officially a published API, since it's 
> the only function that gets this error. I'm just not knowledgeable enough 
> about the guts of the #_ interface to even know where to start.
>
> Any ideas?
>
> --chris
>
>
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>