[Openmcl-devel] Weird code

[Gary Byers]

The #_ does something like the following (with some
details glossed over):

1) reads a token from the stream, with case preservation.
2) interns that token in the "OS" package (where "OS" is
    a nickname for some platform-dependent package.)  This
    step yields a symbol, OS::|foo| for example.
3) looks up that symbol's name in the interface database.
    If it finds a function definition for that name, installs
    information about that foreign function's argument and
    result types in some FFI data structures; if no definition
    was found, signal an error.
4) Set the global macro definition of the OS-package symbol
    (OS::|foo|) to an expansion function that uses the info
    from step (3) to generate an FF-CALL form where the argument/
    result type information is explicit.

So:

(#_malloc 10)

- reads and interns OS::|malloc| (steps 1 & 2)
- looks up "malloc" in the interface database (3) and caches some info
   about that function's arg and result types.
- defines OS::|malloc| to be a macro

and (#_malloc 10) is read as (os::|malloc| 10), e.g., an invocation of
the macro OS::|malloc|.

If that macro call is expanded in the same session (as ordinarily happens;
the form will be compiled soon after it's read in most cases), the macro
expands into some variant of:

(external-call "malloc" :int 10 :address)

That all works if the steps performed by the reader macro have been performed
in the current lisp session.

If we try to write a macro that uses a reader-macro in its expansion - something
like:

(defmacro allocate (size)
  `(#_malloc ,size))

That'll likely work as expected if we expand that macro in the same session
in which #_malloc was read and the read-time side-effects occurred.

If we compile that macro definition and later load it into a session in which
those read-time side-effects haven't occurred, then we'll likely get an error
to the effect that OS::|malloc| isn't defined.

The weird code that was doing READ-FROM-STRING at macroexpand time was trying
to force those side-effects to occur (at macroexpand time).  That makes it possible
to use #_ in the macroexpansion, though as Gail noted it can only work if it's
sure of what readtable's in effect during the READ-FROM-STRING.

(The macro that Ron found probably would be simpler if it expanded
into something like

(external-call "memset" ...)

instead of doing what it's doing.  I wouldn't be shocked if the READ-FROM-STRING
idiom was used in a few other places (or if those other places also neglected
to ensure that the expected readtable was in effect.)

People who used MCL may remember something called REQUIRE-TRAP; it existed for
the reasons outlined above and may have been implemented in terms of READ-FROM-STRING.


On Sat, 11 Sep 2010, james anderson wrote:

>
> On 2010-09-10, at 22:25 , Gail Zacharias wrote:
>
>> On Fri, Sep 10, 2010 at 2:27 PM, Ron Garret <ron at flownet.com> wrote:
>>> I happened upon the following odd behavior while mucking with some reader 
>>> macros:
>>> 
>>> ? (set-macro-character #\m
>>> (lambda (stream c)
>>>   (unread-char c stream)
>>>   (let ((*readtable* (copy-readtable  nil)))
>>>     (read stream))))
>>> T
>>> ? 'moo
>>> MOO
>>> ? (defun foo (a b) (#/sizeWithAttributes: a b))
>>>> Error: Foreign function not found: X86-DARWIN64::MEMSET
>>>> While executing: CCL::LOAD-EXTERNAL-FUNCTION, in process Listener(6).
>>>> Type cmd-. to abort, cmd-\ for a list of available restarts.
>>>> Type :? for other options.
>>> 1 >
>>> ? (defun foo (a b) (#/sizeWithAttributes: a b))
>>> ;Compiler warnings :
>>> ;   In FOO: In the call to NEXTSTEP-FUNCTIONS:|sizeWithAttributes:| with 
>>> arguments (A B),
>>> ;     2 arguments were provided, but at most 0 are accepted
>>> ;     by the current global definition of 
>>> NEXTSTEP-FUNCTIONS:|sizeWithAttributes:|
>>> FOO
>>> ?
>>> 
>>> 
>>> I tracked the problem down to the following code in lib/macros.lisp:
>>> [...]
>>> and in particular, to this line:
>>>
>>>        (memset (read-from-string "#_memset")))
>>> 
>>> Is there a reason why read-from-string being invoked here instead of just 
>>> doing, say, (let ((memset '#_memset)) ... ?  Or dispensing with the 
>>> binding entirely and just doing (,#_memset ...) in the body of the LET?
>> 
>> It's so that the side-effects happen at runtime rather than compile
>> time.  But it's a bug that it's just using whatever readtable happens
>> to be in force at the time.
>
> but doesn't that bit of code already know what symbol it wants to resolve?
> in which case, the question is, what is the intended benefit running it 
> through the reader rather than calling load-external-function directly?
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>