[Openmcl-devel] FFI question: passing aligned pointers

[Artem Mironov]

Hello Mattew,

On Mon, Dec 6, 2010 at 10:23 PM, R. Matthew Emerson <rme at clozure.com> wrote:
>
> On Dec 3, 2010, at 5:14 AM, Artem Mironov wrote:
>
>> I'm new to CCL and to Lisp in general. I have a question about passing
>> C pointers to/from C functions using (external-call ...).
>>
>> A C pointers returned by system's malloc() are aligned by at least 8 bytes.
>> Such pointers can be safely passed to/from lisp as fixnums, without boxing
>> into macptr.
>>
>> I'm looking for something like this:
>>
>> (external-call "c_function" :fixnum 0 :int 7 :fixnum)
>>
>> Is there such feature or where to look to try to implement it?
>
> Are you worried about consing a lot of macptrs?  In many cases, macptrs can be stack-allocated.
>
> http://ccl.clozure.com/ccl-documentation.html#Referencing-and-Using-Foreign-Memory-Addresses
>

I use code like following

(defun ioq-new (flags) (external-call "ioqnew" :int flags :address)
(defun ioq-free (queue) (external-call "ioqfree :address queue :void)

(defmacro with-ioq ((var) &body body)
   `(with-macptrs ((,var (ioq-new 0)))
         (unwind-protect (progn , at body)
             (ioq-free ,var))))

(defun scan-blocks (.....)
    (with-ioq (ioq)
       (loop ....)))

(disassemble) shows:
.......

  [133]   (movq (@ '#<EXTERNAL-ENTRY-POINT "ioqnew"
(#x00000008018C1E50) ./XXXXX-amd64.so #x30200097743D> (% fn)) (%
arg_y))
  [140]   (movq (@ 3 (% arg_y)) (% arg_z))
  [144]   (cmpb ($ 11) (% arg_z.b))
  [148]   (je L993)
  [154]   (xorl (% imm0.l) (% imm0.l))
  [156]   (nop)
  [158]   (callq (@ .SPFFCALL))
  [165]   (leaq (@ (:^ L0) (% rip)) (% fn))
  [172]   (movd (% imm0) (% mm0))
  [176]   (movl ($ #x31A) (% imm0.l))
  [181]   (movl ($ 19) (% imm1.l))
  [186]   (subq (% imm1) (@ (% gs) #xD8))
  [195]   (movq (@ (% gs) #xD8) (% temp0))
  [204]   (cmpq (@ (% gs) #xE0) (% temp0))
  [213]   (ja L217)
  [215]   (uuo-alloc)
L217
  [217]   (movq (% imm0) (@ -13 (% temp0)))
  [221]   (andb ($ #xF0) (@ (% gs) #xD8))


Return value of ioqnew is boxed into heap allocated macptr.

My application performs huge amount of IO (tens of Mb per second).
This particular case isn't problem, but inside (loop) of example above,
a lot more functions with take/returns :address are called.

All pointers used in this example are page aligned.

The other issue that my application passed a lot of (signed 64) values.
This file offsets all of them are 4096 bytes aligned too.

Also, as I mentioned above, majority of foreign pointers (ones returned by
system's malloc()) are at least 8 byte aligned and some implementations
(like FreeBSD) are 16 byte aligned.

So passing around such pointers are fixnums seems to me quite natural.

I'm slowly getting around CCL sources and gathering information in order
to implement such kind of external-call.

But %ff-call is very integrated into CCL system. It's not clear where to add
new code.

I'm appreciate any pointers which help to either extend existing system or
to add something simple and specialized like

(fixnum-external-call name &rest args)

which pass one or more fixnums to an external function and returns a
fixnum.


-- 
Best Regards
Artem