[Openmcl-devel] make-heap-buffer question

[Cyrus Harmon]

On Sep 19, 2004, at 9:31 PM, Gary Byers wrote:

>> So I'm not sure exactly how to use the make-heap-buffer,
>> make-heap-ivector stuff, or why it's in streams, for that matter, but
>> this seems a bit odd:
>>
>
> Those functions are in l1-streams.lisp because the stream code is the
> only code that uses them.  They aren't particularly useful or general.

Hmm... a malloc'ed buffer that I can call aref, %get-xxx, and pass to 
foreign code sounds pretty useful to me! The general part I'm willing 
to buy, but that sounds like it might be temporary condition. On an 
aside, part of the usual lisp dogma is Greenspun's tenth law; lately I 
feel like the more in-depth I dig into the OpenMCL internals, the 
closer I get to writing C code in LISP syntax...

>> (multiple-value-bind (buf p)
>>      (ccl::make-heap-buffer '(unsigned-byte 32) 256)
>>    (print (array-dimensions buf))
>>    (setf (aref buf 0) #x7f7f7f7f)
>>    (setf (aref buf 1) #xaabbccdd)
>>    (setf (aref buf 1023) #xdeadbeef)
>>    (let ((*print-base* 16))
>>      (print (aref buf 0))
>>      (print (aref buf 1))
>>      (print (aref buf 1023)))
>>    (print p))
>>
>> gives the following results:
>>
>> (1024)
>> 7F7F7F7F
>> AABBCCDD
>> DEADBEEF
>> #<A Mac Pointer #x10A2CC>
>>
>>
>> I can understand buf beingg 1024 _bytes_ (in the normal 8-bit sense)
>> long, but it looks to me like it's 1024 unsigned-byte 32's long. Is
>> this a bug or am I missing something?
>
> Sometimes, to avoid confusion (or possibly to introduce it) the term
> "octet" is used to mean "8-bit byte".
>
> CCL::MAKE-HEAP-BUFFER accepts an array-element-type specifier and
> an element-count as arguments; it returns 4 values
>
> 1) a vector of the indicated type, with the indicated number of 
> elements
> 2) a pointer, which happens to share storage with the vector
> 3) the number of usable octets in the buffer (the product of the number
>    of elements in the vector and the size of each element in octets,
>    at least in simple cases.  Bitvectors and some floating-point types
>    may not be simple.)
> 4) the internal typecode of the vector.

Yeah, but the vector seems to have 1024 32-bit elements. That seems 
odd. I can understand 1024 bytes (256 * 4) but 4096 bytes strikes me as 
add, since the same call unsigned-byte 8 just returns 256 bytes. But, 
like you said, this isn't exactly for external consumption.

> For a number of reasons, I'd discourage use of CCL::MAKE-HEAP-BUFFER;
> it only tries to handle a few element-type specifiers (not 
> coincidentally,
> the ones that the stream-buffer-creating code might call it with), and
> doesn't even do that in a very reasonable way.

Got it.

> CCL::%MAKE-HEAP-IVECTOR is even lower-level and less general.
>
> I mailed out yet another variant a month or two ago; this version
> tries to be a bit more reasonable, and actually does a little bit
> of error/sanity-checking:
>
> (defun make-heap-ivector (element-count element-type)
>    (let* ((subtag (ccl::element-type-subtype element-type)))
>      (unless (= (logand subtag target::fulltagmask)
>                 target::fulltag-immheader)
>        (error "~s is not an ivector subtype." element-type))
>      (let* ((size-in-bytes (ccl::subtag-bytes subtag element-count)))
>        (ccl::%make-heap-ivector subtag size-in-bytes element-count))))

It would be nice to stick this in the core somewhere.

> Note that (gratuitously) this function takes the same arguments as
> CCL::MAKE-HEAP-BUFFER, but takes them in the opposite order.
>
> I'd be willing to export MAKE-HEAP-IVECTOR, document it, commit to not
> changing it without advance notice, address any remaining bugs,
> etc. (The same would be true for any complementary functions that
> would free the storage involved, etc.)  I'm less inclined to commit to
> making CCL::MAKE-HEAP-BUFFER or CCL::%MAKE-HEAP-IVECTOR behave like
> they were anything other than murky components of the implementation
> layer, that might change or disappear for murky, implentation-dependent
> reasons.

Sounds like make-heap-ivector (not %make-heap-ivector) is the way to 
go. What I think I want (and I'm willing to be dissuaded here) is 
blocks of data that I can access both from native code and from LISP. I 
can accept the fact that this will rely on implementation-depended 
pieces. It would be nice if those were as unmurky as possible. It seems 
to me that being able to pass data in these blocks off to code that 
only knows about vectors and can just AREF instead of %GET-XXX to 
access these blocks is pretty useful. I'm trying to do this for I/O to 
the vImage framework, but this would also be useful for avoiding 
round-trips to things like BLAS and LAPACK, I would think.

Thanks,

Cyrus