<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>thank you for the code and explanations. Some followup questions:</div><div><br></div><div>- how does the .cdb file get created. I assume that is a CCL thing and that the .cdb file is the result of parsing the header file contained in the framework? </div><div><br></div><div>- how hard would it be to parse and include the parameter names? Could the header file parser be extended to build a .cdb file that would include parameter name info? Just seems to be a real shame to skip that kind of info given that it is in the header files.</div><div><br></div><div>- what about constants? The .cdb files does not appear to include "CL_..." constants such as GL_COLOR_BUFFER_BIT, yet CCL clearly has access to this info, e.g., #$GL_COLOR_BUFFER_BIT expands into DARWIN32::GL_COLOR_BUFFER_BIT of value 16384.</div><div><br></div><div>- type names: Can one use the type names returned by the gl-function-info, e.g., :single-float or does one have to use the :<gl>float? </div><div><br></div><div><br></div><div><div>(defun gl-vertex3f (arg-0 arg-1 arg-2)</div><div> (ccl:external-call "_glVertex3f" ; the CCL::EFD-ENTRY-NAME</div><div> :single-float arg-0</div><div> :<gl>float arg-1</div><div> :<gl>float arg-2</div><div> :void))</div></div><div><br></div><div><br></div><div>alex</div><div><br></div><div><br></div><br><div><div>On Jan 8, 2009, at 4:59 PM, Gary Byers wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>#_ basically hashes the function name which follows and tries to find<br>an entry for that name in some "functions.cdb" file (where an "entry"<br>is a sort of byte-coded representation of the function's argument and<br>return types.) #_ parses the encoded argument into a little lisp<br>structure and adds that structure as the value of the key<br>OS::|caseSensitiveName| in a hash table. The final read-time actions<br>are to define OS::|caseSensitiveName| as a macro (all of the things<br>defined via #_ share the same macro definition, named something like<br>CCL::%EXTERNAL-CALL-EXPANDER) and return OS::|caseSensitiveName|.<br><br>So, reading:<br><br>(#_read fd buf n)<br><br>tries to ensure that some foreign function definition info is associated<br>with OS::|read| and that OS::|read| is defined as the magic macro (when<br>processing the #_) and the reader returns:<br><br>(os::read fd buf n)<br><br>When macroexpanded, that turns into something like:<br><br>(external-call "read" :int fd :address buf :some-kind-of-int n :some-kind-of-int)<br><br>which eventually macroexpands down to %FF-CALL.<br><br>What you presumably want to do is something similar. only avoiding<br>the use of #_ and defining (possibly inlined) functions or macros<br>with lispier names (gl-color-3f rather than |glColor3f|, etc.)<br><br>It's possible to enumerate all of the keys in a .cdb file - I think that<br>the function/macro that does this is called CCL::CDB-ENUMERATE-KEYS, of all things; this returns a list of all strings used as keys in the .cdb<br>file (in random order). If you do:<br><br><br>(ccl::cdb-enumerate-keys (ccl::db-functions (use-interface-dir :gl)))<br><br>you'll get a list of > 7000 strings; that includes everything that's<br>defined in the :gl functions.db file.<br><br>Unfortunately, that list will include the names of things that are<br>referenced in some GL-related header file (including a lot of Carbon<br>and C library stuff defined in agl.h). If a heuristic is that all<br>real GL function names start with the string "gl" (and few non-GL<br>function names do), you can pass a second "filter/predicate" argument<br>to CDB-ENUMERATE-KEYS that decides whether or not to include a given<br>string in the list. (This may or may not be easier than generating<br>the full list and post-processing it.)<br><br>However we do it, if we restrict the set of defined keys to names<br>that start with "gl", we get a list with ~1136 entries.<br><br>The enclosed function OBTAIN-GL-FUNCTION-INFO will return a hash<br>table whose keys are GL function names (mixed-case strings) and<br>whose values are structures (CCL::EXTERNAL-FUNCTION-DEFINITIONs);<br>accessors with the prefix (defstruct :conc-name) CCL::EFD- can<br>be used to access the fields of an EXTERNAL-FUNCTION-DEFINITION<br>structure. If we look at one of these (chosen at random based<br>on my fading memory of OpenGL):<br><br>? (defvar *gl-info* (obtain-gl-function-info))<br><br>? (describe (gethash "glColor3f" *gl-info*))<br>#S(CCL::EXTERNAL-FUNCTION-DEFINITION :ENTRY-NAME "glColor3f"<br> :ARG-SPECS (:<GL>FLOAT<br> :<GL>FLOAT<br> :<GL>FLOAT)<br> :RESULT-SPEC :VOID :MIN-ARGS 3)<br>Type: CCL::EXTERNAL-FUNCTION-DEFINITION<br>Class: #<STRUCTURE-CLASS CCL::EXTERNAL-FUNCTION-DEFINITION><br>ENTRY-NAME: "glColor3f"<br>ARG-SPECS: (:<GL>FLOAT :<GL>FLOAT :<GL>FLOAT)<br>RESULT-SPEC: :VOID<br>MIN-ARGS: 3<br><br>we see that there isn't a whole lot there. If the MIN-ARGS field has<br>any useful purpose anymore, I don't remember what that purpose would<br>be; assuming that we can think of a reasonable way to map the foreign<br>name ("glColor3f") to a lisp symbol (maybe something like OPENGL:GL-COLOR3F,<br>maybe something better ...) it's not too hard to imagine tha we could<br>automatically generate:<br><br>(declaim (inline opengl::gl-color3f))<br>(defun opengl::gl-color3f (arg-0 arg-1 arg-2)<br> (ccl:external-call "glColor3f" ; the CCL::EFD-ENTRY-NAME<br> :<gl>float arg-0<br> :<gl>float arg-1<br> :<gl>float arg-2<br> :void))<br><br>A few things worth noting:<br><br>1) Any information about parameter names that might have existed in the .h<br>file in the declaration of glColor3f is long gone:<br><br><br>extern void glColor3f (GLfloat red, GLfloat green, GLfloat blue);<br><br>It might be nice to be able to generate more meaningful names.<br><br>2) Argument and result type specifiers might be primitive, built-in<br>things (like :int and :void) or might be things that're themselves<br>defined in the .cdb files. You could:<br><br> a) yank those things in and predefine them (by enumerating the<br>keys in the "types.cdb" file for :gl)<br><br> b) let CCL:EXTERNAL-CALL look them up at macroexpand time<br><br>(a) can be machine-dependent. (I'm not sure how true this is<br>for OpenGL types, but it can generally be true that the definition<br>of something like a 64-bit signed integer can be sensitive to word-size<br>issues, structure types and bitfields and other things can have<br>machine-dependent alignment and packing rules and may be sensitive<br>to endianness, etc.)<br><br>(b) should do the right thing, but depends on details of CCL's FFI<br>to do so.<br><br>3) I'm not sure how often this comes up in OpenGL, but if the last<br>arg-spec in a function definition is the symbol :VOID (which is<br>otherwise meaningless as an argument type), that means "the function<br>accepts an indefinite number of extra arguments." (Using the "void"<br>type as a sort of &rest marker is actually how GCC represents this<br>internally, and that gets passed through the ffi translation process.)<br>I checked, and this doesn't seem to happen in OpenGL. Never mind.<br><br>4) A lot of OpenGL functions take arguments and return results<br>by value (as glColor3f does), so the idea of treating them as<br>simple lisp functions makes sense. Other cases may be harder<br>to map directly: glColor3fv takes a "vector" (C pointer) that's<br>presumed to point to 3 :<GL>floats. If you want to support<br>that, you may have to decide whether to expose the FFI (e.g.,<br>make the caller stack-cons a "vector of 3 floats" and pass the raw pointer in, or do it for them (e.g., have the function<br>accept a lisp vector and copy its contents to a foreign vector<br>that the function allocates itself.) It's probably hard to<br>automate the translation process, and it might be wise to<br>just do the low-level thing and discourage students from using<br>the "v" functions, at least initialy.<br><br>5) Lastly: I'm sure that this (generating lisp FFI bindings for<br>OpenGL) has been done before (probably many times) and there may be<br>things out there that are accurate, complete, and relatively portable<br>(using CFFI or UFFI or some other FFI abstraction layer.) We did<br>a fairly large project with a fairly large OpenGL component a<br>few years ago (everyone, rush out and buy a copy of InspireData<br>now), and at that time we had to manually define interfaces for<br>OpenGL things that weren't present in the vendor's OpenGL interface.<br>That was 4-5 years ago, and the situation may have improved a lot<br>since then; if someone's already put a lot of thought into how<br>to solve some of these issues - like naming conventions - it might<br>be wise to look on common-lisp.net to see what approaches others<br>have taken.<br><br><br><br>On Thu, 8 Jan 2009, Alexander Repenning wrote:<br><br><blockquote type="cite">I promise some technical questions, mostly aimed at Gary I assume, but first a motivational, short rant. Unlike some of the voices on comp.lang.lisp I do not think that CL is either doomed or is about to take off like crazy by adjusting itself (e.g., its syntax). Education is key. Get the CS undergraduates excited about CL somehow and CL could actually gain new ground instead of just holding on to its legacy developers. But how? Intro to CS courses, e.g., Scheme based on Abelson are probably doomed given that even Abelson himself is no longer using Lisp at MIT for that course. Instead, I claim, it makes more sense to have a compelling package fitting a course such a graphics course where students typically follow the OpenGL red book and do weekly programming assignments. The point of these courses it to learn about graphics and to have a nice tool. Running your event, animation loop and change your code on the fly -WOW. Yes, Lisp can do this. I have been teaching these kinds of course and I tell you a good percentage of students started to really like to program Lisp because of this experience. At the time the MCL license and sales "strategy" became a show stopper. CCL could become this kind of tool. If it can be delivered in a way that I, as prof, have my students download Mac or Windows versions and they can open up OpenGL windows and start typing in OpenGL code right away. Then, when the projects become a bit more complex CL even delivers good speed in contrast to most scripting languages that typically used to make OpenGL programming simple.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Technical challenges:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">In general I am not against the MCL/CCL #_<function_name> syntax to invoke system calls. For instance, with OpenGL, to make the obligatory RGB triangle your code would like this:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(defmethod DRAW ((Self opengl-view))<br></blockquote><blockquote type="cite">(#_glBegin #$GL_TRIANGLES)<br></blockquote><blockquote type="cite"> (#_glColor3f 1.0 0.0 0.0)<br></blockquote><blockquote type="cite"> (#_glVertex3f 0.0 0.6 0.0)<br></blockquote><blockquote type="cite"> (#_glColor3f 0.0 1.0 0.0)<br></blockquote><blockquote type="cite"> (#_glVertex3f -0.2 -0.3 0.0)<br></blockquote><blockquote type="cite"> (#_glColor3f 0.0 0.0 1.0)<br></blockquote><blockquote type="cite"> (#_glVertex3f 0.2 -0.3 0.0)<br></blockquote><blockquote type="cite">(#_glEnd))<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">but as it turns out most people prefer:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(defmethod DRAW ((Self opengl-view))<br></blockquote><blockquote type="cite">(glBegin GL_TRIANGLES)<br></blockquote><blockquote type="cite"> (glColor3f 1.0 0.0 0.0)<br></blockquote><blockquote type="cite"> (glVertex3f 0.0 0.6 0.0)<br></blockquote><blockquote type="cite"> (glColor3f 0.0 1.0 0.0)<br></blockquote><blockquote type="cite"> (glVertex3f -0.2 -0.3 0.0)<br></blockquote><blockquote type="cite"> (glColor3f 0.0 0.0 1.0)<br></blockquote><blockquote type="cite"> (glVertex3f 0.2 -0.3 0.0)<br></blockquote><blockquote type="cite">(glEnd))<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Now one can argue why, at least in the case of OpenGL, this is preferable. Here are some reasons:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">- the #_ function call mechanism establishes binding at read time via a DB. This is good for the program, perhaps, but not good for the programmer. Before Lisp reads a correct opengl function call Lisp does not have an interned symbol relating to this function name. As a consequence symbol completion cannot work. In an API with close to 1000 functions/constants/ this makes a huge difference.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">- the #_ syntax may not be cross platform or cross Lisp<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">- most OpenGL apps have concentrated OpenGL call where virtually ALL the function calls are GL function calls. Ergo you get just about each line of line of code containing some #_ which after a while starts to look awkward and redundant.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">How could we address this? Here are some bad ideas:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">1) define a function for each foreign function, e.g.,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(defun glColor3f (R G B)<br></blockquote><blockquote type="cite">(#_glColor3f R G B))<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">- not good: function call overhead<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">2) define a macro for each foreign function, e.g.,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(defmacro glColor3f (R B G)<br></blockquote><blockquote type="cite"><call read-dispatch function via *readtable* to expand into same thing as (#_ glColor3f ...) ><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-> (%FF-CALL (%REFERENCE-EXTERNAL-ENTRY-POINT<br></blockquote><blockquote type="cite"> (LOAD-TIME-VALUE (EXTERNAL "_glColor3f")))<br></blockquote><blockquote type="cite"> :SINGLE-FLOAT R<br></blockquote><blockquote type="cite"> :SINGLE-FLOAT G<br></blockquote><blockquote type="cite"> :SINGLE-FLOAT B<br></blockquote><blockquote type="cite"> :VOID)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">- as fast as #_ glColor3f ...)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The biggest issue with both approaches is that I do not know the name of the function and its parameters to create a function or a macro.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">In MCL and Allegro we used these LONG lists:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">....<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">(foreign-function glvertex3f (float FLOAT FLOAT) VOID "glVertex3f")<br></blockquote><blockquote type="cite">(foreign-function glvertex3i (INT INT INT) VOID "glVertex3i")<br></blockquote><blockquote type="cite">(foreign-function glvertex3s (SHORT SHORT SHORT) VOID "glVertex3s")<br></blockquote><blockquote type="cite">(foreign-function glvertex4d (DOUBLE DOUBLE DOUBLE DOUBLE) VOID "glVertex4d")<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">...<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">to create bindings to OS X AGL and Windows WGL<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Manually maintaining these kinds of lists is tedious and error prone.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">What can we do? Could we have a function to parse the "/System/Library/Frameworks/OpenGL.framework/OpenGL" framework to generate foreign interfaces. CCL must have some code to generate access framework entries and generate interfaces.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Alex<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Prof. Alexander Repenning<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">University of Colorado<br></blockquote><blockquote type="cite">Computer Science Department<br></blockquote><blockquote type="cite">Boulder, CO 80309-430<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">vCard: <a href="http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf">http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"></blockquote><span><gl-function-info.lisp></span></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Prof. Alexander Repenning</font></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><br class="khtml-block-placeholder"></p><p style="margin: 0.0px 0.0px 0.0px 0.0px">University of Colorado</p><p style="margin: 0.0px 0.0px 0.0px 0.0px">Computer Science Department</p><p style="margin: 0.0px 0.0px 0.0px 0.0px">Boulder, CO 80309-430</p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><br class="khtml-block-placeholder"></p><p style="margin: 0.0px 0.0px 0.0px 0.0px"><font face="Helvetica" size="3" style="font: 12.0px Helvetica">vCard: <a href="http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf">http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf</a></font></p><br class="Apple-interchange-newline"></span></span></span></div></span> </div><br></body></html>