[Openmcl-devel] Documentation and Arglist

[Sven Van Caekenberghe]

On Tuesday, December 17, 2002, at 11:06 PM, Gary Byers wrote:

>
>
> On Tue, 17 Dec 2002, Sven Van Caekenberghe wrote:
>
>> On Monday, December 16, 2002, at 11:23 PM, Gary Byers wrote:
>>
>>> Basically, you set CCL:*FASL-SAVE-LOCAL-SYMBOLS* (and perhaps
>>> CCL:*SAVE-LOCAL-SYMBOLS*) to T (likewise for the doc string 
>>> variables,
>>> I -think-), then do:
>>>
>>> ? (compile-ccl t)
>>>
>>> and
>>>
>>> ? (xload-level-0)
>>>
>>
>> I can't make this work:
>
>
> Sorry.  Because of the way in which the bootstrapping image is
> built, the debugging information not only doesn't work, it seems to
> confuse that process.
>
> So (I actually tried this this time ...):
>
> 1) With debugging switches turned off (e.g., as OpenMCL is
> distributed) do:
>
> ? (xload-level-0 :force)
>
> That'll force recompilation of the source files in 
> "ccl:level-0;**;*.lisp"
> and "load" them into a new "ccl:ppc-boot[.image]" file.
>
> 2) Enable the debugging switches as you tried:
>
>
>> ? (setf  *save-doc-strings* t
>>         *save-local-symbols* t
>>         *save-definitions* t
>>         *fasl-save-doc-strings* t
>>         *fasl-save-local-symbols* t
>>         *fasl-save-definitions* t)
>> T
>
> and recompile everything -but- what's in the level-0 directory
>> ? (compile-ccl t)
>> ;Compiling "/Users/sven/apps/ccl/compiler/nxenv.lisp"...
>  ...
>> ;Compiling "/Users/sven/apps/ccl/lib/describe.lisp"...
>> ;Compiler warnings for "/Users/sven/apps/ccl/lib/describe.lisp" :
>> ;   Undefined function INSPECTOR::DISASSEMBLY-COMMENT-TYPE, in
>> INSPECTOR::DISASSEMBLY-LINE-N.
>> T
>> ?
>
> 3) Now, we have the level-0 sources compiled with debugging info
> disabled and everything else compiled with it enabled.  Build an
> image.
>
> % ./dppccl ppc-boot.image      # or % ./ppccl ppc-boot for Linux.
> ;Loading level-1.dfsl
> [...]
> ;Loading ./library/lispequ
> ?
>
> 4) At this point, the debugging flags are all off (unless you edited
> their definitions before doing the COMPILE-CCL above.)  You may want
> to turn them all back on.  (The variables were all T when you compiled
> the files that were just loaded; loading those files set them back
> to NIL, probably.)
>
> Then save an image.
>
> 5) Try it out:
> Welcome to OpenMCL Version (Beta: Darwin) 0.13.2!
> ? (arglist 'delete-if-not)
> (CCL::TEST SEQUENCE &KEY CCL::FROM-END CCL::START CCL::END COUNT 
> CCL::KEY)
> :DEFINITION
> ?
>
> (Well, ILISP makes it look nicer ...)
>
> Someone reported the problem with building level-0 with debugging info
> enabled a long time ago; it's probably not too hard to fix, but I 
> haven't
> really looked into it.
>
> In the meantime, the procedure outlined above -does- work to get 
> debugging
> info (such as it is) for everything else.
>
>

Thx, now it works!

But it only preserves arglists, not documentation strings (I don't 
think there are many documentation strings in the open mcl source 
code). I wrote some code to scavenge most doc strings of commercial mcl 
(or any other CL ;-) ) and store them inside openmcl:

;;;; -*- mode: lisp -*-

(in-package :common-lisp-user)

(defparameter *all-symbols*
   (let ((all-symbols '()))
     (do-symbols (x 'common-lisp)
       (push x all-symbols))
     (setf all-symbols (sort all-symbols #'string-lessp))))

(defun collect-info ()
   (mapcar #'collect-symbol-info *all-symbols*))

(defun collect-symbol-info (symbol)
   (let ((fdoc (documentation symbol 'function))
	(vdoc (documentation symbol 'variable))
	(mdoc (documentation symbol 'compiler-macro))
	(doc '()))
     (when (fboundp symbol)
       (push (cons :arglist (arglist symbol)) doc)
       (push (cons :fdoc fdoc) doc))
     (when (boundp symbol)
       (push (cons :vdoc vdoc) doc))
     (when (and (documentation symbol 'compiler-macro)
	       (string/= mdoc fdoc))
       (push (cons :mdoc mdoc) doc))
     (cons symbol doc)))

(defun collect-info-to-file (name)
   (with-open-file (stream name
			  :direction :output
			  :if-exists :supersede
			  :if-does-not-exist :create)
     (format stream ";; Documentation for all Common Lisp symbols~%")
     (write (collect-info) :stream stream :pretty t)))

(defparameter *symbol-documentation* nil)

(defun load-info-from-file (name)
   (with-open-file (stream name
			  :direction :input)
     (setf *symbol-documentation* (read stream))
     t))

(defun get-symbol-info (symbol)
   (assoc symbol *symbol-documentation*))

(defun install-symbol-documentation (info)
   (let ((symbol (car info))
	(docs (cdr info)))
     (when docs
       (dolist (doc docs)
	(case (car doc)
	  (:mdoc (setf (documentation symbol 'compiler-macro) (cdr doc)))
	  (:vdoc (setf (documentation symbol 'variable) (cdr doc)))
	  (:fdoc (setf (documentation symbol 'function) (cdr doc))))))))

(defun install-all-symbol-documentation ()
   (mapc #'install-symbol-documentation *symbol-documentation*)
   t)

(defun cleanup ()
   (setf *all-symbols* nil
	*symbol-documentation* nil))

;; eof

If you execute (collect-info-to-file "mcl-doc.lisp") in MCL, and later 
(load-info-from-file "mcl-doc.lisp") and then 
(install-all-symbol-documentation) in OpenMCL, most CL level 
documentation gets installed.

Technically, the above code is not perfect (not all documentation types 
are handled, and the distinction between functions and macros isn't 
done well), but it works great from ILISP.

The question is: can we do this legally ?

I think a procedure (by means of a script like .lisp file) that turns 
the standard image into a new development image with arglist and 
documentation info would be a great idea.

Alternatively, you could make the standard image contain all this info, 
and have a procedure to remove the info to save some space.

Sven

--
Sven Van Caekenberghe - mailto:sven at beta9.be
Beta Nine - software engineering - http://www.beta9.be
.Mac - svc at mac.com - http://homepage.mac.com/svc


_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel