[Openmcl-devel] "internal" functions and xref

[Gary Byers]

I think that the convention of using lists whose car is :INTERNAL was
added in 1.2; AFAIK, the trunk/1.3 and 1.2 behave the same way in this
regard.

In general, functions can have "names"; names are often symbols or
lists of symbols.  In CL, the third value returned by the function
FUNCTION-LAMBDA-EXPRESSION is called "name", and the CLHS dictionary
entry notes:

"The tertiary value, name, is the ``name'' of function. The name is
intended for debugging only and is not necessarily one that would be
valid for use as a name in defun or function, for example. By
convention, nil is used to mean that function has no name. Any
implementation may legitimately return nil as the name of any
function."

;;; A defining form like DEFUN ...
? (defun foo (x) x)
FOO
;;; ... tries to arrange that the function it creates will
;;; have a name that matches the symbol (or SETF name)
;;; being defined ...
? (nth-value 2 (function-lambda-expression #'foo))
FOO
;;; But the function's name is just an attribute of
;;; the function, and we can do things that might
;;; eventually lead to confusion.
? (setf (fdefinition 'bar) (fdefinition 'foo))
#<Compiled-function FOO #x3000410DCF5F>
? (nth-value 2 (function-lambda-expression #'bar))
FOO

Prior to 1.2 (or whenever the change was made), a function that wasn't
explicitly named was just given a function name of NIL:

Welcome to Clozure Common Lisp Version 1.1-r7641 (FreebsdX8664)!
? (defun adder (m)
     (lambda (n) (+ m n)))
ADDER
? (adder 3)
#<COMPILED-LEXICAL-CLOSURE #x300040D7236F>
? (nth-value 2 (function-lambda-expression *))
NIL

In 1.2 or later, an otherwise unnamed function that's defined within
a named function is given a name of the form (:INTERNAL <outer-function-name>):

Welcome to Clozure Common Lisp Version 1.3-dev-r11385M-trunk  (FreebsdX8664)!
? (defun adder (m)
   (lambda (n) (+ m n)))
ADDER
? (adder 3)
#<COMPILED-LEXICAL-CLOSURE (:INTERNAL ADDER) #x3000410D862F>
? (nth-value 2 (function-lambda-expression *))
(:INTERNAL ADDER)

If a function named (:INTERNAL ADDER) showed up in an error message
or backtrace (as it might if M or N weren't numbers), it might be
easier to figure out what's going on than it would be if the function
was unnamed.

This is probably at least a step in the right direction, though there
are probably still cases where there's amibiguity (what if there are
lots of unnamed functions defined inside a named function ?  I think
that some implementations might name them (:INTERNAL 0 FOO), (:INTERAL 1
FOO), ..., where there's some way of predicting which function is assigned
which ordinal), I don't think that there's any attempt to name otherwise
anonymous toplevel functions ("the function at position 12345 in foo.lisp",
"the thing that you typed into the listener 10 minutes ago" ...), and
there may be other cases where things -could- be given a meaningful
name but aren't.  It's probably fair to say the current implementation
does make debugging a bit easier in some cases, and it's also true that
more functions will have non-NULL names than in previous versions.

CCL::CALLERS walks memory, looking for reference a specified object
(often a symbol) and returning a list of such functions; it can be
used as a sort of coarse approximation to what a WHO-CALLS function
would return.  (If #'FOO references the symbol BAR as a constant,
we can't generally tell whether it does so because it calls the
global definition of BAR or if it does so for some other reason.)
Aside from all of the other issues related to walking memory, 
CCL::CALLERS has had to use heuristic means to decide whether
or not a function is "interesting": if #'FOO is redefined so that
it no longer references BAR but the old definition of #'FOO is
still somewhere in memory (because it hasn't been GCed yet), it's
not likely to be useful to report that the old definition of #'FOO
references BAR.  CCL::CALLERS used/uses heuristics (like "if a
candidate function is as a symbol as its name but that function
isn't the current global definition of that symbol, don't return
it") to decide whether or not the candidate function was likely
to be interesting.  I honestly don't know whether it now just
unconditionally includes a candidate function whose name is
a list that starts with :INTERNAL or whether it applies other
heuristics in that case, but in your example involving PRINT-OBJECT's
"callers", it now returns some named internal functions that were
previously rejected.

I don't know how (if at all) any of this affects XREF.  As I recall,
the XREF utility hooks itself into the compiler frontend and notes
references (function calls and other things) from the function being
compiled to other things.  References from/to things that had previously
been unnamed are now more likely to have names that start with :INTERNAL;
as note above, those :INTERNAL function names don't always uniquely
identify a function, but neither did function names of NIL ...  We
-might- be able to keep track of the fact that "some internal function
in the method-function for PRINT-OBJECT with specializers STANDARD-OBJECT
and T calls PRINT-OBJECT" where previously we might or might not have
been able ot say "some unnamed function calls PRINT-OBJECT ...".


> (while trying to fix slime's who-calls for ccl-1.3, I noticed)
>
> ccl::make-xref-entry returns bogus xrefs when it is given "internal"
> functions.
>
> Example:
>
>   $ ccl -n
>   Welcome to Clozure Common Lisp Version 1.3-dev-r  (LinuxX8664)!
>   ? (require 'xref)
>   XREF
>   ("XREF")
>   ? (ccl::callers 'print-object)
>   (#<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT
> (INSPECTOR::UNBOUND-MARKER T)>
>    #<CCL::STANDARD-KERNEL-METHOD CCL::REPORT-CONDITION (CONDITION T)>
>    CCL::WRITE-A-FROB #<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT
> (COMPLEX T)>
>    (:INTERNAL (PRINT-OBJECT (CLASS T)))
>    (:INTERNAL (PRINT-OBJECT (STANDARD-OBJECT T)))
>    (:INTERNAL CCL::WRITE-AN-ISTRUCT))
>   ? (car (last *))
>   (:INTERNAL CCL::WRITE-AN-ISTRUCT)
>   ? (ccl::make-xref-entry * :direct-calls)
>   #<XREF-ENTRY INTERNAL METHOD (WRITE-AN-ISTRUCT)>
>   ?
>
> A simple fix I came up with was to add a new CASE clause for :internal:
>
> (defun make-xref-entry (input relation)
>   (etypecase input
>     (symbol
>      ...)
>     (method
>      ...)
>     (cons
>      (case (car input)
>        ((ppc-lap-macro compiler-macro-function)
>         ...)
> +      ((:internal)
> +       (make-xref-entry (cadr input) relation))
>        (t
>         )))))
>
> But then I noticed that there are inputs for flet/labels local
> functions too. e.g.
>
>   ? (ccl::callers 'ccl::%multiply-and-add-loop)
>   ((:INTERNAL CCL::MULTIPLY-UNSIGNED-BIGNUMS CCL::MULTIPLY-BIGNUMS))
>
> So make-xref-entry now becomes:
>
> (defun make-xref-entry (input relation)
>   (etypecase input
>     (symbol
>      ...)
>     (method
>      ...)
>     (cons
>      (case (car input)
>        ((ppc-lap-macro compiler-macro-function)
>         ...)
> +      ((:internal)
> +       (if (= (length input) 3)
> +         (make-xref-entry (nth 2 input) relation)
> +         (make-xref-entry (cadr input) relation)))
>        (t
>         )))))
>
> Bah. I feel like I should ask what these "internal" functions are.
>
> regards,
> T.
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>