[Openmcl-devel] Shark

[Gary Byers]

On Sat, 9 Apr 2005, Dan Knapp wrote:

>    I've spent a little time messing with Shark, using a modified version
> of the code
> Hamilton posted in October.  I've attached my version of his file, in
> case it's
> useful to anybody; it just makes a couple trivial changes to output all
> the
> symbols in a specified package, instead of only one symbol at a time.
>
>    I am seeing weird behavior when I try to actually use this code.
> Shark fails to
> identify most of the symbols; well, that's not necessarily a bug.  But
> it claims that
> most of the time is spent in functions which I know weren't being
> called, since
> they are in a totally different program module.  It also claims that
> functions which
> I know did run for at least several seconds weren't called at all.  In
> other words,
> somehow the symbol addresses Shark gets are wrong.  How can that be?  I
> thought the call to (ccl::purify) moved function definitions into a
> read-only area,
> where they wouldn't be moved again...  Are functions being aggressively
> inlined?
>


Here's a slightly modified version of your PRINT-SHARK-SPATCH-RECORD
function.  (It could possibly stand a bit more tweaking.)  It basically
differs from the code that you sent out in how STARTADDR and ENDADDR
are calculated.


(defun print-shark-spatch-record (symbol &optional (stream t))
  (when (fboundp symbol)
    (let ((cl (find-package "COMMON-LISP-USER"))
          (package (symbol-package symbol))
          (name (symbol-name symbol))
          (fn (or (macro-function symbol)
                  (fdefinition symbol))))
      (when fn
        ;; We should probably make sure that the code-vector
        ;; has a static address (e.g., that it wasn't loaded
        ;; or compiled - somehow - after CCL::PURIFY was last
	;; called.)
        (let* ((code-vector (uvref fn 0))
               (startaddr (+ (ccl::%address-of code-vector)
                             target::misc-data-offset))
               (endaddr (+ startaddr (* 4 (uvsize code-vector)))))
          ;; i hope all lisp sym characters are allowed... we'll see
          (format stream "{~@
                          ~@[~a~]~@[~[_~;__~]~]~a~@
                          0x~8,'0x~@
                          0x~8,'0x~@
                          }~%"
    [...]

I think that the version you mailed out calculated ENDADDR by adding
a constant to STARTADDR.  That isn't right, but I'm not sure that
that would have led to all of the anomalies you saw.  (It -might-
have caused some address ranges to overlap, and that -might- have
confused Shark.)

With this change, I created an .spatch file and ran a profiling
session with Shark 4.0.0 (v419).  I had no luck whatsover getting
it to identify lisp function names.

I updated CHUD and tried again with Shark 4.0.2 (v433, and it worked
fine.  I'm not sure what to conclude from this, aside from the fact
that Shark's .spatch support has sometimes been fragile.

[This all suggests that -some- support for creating .spatch files
and working with Shark should go into a near-future release.]



>    In an old post, Gary, you mentioned that there was a way to get
> OpenMCL to
> cough up what function a given address is in.  That might be a bit
> helpful in
> diagnosing this problem.  How does that work?

Slowly (but waiting to get an answer might be better than not getting
an answer.)

The general idea is to use a little piece of LAP code to identify
the CCL::CODE-VECTOR that contains a given address (you can probably
do this in very low-level Lisp code, which might or might not be
any easier to read), and then walking memory, looking for the
function that that code-vector belongs to.

I remember writing that code, but can't find it at the moment.

>
>    For now, I'm just going to do my profiling the old-fashioned way,
> putting timers
> around each of the functions I suspect of being problematic.  So I
> don't really
> need to solve these Shark issues for myself, right now, but it would be
> great to
> have a general framework for others to use.

Agreed.  It'd be good to have something that worked with Shark (there
are ways of controlling Shark remotely, e.g.:

   (unwind-protect
    (progn
     (start-shark-profiling-session)
     (long-computation))
    (stop-shark-profiling-session))

which might be a little better than trying to press Control-Escape quickly
enough), and there are also ways of accessing the kernel profiling
facility directly.


>
> -- Dan Knapp
>
>