[Openmcl-devel] hemlock and tags files

[Hamilton Link]

The big difference is that tags files of course deal with all the files 
you run through it,
while the source-file, who-calls, etc. information in lisp only knows 
about what has been
loaded. There are good things about both, but normally I care about the 
definitions of things
that are loaded, and I can usually grep for what I want outside of 
that. Just my 2 cents.
h

On Oct 7, 2004, at 8:24 PM, Gary Byers wrote:

> The fact that OpenMCL already records the source file associated with
> most defining forms (and uses this information so that Meta-. works in
> ILisp/SLIME) strongly suggests one implementation strategy.
>
> (Yes, I realize that the information parsed into a tags file is
> potentially different from the information that the lisp already
> maintains.  No, I don't see a reason to maintain two sets of source
> file info.)
>
> Other people might prefer a tags file scheme, or having the choice
> somehow (I'm not sure how well the two schemes would coexist, but
> it might be possible to make them do so.)
>
>
> On Thu, 7 Oct 2004, alex crain wrote:
>
>> I'm getting ready to work on the source tagging features in hemlock. 
>> My
>> immediate goals are to
>> be able to click on a function and get either source code or
>> documentation for the function.
>>
>> Documentation is moving along nicely and now I'm thinking about source
>> code indexing.
>>
>> I want to model tagging after the emacs implementation, with the one
>> added feature that I want
>> to be able to do find-tag on some text and have hemlock produce
>> pointers to all the source
>> that defines that tag, maybe using a drawer or something to show the
>> options. I get annoyed when
>> I do find-tag and get referred to a piece of commented out code.
>
> There is (of course) a tradeoff or two here.  If you maintain exact
> source-file/byte position information for each definition, then yes,
> that's helpful and the editor should be able to find that definition
> with a high degree of reliability.
>
> As soon as the file has been edited with an external editor, that
> breaks down (at least a bit), and even if you only consider cases
> where the file's only edited in Hemlock, you have to keep those
> byte positions updated (perhaps by having translated them to marks
> a bit earlier.)  In some cases, this may degenerate into a heuristic
> search.
> v
> The other case - where you just try to associate the file with
> the definition, without trying to maintain accurate position 
> information -
> sort of uses heuristic search as a starting point.  It avoids the
> synchronization issues entirely, but how robust it is depends on how
> good the heuristics that guide the search are.  This approach works
> well some of the time and very poorly some of the time; the heuristic
> search sort of needs to be a full-blown lisp parser to do a good job
> of avoiding commented-out/conditionalized-out definitions.
>
> I think that people who've thought about this before noticed that
> a lisp-aware parser was already a part of a lisp-aware editor, and
> that it was possible to "sectionize" (I -think- that that was the
> term; it might have been "sectionalize") a buffer into regions or
> region-like things.  If a buffer contains:
>
> ;;; -*- Mode: Lisp ... *-*
>
> (defun fact (n)
>   (if (zerop n)
>     1
>     (* n (fact (1- n)))))
>
> ;; I used to be able to do "(defun fib (" with my eyes closed.
> (defun fib (m n)
>   ;; whatever the definition of FIB is ..
>   )
>
> then the buffer would contain at least 2 sections.   Finding the
> definition of FIB in that buffer doesn't need to start at the beginning
> of the file and start searching.(and doesn't need to get confused
> by the string "(defun fib" in a comment.).
>
> This is a little tricker than it may sound: real buffers (while people
> are editing in them) contain partial, syntactically invalid forms,
> duplicate definitions, conditionalization, etc.  I think that a lot
> of things (syntax coloring/highlighting, "list definitions", "track
> changed definitions", ...) as well as meta-. would be easier to do
> if the buffer had been sectionized (or maybe sectionalized) and if
> editing changes maintained that information.
>
> The other classic example is:
>
> ;;; -*- Mode: Lisp; Package: Foo -*-
>
> (in-package "FOO")
>
> (defun ...)
> ....
>
> (in-package "BAR")
> (defvar ...)
>
> For many buffers, the notion that there's a "buffer package" is 
> adequate;
> for some, that isn't possible or practical.  There may be trickier
> cases, but you'd sort of like the editor to behave reasonably in this
> example (and cause the DEFVAR to be read in the BAR package.)
>
>
>>
>> Anyway, is there a way that I could implement this that would really
>> piss people off?
>> If so I'll avoid that one :-)
>>
>> More importantly, I'm fishing for ideas on the best way to do it. I'd
>> like to utilized multiple
>> tags tables, maybe on a per package basis, both to keep the namespace
>> clean and
>> to avoid regenerating tables for source that is largely static, like
>> the OpenMCL core.
>
> It's an oversimplification, but I think that any solution (external
> tags files or using what's already there) eventually degenerates into
> searching a buffer (with or without a strong hint about where to start
> that search.)  I think that the important thing is to make that search
> smarter.
>
>>
>> Anybody have any ideas one the best way to keep track of the files? 
>> One
>> way might
>> be to put a link to a table as meta-data in the source file and have
>> the table updated
>> whenever the file is saved. You could then have some concept of a
>> project manager
>> that would maintain preferences for groups of files and their
>> associated indexes.
>
> As files are loaded, the lisp records the source file associated with
> each top-level definition.  The source-file information for everything
> defined in OpenMCL is fairly complete.
>
> The argument in favor of tags files is that they provide a mechanism
> to record source file information about files that have not (yet)
> been compiled.  The arguments against them is that they introduce
> a different class of (and more) synchronization issues.
>
> I've never missed the ability to navigate around (via meta-.) in the
> sources of a system that I haven't loaded, but this may be more an
> issue of what I'm used to and what my expectations are than a matter
> of what The Right Thing is.  I would say that I think people are
> more likely to be interested in the sources of code that they've
> loaded and of OpenMCL itself than they would be of arbitrary other
> systems, and that it's probably true that code you're running/working
> on is more likely to be interesting than code that's in the filesystem
> somewhere and has been indexed somehow.
>
>>
>> :alex
>>
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>>
>>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>