[Openmcl-devel] hemlock and tags files

[Hamilton Link]

... upon re-reading this I realized I wasn't very clear.

My _point_ was actually that if the lisp editor doesn't focus on what's 
loaded into the environment, there's no other way of getting 
information about that, whereas there's plenty of tools to see what's 
in all my lisp files, depending on what I'm looking for. SO if you 
support anything, support the whats-loaded view (and support tag file 
data as a secondary thing).

h

On Oct 7, 2004, at 8:50 PM, Hamilton Link wrote:

> The big difference is that tags files of course deal with all the 
> files you run through it,
> while the source-file, who-calls, etc. information in lisp only knows 
> about what has been
> loaded. There are good things about both, but normally I care about 
> the definitions of things
> that are loaded, and I can usually grep for what I want outside of 
> that. Just my 2 cents.
> h
>
> On Oct 7, 2004, at 8:24 PM, Gary Byers wrote:
>
>> The fact that OpenMCL already records the source file associated with
>> most defining forms (and uses this information so that Meta-. works in
>> ILisp/SLIME) strongly suggests one implementation strategy.
>>
>> (Yes, I realize that the information parsed into a tags file is
>> potentially different from the information that the lisp already
>> maintains.  No, I don't see a reason to maintain two sets of source
>> file info.)
>>
>> Other people might prefer a tags file scheme, or having the choice
>> somehow (I'm not sure how well the two schemes would coexist, but
>> it might be possible to make them do so.)
>>
>>
>> On Thu, 7 Oct 2004, alex crain wrote:
>>
>>> I'm getting ready to work on the source tagging features in hemlock. 
>>> My
>>> immediate goals are to
>>> be able to click on a function and get either source code or
>>> documentation for the function.
>>>
>>> Documentation is moving along nicely and now I'm thinking about 
>>> source
>>> code indexing.
>>>
>>> I want to model tagging after the emacs implementation, with the one
>>> added feature that I want
>>> to be able to do find-tag on some text and have hemlock produce
>>> pointers to all the source
>>> that defines that tag, maybe using a drawer or something to show the
>>> options. I get annoyed when
>>> I do find-tag and get referred to a piece of commented out code.
>>
>> There is (of course) a tradeoff or two here.  If you maintain exact
>> source-file/byte position information for each definition, then yes,
>> that's helpful and the editor should be able to find that definition
>> with a high degree of reliability.
>>
>> As soon as the file has been edited with an external editor, that
>> breaks down (at least a bit), and even if you only consider cases
>> where the file's only edited in Hemlock, you have to keep those
>> byte positions updated (perhaps by having translated them to marks
>> a bit earlier.)  In some cases, this may degenerate into a heuristic
>> search.
>> v
>> The other case - where you just try to associate the file with
>> the definition, without trying to maintain accurate position 
>> information -
>> sort of uses heuristic search as a starting point.  It avoids the
>> synchronization issues entirely, but how robust it is depends on how
>> good the heuristics that guide the search are.  This approach works
>> well some of the time and very poorly some of the time; the heuristic
>> search sort of needs to be a full-blown lisp parser to do a good job
>> of avoiding commented-out/conditionalized-out definitions.
>>
>> I think that people who've thought about this before noticed that
>> a lisp-aware parser was already a part of a lisp-aware editor, and
>> that it was possible to "sectionize" (I -think- that that was the
>> term; it might have been "sectionalize") a buffer into regions or
>> region-like things.  If a buffer contains:
>>
>> ;;; -*- Mode: Lisp ... *-*
>>
>> (defun fact (n)
>>   (if (zerop n)
>>     1
>>     (* n (fact (1- n)))))
>>
>> ;; I used to be able to do "(defun fib (" with my eyes closed.
>> (defun fib (m n)
>>   ;; whatever the definition of FIB is ..
>>   )
>>
>> then the buffer would contain at least 2 sections.   Finding the
>> definition of FIB in that buffer doesn't need to start at the 
>> beginning
>> of the file and start searching.(and doesn't need to get confused
>> by the string "(defun fib" in a comment.).
>>
>> This is a little tricker than it may sound: real buffers (while people
>> are editing in them) contain partial, syntactically invalid forms,
>> duplicate definitions, conditionalization, etc.  I think that a lot
>> of things (syntax coloring/highlighting, "list definitions", "track
>> changed definitions", ...) as well as meta-. would be easier to do
>> if the buffer had been sectionized (or maybe sectionalized) and if
>> editing changes maintained that information.
>>
>> The other classic example is:
>>
>> ;;; -*- Mode: Lisp; Package: Foo -*-
>>
>> (in-package "FOO")
>>
>> (defun ...)
>> ....
>>
>> (in-package "BAR")
>> (defvar ...)
>>
>> For many buffers, the notion that there's a "buffer package" is 
>> adequate;
>> for some, that isn't possible or practical.  There may be trickier
>> cases, but you'd sort of like the editor to behave reasonably in this
>> example (and cause the DEFVAR to be read in the BAR package.)
>>
>>
>>>
>>> Anyway, is there a way that I could implement this that would really
>>> piss people off?
>>> If so I'll avoid that one :-)
>>>
>>> More importantly, I'm fishing for ideas on the best way to do it. I'd
>>> like to utilized multiple
>>> tags tables, maybe on a per package basis, both to keep the namespace
>>> clean and
>>> to avoid regenerating tables for source that is largely static, like
>>> the OpenMCL core.
>>
>> It's an oversimplification, but I think that any solution (external
>> tags files or using what's already there) eventually degenerates into
>> searching a buffer (with or without a strong hint about where to start
>> that search.)  I think that the important thing is to make that search
>> smarter.
>>
>>>
>>> Anybody have any ideas one the best way to keep track of the files? 
>>> One
>>> way might
>>> be to put a link to a table as meta-data in the source file and have
>>> the table updated
>>> whenever the file is saved. You could then have some concept of a
>>> project manager
>>> that would maintain preferences for groups of files and their
>>> associated indexes.
>>
>> As files are loaded, the lisp records the source file associated with
>> each top-level definition.  The source-file information for everything
>> defined in OpenMCL is fairly complete.
>>
>> The argument in favor of tags files is that they provide a mechanism
>> to record source file information about files that have not (yet)
>> been compiled.  The arguments against them is that they introduce
>> a different class of (and more) synchronization issues.
>>
>> I've never missed the ability to navigate around (via meta-.) in the
>> sources of a system that I haven't loaded, but this may be more an
>> issue of what I'm used to and what my expectations are than a matter
>> of what The Right Thing is.  I would say that I think people are
>> more likely to be interested in the sources of code that they've
>> loaded and of OpenMCL itself than they would be of arbitrary other
>> systems, and that it's probably true that code you're running/working
>> on is more likely to be interesting than code that's in the filesystem
>> somewhere and has been indexed somehow.
>>
>>>
>>> :alex
>>>
>>> _______________________________________________
>>> Openmcl-devel mailing list
>>> Openmcl-devel at clozure.com
>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>>
>>>
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>