[Openmcl-devel] slow read-char

Thu Jul 27 19:45:31 PDT 2006

I'm not sure what I can do here.  The options seem to be:

1) stop making changes that're hard to bootstrap
2) ensure that every set of changes is accompanied by a bootstrapping image
    that'll work until the next round of changes breaks things
3) announce that the bleeding-edge CVS tree will be very hard to bootstrap
    from until a bunch of hard-to-bootstrap, low-level stream changes
    are complete.

(3) seemed to make the most sense.  I said a week ago that that'd be
"a day or two" until the dust settled; I obviously wish that it had
been.  (I also wish that I was younger, that it wasn't so hot, that
I didn't have other work that needed doing, and that I didn't keep
seeing other changes that would lead to performance gains but which
made the whole project more ambitious.)

I did tag the stuff that was in CVS as of a few days ago, so that
anyone who really, really wanted to see the stream changes as of
the point when the last bootstrapping images were made can obtain
those exact sources.  The reason for tagging them was the certain
knowledge that the next round of changes would break things, and
of course it did.

If there's another approach to this sort of thing, I'd be glad to
explore it.  If I'm correct in thinking that there are only the
three approaches that I enumerated above, I don't believe that
anything but (3) is really viable.  ((1) might be a distant second;
that'd be "leave things more or less alone, because it's too
hard to bootstrap changes to something as fundamental as what
a stream is and how it's accessed."  There are some arguments
in favor of that, but - as someone who's expresed concerns about
I/O performance - I think that you'd have ultimately found that
to be undesirable; it's basically saying "stream performance is
mediocre at best, and the primary reason it doesn't improve is
that that would make the system harder for people to bootstrap
while those changes are made.")

On Thu, 27 Jul 2006, Erik Pearson wrote:

> Okay, so I discovered xcompile-ccl, which allows things to work
> better since it doesn't load funky stuff as it goes along. It can't
> compile l1-streams:
>
> Compiling "/usr/local/ccl/ccl/level-1/l1-streams.lisp"...
> > Error: While compiling (STREAM-READ-CHAR-NO-HANG (BUFFERED-
> CHARACTER-INPUT-STREAM-MIXIN)) :
> >        Not enough args in Lambda form: (STREAM ERROR-IF-NIL)
> (STREAM)., in process listener(1).
> > Type :POP to abort, :R for a list of available restarts.
> > Type :? for other options.
> 1 > (:b)
>
> Without l1-streams compiling, the system can't load from ppc-
> boot.image in order to create the standard image. Of course, in side
> the guts of this thing, I'm hopeless as far as determining what is
> really wrong here... (yeah, it is interesting to glaze over the
> backtrace, but...)
>
> Erik.
>
>
> On Jul 26, 2006, at 12:28 PM, Erik Pearson wrote:
>
>> Hi Gary,
>>
>> It looks like, as predicted, cvs bleeding is busted -- even using the
>> new test image.
>>
>> compile-ccl gets to
>>
>> ;Loading #P"/usr/local/ccl/ccl/bin/streams.dfsl"...
>> ;Compiling "/usr/local/ccl/ccl/lib/pathnames.lisp"...
>> Read error between positions 0 and 726 in /usr/local/ccl/ccl/lib/
>> pathnames.lisp.
>>> Error: Unknown type specifier: BASIC-STREAM
>>> While executing: %%TYPEP, in process listener(1).
>>
>> and then dies, leaving the REPL in a sad unusable state.
>>
>> Dare I guess this is a boostrapping issue? Some of the stream code
>> has already been loaded that requires BASIC-STREAM, but basic stream
>> hasn't been loaded yet, or incompatible streams have already been
>> created, or something like that? Is this a matter of magically
>> creating a compatible image, or can it be solved by changing the
>> build procedure, or is such an outlier case (messing with system
>> fundamentals) that you'd rather just special-case it? I would hope
>> that rebuild-ccl (thanks for that!) would be able to handle
>> reconstructing the system from pieces. Perhaps a very primitive lisp
>> that doesn't rely on high(er) level functionality (single-threaded,
>> just reads bytes, etc.) could be invoked to rebuild the system, even
>> from the Unix command line? But what the bleep do I know.
>>
>> Erik.
>>
>> On Jul 22, 2006, at 5:03 AM, Gary Byers wrote:
>>
>>>
>>>
>>> On Sat, 22 Jul 2006, Eric Marsden wrote:
>>>
>>>>>>>>> "gb" == Gary Byers <gb at clozure.com> writes:
>>>>
>>>>  gb> It's tricky to bootstrap a few of those changes (I actually
>>>> managed
>>>>  gb> to produce images that were much less functional than yours
>>>> seems to
>>>>  gb> be ...).
>>>>
>>>>  I have finally found a lisp implementation that is even harder to
>>>>  build than CMUCL ; at least CMUCL developers generally provide
>>>>  bootstrap files that help automate the build process :-)
>>>>
>>>
>>>
>>> There are now bootstrapping images for 32/64-bit DarwinPPC and
>>> x86-64 Linux in the testing directory; if anyone wants LinuxPPC
>>> images, please let me know.
>>>
>>> I also tagged the current bleeding-edge tree (whose contents match
>>> those images) with the tag "work_in_progress_060720".  This may
>>> make it easier for anyone who wants to insulate themselves from
>>> subsequent changes to do so.  (Recall that CVS tags are "sticky"
>>> and that it may be necessary to clear them in order to get back
>>> in synch with the head.)
>>>
>>> Hopefully, these will become obsolete (incompatible with bleeding-
>>> edge
>>> CVS and unable to easily build from it) later today (or as soon as
>>> I can check in the next round of incompatible, difficult to bootstrap
>>> low-level stream changes.)
>>>
>>> Since this is likely to be pretty volatile - and since I don't really
>>> have a better way of keeping all platforms in synch without checking
>>> in a lot of incremental (aka half-finished) changes, I think that the
>>> advice that I gave the other day still holds: it's likely to be
>>> tricky to build the lisp from CVS while these changes are taking
>>> place, it's still not likely to take more than a few days, and when
>>> that dust settles I'd certainly -want- to provide images and/or
>>> snapshot tarballs to make it easy for people to use and test the new
>>> code.
>>>
>>> Aside from the fact that it's evidence of a step in the right
>>> direction, I don't think that the 060720 code is particularly
>>> interesting.  If people are concerned about I/O performance and
>>> want to see evidence of that first step, then I suppose that
>>> these images might be of greater interest.
>>>
>>>
>>>> --
>>>> Eric Marsden
>>>>
>>>> _______________________________________________
>>>> Openmcl-devel mailing list
>>>> Openmcl-devel at clozure.com
>>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>>>
>>>>
>>> _______________________________________________
>>> Openmcl-devel mailing list
>>> Openmcl-devel at clozure.com
>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>