[Openmcl-devel] bogus objects

[Gary Byers]

On Sat, 13 Nov 2004, Cyrus Harmon wrote:

> using ccl::make-lock and ccl::with-lock-grabbed seem to have
> ameliorated the problem. I'm not convinced there aren't more gotchas
> lurking, but I think the one I was seeing was due to multiple threads
> trying to write to a global error stream at roughly the same time.

I've run Araneida-based servers here relatively recently (~5 months
ago) and didn't encounter this; either the shared usage of string streams
is relatively new, or I didn't push things hard enough to encounter this.
(Or both, I guess.)

>
> > If that -is- practical, I suspect that you'll see the memory corruption
> > go away.  (I'm relieved that this doesn't seem to be a GC problem.)
> >
> > If it isn't practical, please let me know and I'll try to talk to
> > Dan Barlow about it
>
> I'm pretty new to all this CL threading stuff. It would be great if
> there were a portable way to do this kind of locking.

Threads and locks aren't part of CL (though many/most implementations
offer them in some form or another.)  There are enough differences in
those implementations that it's hard to come up with a universal
portability layer; cross-platform packages that have to deal with
threading issues usually offer their own portability layer that's
adequate for the package's specific needs.

>
> Any nice tools for checking memory integrity I should know about?

Hmm ... that depends on what you mean by "nice".

The symbol CCL::*GC-EVENT-STATUS-BITS* is allocated in a static
memory area (yes, there is one, but there isn't an easy way to
make other static areas) where the kernel/GC can find it.  The
value of that variable is a fixnum, and some individual bits
in that fixnum control GC behavior.  (In commercial MCL, this is/
was used to control things like whether the GC should change the
shape of the mouse cursor, etc.  There aren't a whole lot of
bits that still make sense.)

Setting the CCL::$GC-INTEGRITY-CHECK-BIT in CCL::*GC-EVENT-STATUS-BITS*
[via:

? (setq ccl::*gc-event-status-bits*
        (logior (ash 1 ccl::$gc-integrity-check-bit)
                ccl::*gc-event-status-bits*))

] will cause the GC to do fairly rigourous integrity checks before and
after it runs.  This is mostly intended to help debug the GC itself:
if it finds something wrong, it just drops into the kernel debugger
with a terse message.

The integrity checks generally take longer than the GC itself does.
(This is especially true if most GCs are "ephemeral": the ephemeral
GC runs frequently but only looks at relatively small areas containing
newly allocated objects; the integrity checks traverse the entire
heap.)  In practice, enabling GC integrity checks slows things down
significantly; in practice, if your code runs without triggering
any of these checks, you can be pretty confident that nothing's
fouling up memory.

There are cases where an integrity check will trigger on something
that's in fact harmless, but it's probably better to assume that
anything that the check complains about will eventually cause
problems.

This is sort-of nice in that it can find the needle in the haystack
for you.  Figuring out how the needle got in the haystack is often
difficult; in this case, your observation that string-output-streams
seemed to be involved seems (knock wood) to have enabled us to identify
the problem.