[Openmcl-devel] planned trunk instability

[Gary Byers]

I think that 15 days counts as "a few", doesn't it ?

As far as I know, we're through making changes to the trunk that
require extensive bootstrapping, at least for a while.  Changes
that have occurred recently include:

- "nearly lock free" hash tables (whose implementation is described at

    <http://trac.clozure.com/openmcl/wiki/Internals/LockFreeHashTables>

    ).  These are not yet enabled by default; a hash table created
    via (make-hash-table ... :LOCK-FREE T) allows concurrent read-write
    access to the resulting hash-table from multiple threads without
    locking overhead for most operations.  (If I understand correctly,
    locking does come into play when the hash table needs to be
    rehashed due to growth or address-based key movement.)  So far,
    the performance results seem to be encouraging, and I think that
    the plan is to make :LOCK-FREE T the default for shared hash-tables
    (and to try to descibe any usage patterns where it wouldn't be
    an appropriate default better than I'm doing here.)  (Thread-private
    hash-tables - created with :SHARED NIL - are likely to still be
    faster when they're appropriate; they already avoid locking
    overhead and don't need to do the memory-synchronization stuff
    that shared lock-free tables need to do on writes.)


-  the win64 port (which had been on its own branch and was badly
    out of synch with the trunk) is now in the trunk and starting
    to approach usability.

-  a 32-bit x86 Linux port has been added; it and the 64-bit x86
    Linux port have been merged into a new tree at:

    http://svn.clozure.com/publicsvn/openmcl/trunk/linuxx86/ccl

    The old 64-bit only Linux x8664 tree at :...trunk/linuxx8664/ccl
    should be considered "deprecated"; if you've been using it, you
    can do:

shell> cd ccl
shell> svn update      # get in synch with the old linuxx8664 tree
shell> svn switch http://svn.clozure.com/publicsvn/openmcl/trunk/linuxx86/ccl

- both the Darwin and Linux x8632 ports are built with "gc integrity
   checking enabled; this basically means that every time the EGC runs
   (which is typically very often ...), it's preceded and followed by a
   full scan of the entire heap which tries to detect evidence of memory
   corruption.  Each of these full scans usually takes much longer than
   an EGC invocation does and run as often as the EGC does, so code
   that does lots of consing (incidental or otherwise) spends a large
   percentage of its time checking for evidence of memory corruption.

   (When the integrity checks fail, they do so by breaking into
   the kernel debugger with a cryptically-phrased message like
   "pointer at #x12345 has bogus header ..." or something equally
   obscure.  There isn't a whole lot that you can do at this point
   besides report the failure; a reproducible failure is ideal,
   if you can generate one.)

   It's been a while since we've seen such an (x8632-specific) failure,
   and it's tempting to remove the options which force integrity
   checks.  The more code that's run successfully with the integrity
   checks enabled the more confident we can be that they're no longer
   necessary.  (They do tend to catch memory-corruption problems
   soon after whatever causes them occurs, which make them easier
   to find and fix than they would be after the GC has chewed on
   things for a while.)

- The ObjC bridge still doesn't work on darwinx8632, but it's
   getting closer.  Honest.

- The version number that keeps kernels and heap images in synch
   with each other was incremented (so old images won't run with
   new kernels and vice versa) and new kernels/images were checked
   in for all platforms.  The FASL file version number also changed
   (so old fasls won't work with new images and vice versa.)

As far as I can see, we should be back to "some degree of volatility
in the trunk, but not total chaos ..." for the moment, at least.




On Sun, 7 Sep 2008, Gary Byers wrote:

> [So much better than "unplanned trunk instability".]
>
> I want to try to merge in a bunch of changes from the win64 branch and
> make other changes to the trunk over the next few days.  Some of these
> changes may introduce bugs/bootstrapping problems/other instabilities,
> and it may take a few days (for some value of "a few") for the dust to
> settle.
>
> I'll try to remember to send an "all clear" message at that point.
> Until that happens, it's probably wise to avoid updating from the
> svn trunk.
>
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>