[Openmcl-devel] Slow Slime Startup with 1.9 dev OS X

[Gary Byers]

Matt opened a ticket for the problem that Wade reported a couple of weeks 
ago (and that he'd been seeing); see <http://trac.clozure.com/ccl/ticket/1005>.
Matt was able to reproduce the problem and investigated; the short version is
that a data structure that CCL allocates whenever a thread is first made known
to lisp has to be in the low 32 bits of the address space and the way we'd
historically tried to do that (calling #_malloc repeatedly until it returned
a 32-bit pointer) could take an extremely long time.  I don't think that we'd
ever seen or heard of it doing so prior to 10.8, but I think that that was just
due to luck (and good clean living ...).  It's possible that 10.8 tries to do
some sort of address space randomization (this is sometimes seen as a way of
minimizing the damage after malicious could forces a buffer overflow) that previous
OSX versions didn't,  and that may explain why the problem seems to have just
surfaced with 10.8.  (It didn't: the "call #_malloc and cross your fingers"
strategy has never been a viable one.)

This was fixed in r15437 (but I was nervous about some thread-safety issues
and tried to deal with them better in r15439.)  Both of those changes were
just made to the trunk; they haven't yet been propagated to the 1.8 tree but
likely will fairly soon.

This particular problem (trying to force a pointer to be allocated in
the low 32 bits of the address space and doing so in bounded time)
obviously only affects the 64-bit version of CCL.

On Thu, 23 Aug 2012, Wade Humeniuk wrote:

> There was a commit by Gary that has fixed the slow slime startup for me. ?I
> think commit 15437 fixed it.
> 
> Dex:ccl wade$ ./dx86cl64?
> Welcome to Clozure Common Lisp Version 1.9-dev-r15439M-trunk ?(DarwinX8664)!
> ??
> 
> Wade
> 
> On 2012-08-23, at 6:05 PM, Alexander Repenning
> <alexander.repenning at Colorado.EDU> wrote:
>
>       I am beginning to wonder if this is different manifestation of
>       the same problem we have. Does it only happen with OS X
>       10.8/Mountain Lion? When it does happen, and it may not happen
>       all the time, does not only CPU but also RealMemory go up
>       significantly, e.g., from say 100 MB to a couple of GB and
>       sometime even crash (particularly the 32 bit version)?
> 
> Can't be at all sure about it but so far there seems to be some
> correlation between this CPU/Memory surge and creating new threads or
> not servicing events for some time in the main thread. Never seen this
> on Snow Leopard or Lion.?
> 
> Alex
> 
> 
> On Aug 11, 2012, at 8:13 PM, Wade Humeniuk wrote:
>
>       I updated my 1.9 devel branch a few days ago and now the
>       time to start ccl under slime is quite large. ?Sometimes
>       greater than 30 secs (and a few times I had to abort slime
>       connection). ?Looking at the OS X Activity Monitor shows
>       ccl taking up 100% cpu during swank/slime startup and goes
>       negligible after connecting. ?I originally thought it had
>       something to do with Mountain ?Lion and me building ccl
>       under that latest Xcode. ?I svn'ed the 1.8 release version
>       and it worked fine, I also re-build 1.8 under mountain
>       lion and everything was also fine.
> Version with slow behaviour
> 
> Dex:ccl wade$ ./dx86cl64?
> Welcome to Clozure Common Lisp Version 1.9-dev-r15382M-trunk
> ?(DarwinX8664)!
> ??
> 
> 1.8 version
> 
> Dex:ccl18 wade$ ./dx86cl64?
> Welcome to Clozure Common Lisp Version 1.8-r15436M
> ?(DarwinX8664)!
> ??
> 
> 
> Is anyone else having the issue?
> 
> Wade
> 
> Xcode 4.4.1
> 
> Hardware Overview:
> 
> ? Model Name: MacBook Pro
> ? Model Identifier: MacBookPro8,2
> ? Processor Name: Intel Core i7
> ? Processor Speed: 2.2 GHz
> ? Number of Processors: 1
> ? Total Number of Cores: 4
> ? L2 Cache (per Core): 256 KB
> ? L3 Cache: 6 MB
> ? Memory: 8 GB
> ? Boot ROM Version: MBP81.0047.B27
> ? SMC Version (system): 1.69f3
> ? Serial Number (system): C2QHF01ADY1N
> ? Hardware UUID: A9F33418-B914-5D86-9773-522EB924085F
> ? Sudden Motion Sensor:
> ? State: Enabled
> 
> System Software Overview:
> 
> ? System Version: OS X 10.8 (12A269)
> ? Kernel Version: Darwin 12.0.0
> ? Boot Volume: Macintosh HD
> ? Boot Mode: Normal
> ? Computer Name: Dex
> ? User Name: Wade Humeniuk (wade)
> ? Secure Virtual Memory: Enabled
> ? Time since boot: 8 days 21:22
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
> 
> 
> Prof. Alexander Repenning
> 
> University of Colorado
> Computer Science Department
> Boulder, CO 80309-430
> 
> vCard: http://www.cs.colorado.edu/~ralex/AlexanderRepenning.vcf
> 
> 
> 
> 
>