<div dir="ltr"><div>I wonder if you can detect the first time you see corruption, and do so without causing a crash?</div><div><br></div>I know this would change the timing, and perhaps hide the problem or perhaps not, but consider changing the C function to examine the string upon entry to the function. Go back to the statically-allocated version so the pointer to the string doesn't change. During initialization, save that pointer somewhere "far" from where the pointer that is returned to Lisp is stored. (E.g. one statically allocated, one on the heap. That is so that if there is corruption, hopefully not both pointers are corrupted.) <div>
<br></div><div>Upon entry to the C function, compare the pointer that will be returned to the saved pointer. If they are not equal, try and generate a breakpoint. If they are equal, compare the string to the original string (also saved elsewhere). If the strings differ in content or length, generate a breakpoint. <div>
<br></div><div>Because of multiple threads, perhaps have a guard variable. When corruption is detected, set the guard variable. Upon entry to the function, if the guard variable is set, perhaps return a constant NULL from code, avoiding referencing the string in memory, so that subsequent calls from other threads don't continue to corrupt the corrupted memory, or don't reference the corrupted memory, which may cause other exceptions.</div>
<div><br></div><div>Jeff</div><div><br></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Oct 31, 2013 at 5:48 AM, Paul Meurer <span dir="ltr"><<a href="mailto:Paul.Meurer@uni.no" target="_blank">Paul.Meurer@uni.no</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div>Am 31.10.2013 um 01:15 schrieb Gary Byers <<a href="mailto:gb@clozure.com" target="_blank">gb@clozure.com</a>>:</div>
<div class="im"><br><blockquote type="cite">On Wed, 30 Oct 2013, Paul Meurer wrote:<br><blockquote type="cite">I run it now with --no-init and in the shell, with no difference. Immediate failure with :consing in *features*,<br>
bogus objects etc. after several rounds without :consing.<br></blockquote><br>So, I can't rant and rave about the sorry state of 3rd-party CL libraries, and<br>anyone reading this won't be subjected to me doing so ?<br>
<br>Oh well.<br><br>I was able to reproduce the problem by running your test 100 times, </blockquote><div><br></div></div><div>I am not able to provoke it at all on the MacBook, and I tried a lot.</div><div class="im"><br>
<blockquote type="cite">so apparently<br>I won't be able to blame this on some aspect of your machine. (Also unfortunate,<br>since my ability to diagnose problems that only occur on 16-core machines depends<br>on my ability to borrow such machines for a few months.)<br>
</blockquote><div><br></div></div><div>I think you can do without a 16-core machine. I am able to reproduce the failure quite reliably on an older 4-core machine with Xeon CPUs and SuSE, with slightly different code (perhaps to get the timing right):</div>
<div><br></div><div><div>(dotimes (j 100)</div><div> (print (ccl::all-processes))</div><div> (dotimes (i 8)</div><div> (process-run-function</div><div> (format nil "getstring-~a-~a" j i)</div><div> (lambda (i)</div>
<div> (let ((list ()))</div><div> (dotimes (i 500000)</div><div> (push (getstring) list)))</div><div> (print i))</div><div> i))</div><div> (print (list :done j))</div><div> (sleep 1))</div>
<div><br></div></div><div>If you really need a 16-core machine to debug this I can give you access to mine. :-)</div><div class="im"><br><blockquote type="cite"><blockquote type="cite">My machine has 16 true cores and hyperthreading; I am running CentOS 6.0, and a recent CCL 1.9 (I did svn update +<br>
rebuild of everything yesterday).<br>I also observed that the problem goes away when I replace the constant string in the library by a freshly<br>allocated string:<br>char *getstring() {?<br>? int index;<br>? char *buffer = (char *)calloc(100 + 1, sizeof(char));<br>
? for (index = 0; index < 100; index++) {<br>? ? ? buffer[index] = 'a';<br>? ? }<br>? buffer[100] = '\0';<br>? return buffer ;<br>}<br>One should expect the strings in the Postgres library to be freshly allocated, but nevertheless they behave like<br>
the constant string example.<br></blockquote><br>It's unlikely that this change directly avoids the bug (whatever it is); it's more<br>likely that it affects timing (exactly what happens when.) I don't yet know what<br>
the bug is, but I think that it's likely that it's fair to characterize the bug<br>as being "timing-sensitive". (For example: from the GC's point of view, whether<br>a thread is running Lisp or foreign code when that thread is suspended by the GC.<br>
The transition between Lisp and foreign code takes a few instructions, and if<br>a thread is suspended in the middle of that instruction sequence and the GC<br>misintrprets its state, very bad things like what you're seeing could occur.<br>
That's not supposed to be possible, but something broadly similar seems to be<br>happening.)<br></blockquote></div></div><span class="HOEnZb"><font color="#888888"><br><div>
<span style="border-collapse:separate;font-family:'Lucida Grande';border-spacing:0px">-- <br>Paul</span>
</div>
<br></font></span></div><br>_______________________________________________<br>
Openmcl-devel mailing list<br>
<a href="mailto:Openmcl-devel@clozure.com">Openmcl-devel@clozure.com</a><br>
<a href="http://clozure.com/mailman/listinfo/openmcl-devel" target="_blank">http://clozure.com/mailman/listinfo/openmcl-devel</a><br>
<br></blockquote></div><br></div>