[Openmcl-devel] CCL crash in GSLL

Gary Byers gb at clozure.com
Thu Jun 24 10:36:45 PDT 2010 

/home/healy/languages/lisp/clbuild/source/ccl/lx86cl64: free():
invalid pointer: 0x00000000016dfcd0 ***

is basically saying that the C library function free() finds something
wrong with it's argument: either that pointer wasn't allocated by
malloc(), or it's already been freed, or something's written outside
the bounds of some data structure(s) maintained by malloc (e.g., malloc's
heap is corrupt) making it appear that the pointer's invalid.

The backtrace tells us that free() was called as a foreign function from
lisp code:

> ======= Backtrace: =========
> /lib/libc.so.6(+0x71b16)[0x7f29caea9b16]
> /home/healy/languages/lisp/clbuild/source/ccl/lx86cl64(_SPffcall+0x7a)[0x411b6a]

CCL itself makes a little bit of use of malloc/free, but (for bootstrapping
reasons) doesn't call the library routines directly, and the presence of
_SPffcall in the backtrace indicates that free was called from lisp code.

We don't know whether that call to free is incorrect (trying to free
something that's already freed or was never malloc'ed) or if the
problem's elsewhere, but we do know that the call to free that detects
the problem is coming from lisp code (but not CCL itself), which
suggests that it's coming from GSLL or one of its dependant libraries.
(Again, that call might be perfectly correct but one of the first
things to look at is that code: if free is called from a small number
of places and not called all that often, then it might be possible to
spot the problem just by looking at the code.  (If GSLL is a large
system and/or if it makes heavy, non-trivial use of malloc/free, then
this isn't really practical.)

Linux's malloc implementation has a facility ("mtrace") that causes all calls
to malloc/free/realloc/... to be logged to a file; CCL has some utilites
that can parse this log file and report some kinds of inconsistencies.  The
wiki page <http://trac.clozure.com/ccl/wiki/MemoryLeaks> describes thes
facilities.  (I think that it's safe to ignore the claim that those tools
are only available in a development branch, and that they've been in the
trunk/releases for a while now.)

If that approach doesn't get anywhere, you might check the malloc man page
to see if there are ways of persuading malloc to do more extensive integrity
checking.  (I think that this exact problem may have been mentioned on #ccl
a week or two ago and that the person who reported it tried enabling the
extra malloc checks but found that nothing was reported and things failed the
same way.  That's not necessarily conclusive, and YMMV.)

If none of this gets you anywhere ... well, the only thing that I can say
is that this sort of thing can be very hard to debug.


On Thu, 24 Jun 2010, Liam Healy wrote:

> If I run the Mathieu tests from GSLL (http://common-lisp.net/project/gsll/)
> twice in a row, CCL crashes.   Can anyone provide insight in how to
> debug this?  I'm at a loss.  It's clearly a delayed error, i.e., the
> real problem occurs before the error is triggered.  Maybe GSLL is
> scribbling in foreign memory where it shouldn't, or something like
> that?
>
> Thank you for an ideas.
>
> Liam
>
> Welcome to Clozure Common Lisp Version 1.6-dev-r13829M-trunk  (LinuxX8664)!
> ? (require :gsll-tests)
> ; loading system definition from
> /home/healy/languages/lisp/clbuild/systems/lisp-unit.asd into
> #<Package "ASDF0">
> ; registering #<SYSTEM :LISP-UNIT> as LISP-UNIT
> ; loading system definition from
> /home/healy/languages/lisp/clbuild/systems/gsll.asd into #<Package
> "ASDF0">
> ; loading system definition from
> /home/healy/languages/lisp/clbuild/systems/fsbv.asd into #<Package
> "ASDF1">
> ; registering #<SYSTEM :FSBV> as FSBV
> ; registering #<SYSTEM GSLL> as GSLL
> ; loading system definition from
> /home/healy/languages/lisp/clbuild/systems/grid.asd into #<Package
> "ASDF0">
> ; registering #<SYSTEM GRID> as GRID
> ; registering #<SYSTEM-CONNECTION GRID-ITERATE-EXTENSION> as
> GRID-ITERATE-EXTENSION
> ; loading system definition from
> /home/healy/languages/lisp/clbuild/systems/c-array.asd into #<Package
> "ASDF0">
> ; registering #<SYSTEM C-ARRAY> as C-ARRAY
> :GSLL-TESTS
> NIL
> ? (in-package :gsl)
> #<Package "GSLL">
> ? (lisp-unit:run-tests mathieu)
> MATHIEU: 75 assertions passed, 0 failed.
> ? (lisp-unit:run-tests mathieu)
> MATHIEU: 75 assertions passed, 0 failed.
> ? *** glibc detected ***
> /home/healy/languages/lisp/clbuild/source/ccl/lx86cl64: free():
> invalid pointer: 0x00000000016dfcd0 ***
> ======= Backtrace: =========
> /lib/libc.so.6(+0x71b16)[0x7f29caea9b16]
> /home/healy/languages/lisp/clbuild/source/ccl/lx86cl64(_SPffcall+0x7a)[0x411b6a]
> ======= Memory map: ========
> 00012000-00014000 rwxp 00000000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 00015000-00016000 rwxp 00000000 00:00 0
> 00400000-0042c000 r-xp 00000000 09:00 811013769
>  /home/healy/languages/lisp/clbuild/source/ccl/lx86cl64
> 0062b000-0062c000 rwxp 0002b000 09:00 811013769
>  /home/healy/languages/lisp/clbuild/source/ccl/lx86cl64
> 0062c000-0062d000 rwxp 00000000 00:00 0
> 0166a000-01714000 rwxp 00000000 00:00 0                                  [heap]
> 300000000000-30000020a000 r-xp 00002000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 30000020a000-30000020b000 rwxp 0020c000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 30000020b000-300000c02000 r-xp 0020d000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 300000c02000-300000c04000 rwxp 00c04000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 300000c04000-300001532000 r-xp 00c06000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 300001532000-300040000000 ---p 00000000 00:00 0
> 300040000000-30004077c000 rwxp 01f7d000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 30004077c000-302000000000 ---p 00000000 00:00 0
> 302000000000-302000a49000 rwxp 01534000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 302000a49000-302002a60000 rwxp 00000000 00:00 0
> 302002a60000-307c00000000 ---p 00000000 00:00 0
> 307c00000000-307c00055000 rwxp 00000000 00:00 0
> 307c00055000-307e00000000 ---p 00000000 00:00 0
> 307e00000000-307e0000f000 rwxp 026f9000 09:00 540990197
>  /home/healy/languages/lisp/clbuild/monster.image
> 307e0000f000-307e3f800000 ---p 00000000 00:00 0
> 307e3f800000-307e3f855000 rwxp 00000000 00:00 0
> 307e3f855000-308000020000 ---p 00000000 00:00 0
> 7f29c8cb2000-7f29c8cc8000 r-xp 00000000 09:00 810489408
>  /lib/libgcc_s.so.1
> 7f29c8cc8000-7f29c8ec7000 ---p 00016000 09:00 810489408
>  /lib/libgcc_s.so.1
> 7f29c8ec7000-7f29c8ec8000 rwxp 00015000 09:00 810489408
>  /lib/libgcc_s.so.1
> 7f29c8ec8000-7f29c90d5000 r-xp 00000000 09:00 538597194
>  /usr/lib/libgsl.so.0.15.0
> 7f29c90d5000-7f29c92d4000 ---p 0020d000 09:00 538597194
>  /usr/lib/libgsl.so.0.15.0
> 7f29c92d4000-7f29c92e7000 rwxp 0020c000 09:00 538597194
>  /usr/lib/libgsl.so.0.15.0
> 7f29c92e7000-7f29c9320000 r-xp 00000000 09:00 538597195
>  /usr/lib/libgslcblas.so.0.0.0
> 7f29c9320000-7f29c951f000 ---p 00039000 09:00 538597195
>  /usr/lib/libgslcblas.so.0.0.0
> 7f29c951f000-7f29c9520000 rwxp 00038000 09:00 538597195
>  /usr/lib/libgslcblas.so.0.0.0
> 7f29c9520000-7f29c9527000 r-xp 00000000 09:00 537339989
>  /usr/lib/libffi.so.5.0.10
> 7f29c9527000-7f29c9727000 ---p 00007000 09:00 537339989
>  /usr/lib/libffi.so.5.0.10
> 7f29c9727000-7f29c9728000 rwxp 00007000 09:00 537339989
>  /usr/lib/libffi.so.5.0.10
> 7f29c9728000-7f29c9742000 r-xp 00000000 00:00 0
> 7f29c9742000-7f29c9943000 rwxp 00000000 00:00 0
> 7f29c9943000-7f29c9ba7000 rwxp 00000000 00:00 0
> 7f29c9ba7000-7f29c9bf6000 r-xp 00000000 09:00 537803533
>  /usr/lib/libssl.so.0.9.8
> 7f29c9bf6000-7f29c9df5000 ---p 0004f000 09:00 537803533
>  /usr/lib/libssl.so.0.9.8
> 7f29c9df5000-7f29c9dfc000 rwxp 0004e000 09:00 537803533
>  /usr/lib/libssl.so.0.9.8
> 7f29c9dfc000-7f29c9e03000 r-xp 00000000 09:00 809931197
>  /lib/libnss_compat-2.11.1.so
> 7f29c9e03000-7f29ca002000 ---p 00007000 09:00 809931197
>  /lib/libnss_compat-2.11.1.so
> 7f29ca002000-7f29ca003000 r-xp 00006000 09:00 809931197
>  /lib/libnss_compat-2.11.1.so
> 7f29ca003000-7f29ca004000 rwxp 00007000 09:00 809931197
>  /lib/libnss_compat-2.11.1.so
> 7f29ca004000-7f29ca019000 r-xp 00000000 09:00 810180873
>  /lib/libnsl-2.11.1.so
> 7f29ca019000-7f29ca218000 ---p 00015000 09:00 810180873
>  /lib/libnsl-2.11.1.so
> 7f29ca218000-7f29ca219000 r-xp 00014000 09:00 810180873
>  /lib/libnsl-2.11.1.so
> 7f29ca219000-7f29ca21a000 rwxp 00015000 09:00 810180873
>  /lib/libnsl-2.11.1.so
> 7f29ca21a000-7f29ca21c000 rwxp 00000000 00:00 0
> 7f29ca21c000-7f29ca226000 r-xp 00000000 09:00 810015522
>  /lib/libnss_nis-2.11.1.so
> 7f29ca226000-7f29ca425000 ---p 0000a000 09:00 810015522
>  /lib/libnss_nis-2.11.1.so
> 7f29ca425000-7f29ca426000 r-xp 00009000 09:00 810015522
>  /lib/libnss_nis-2.11.1.so
> 7f29ca426000-7f29ca427000 rwxp 0000a000 09:00 810015522
>  /lib/libnss_nis-2.11.1.so
> 7f29ca427000-7f29ca432000 r-xp 00000000 09:00 810190371
>  /lib/libnss_files-2.11.1.so
> 7f29ca432000-7f29ca631000 ---p 0000b000 09:00 810190371
>  /lib/libnss_files-2.11.1.so
> 7f29ca631000-7f29ca632000 r-xp 0000a000 09:00 810190371
>  /lib/libnss_files-2.11.1.so
> 7f29ca632000-7f29ca633000 rwxp 0000b000 09:00 810190371
>  /lib/libnss_files-2.11.1.so
> 7f29ca633000-7f29ca7a8000 r-xp 00000000 09:00 537604919
>  /usr/lib/libcrypto.so.0.9.8
> 7f29ca7a8000-7f29ca9a8000 ---p 00175000 09:00 537604919
>  /usr/lib/libcrypto.so.0.9.8
> 7f29ca9a8000-7f29ca9d0000 rwxp 00175000 09:00 537604919
>  /usr/lib/libcrypto.so.0.9.8
> 7f29ca9d0000-7f29ca9d4000 rwxp 00000000 00:00 0
> 7f29ca9d4000-7f29ca9eb000 r-xp 00000000 09:00 539438856
>  /usr/lib/libz.so.1.2.3.4
> 7f29ca9eb000-7f29cabea000 ---p 00017000 09:00 539438856
>  /usr/lib/libz.so.1.2.3.4
> 7f29cabea000-7f29cabeb000 rwxp 00016000 09:00 539438856
>  /usr/lib/libz.so.1.2.3.4
> 7f29cabeb000-7f29cac05000 r-xp 00000000 00:00 0
> 7f29cac05000-7f29cae38000 rwxp 00000000 00:00 0
> 7f29cae38000-7f29caf90000 r-xp 00000000 09:00 810228467
>  /lib/libc-2.11.1.so
> 7f29caf90000-7f29cb18f000 ---p 00158000 09:00 810228467
>  /lib/libc-2.11.1.so
> 7f29cb18f000-7f29cb193000 r-xp 00157000 09:00 810228467
>  /lib/libc-2.11.1.so/home/healy/languages/lisp/clbuild/clbuild: line
> 414: 23314 Aborted                 ${lisp} $common_options $eval
> "$require_asdf" $eval "$set_central_registry" $eval
> "$EXTRA_CLIM_FEATURES" "$@"
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>

More information about the Openmcl-devel mailing list