[Openmcl-devel] Solaris x86-64 and %gs

[Gary Byers]

The definition of _lwp_private from
/usr/src/lib/libc/amd64/sys/lwp_private.s (in the OpenSolaris sources)
is basically just a wrapper around a system call; if you look at that
file, you'll probably notice that the confusion about what the function's
supposed to be called hasn't been resolved yet.

There's some macrology in the definition of that little wrapper, but
all that it really does is load the system call index into %rax
(according to <sys/syscalls.h>, SYS_lwp_private = 166), copy the
fourth argument (if there is one; lwp_private only takes 3 args)
from %rcx to %r10, execute a "syscall" instruction, return to
the caller if the carry flag's clear, or interpret the return value
as an error if the carry flag was set on return.

In lisp-kernel/x86-asmutils64.s, there could be some code like:

         __ifdef([SOLARIS])

SYS_lwp_private = 166   /* As per <sys/syscalls.h> */

         .globl C(__cerror)

_exportfn(C(our_lwp_private))
         __(movl $SYS_lwp_private,%eax)
         __(syscall)
         __(jc C(__cerror))
         __(repret)
_endfn
         __endif

If Sun someday decides to renumber system calls or change the interface
to _lwp_private, this would obviously stop working; hopefully, they'd
fix their headers and provide an exported way of doing this before
that happens.

(Note that code typed into a mail buffer doesn't always compile cleanly
or work the first time, but the code above is probably the right idea
modulo typos.)

You could also do (with the same disclaimer)
         __ifdef([SOLARIS])

SYS_lwp_private = 166   /* As per <sys/syscalls.h> */
_LWP_SETPRIVATE = 0     /* As per <sys/lwp.h> */
_LWP_GSBASE = 1         /* As per <sys/lwp.h> */

         .globl C(__cerror)

/* solarisx64_set_gsbase(void *arg) */
_exportfn(C(solarisx64_set_gsbase))
         __(movq %rdi,%rdx)   /* Make our arg the third arg to lwp_private */
         __(movq $_LWP_SETPRIVATE,%rdi) /* first arg to lwp_private */
         __(movq $_LWP_GSBASE,%rsi)     /* second arg to lwp_private */
         __(movl $SYS_lwp_private,%eax)
         __(syscall)
         __(jc C(__cerror))
         __(repret)
_endfn
         __endif



On Sun, 13 Jan 2008, Chris Curtis wrote:

> Well, it was encouraging for a while....
>
> It actually looks like the right function from <sys/lwp.h> would be 
> _lwp_private(_LWP_SETPRIVATE, _LWP_GSBASE, ...); the only problem is that 
> that function is only local in libc. (Then why bother definining it publicly 
> in a header, one might ask, but oh well.)
>
> So I don't know if there's a magic linker incantation to make it visible, 
> though it seems like one such would be a Bad Idea in principle. I'm not quite 
> fully defeated yet, but it might be time to start thinking about how we'd 
> work around not being able to set %gs or GSBASE.
>
> --chris
>
>
> On Jan 12, 2008, at 9:32 AM, Chris Curtis wrote:
>
>> Aha! Sounds like the right track; I've been looking along the GSBASE
>> line, and you beat me to it. ;-)
>> 
>> From my reading of the CCL source code, it looks like it's only
>> technically true that the thread manager uses %gs to dereference the
>> TCR... on both FreeBSD and Linux the system calls set the GSBASE MSR
>> (amd64_set_gsbase on FreeBSD), which is the hardware base offset for
>> %gs, which itself is then always zero.
>> 
>> (This seems to be true even though the linux call is
>> arch_prctl(ARCH_SET_GS,...) ... the manpage talks about setting the
>> base; the source code itself is fairly rococo.)
>> 
>> As far as the LWP/pthread correspondence goes... LWPs are the kernel
>> threads, and while originally they were a different pool that user
>> pthreads (and the non-POSIX "Solaris Threads") were scheduled onto,
>> since Solaris 9 they've been 1:1 with user threads, so there shouldn't
>> be anything we need to worry about there.
>> 
>> --chris
>> 
>> On Jan 12, 2008, at 2:49 AM, Gary Byers wrote:
>> 
>>> I poked around a little in the OpenSolaris sources.
>>> 
>>> I think that the incantation we want to use in the #ifdef SOLARIS
>>> case of setup_tcr_extra_segment() is something like:
>>> 
>>> #include <sys/lwp.h>
>>> #include <sys/proc.h>
>>> 
>>> _lwp_setprivate(lwp, _LWP_GSBASE, tcr);
>>> 
>>> where lwp is a "lightweight process" and there's some correspondence
>>> between an lwp and a pthread which would probably be clear to
>>> anyone who's read Solaris thread documentation recently.
>>> 
>>> (Or at least more recently than I have ...)
>>> 
>>> My best guess is that we want to try to arrange that the
>>> correspondence between pthreads and Solaris LWPs is 1:1.  I don't know
>>> if that's the default, but I assume that there's some way to arrange
>>> that 1:1 correspondence and that other things (reasonable behavior of
>>> alternate signal stacks ?)  may depend on that 1:1 mapping.
>> 
>> _______________________________________________
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
>> http://clozure.com/mailman/listinfo/openmcl-devel
>