[Openmcl-devel] RUN-PROGRAM and memory overcommit

[R. Matthew Emerson]

On Feb 7, 2014, at 8:53 AM, Stelian Ionescu <sionescu at cddr.org> wrote:

> On Thu, 2014-02-06 at 19:16 -0500, R. Matthew Emerson wrote:
>> On Feb 6, 2014, at 6:36 PM, Ron Garret <ron at flownet.com> wrote:
>>> 
>>> Before submitting a ticket I thought I would ask: is there a reason that run-program uses fork instead of vfork?
>> 
>> Speaking generally, vfork() and threads don't get along.  For that
>> matter, fork() and threads don't get along that well either. But in
>> the vfork() case they really don't get along.
> 
> The reason why fork and threads don't play is because upon fork, only
> the current thread is cloned and if you invoke libc functions that try
> to acquire locks that were held by some other thread you'll deadlock
> because, the other thread being gone, the lock won't get released ever.
> 
> That said, vfork() is explicitly limited(and optimized) so that you are
> only allowed to execute the following two syscalls afterwards: _exit(2)
> and execve(2) - not even the libc exit(3) - so it should be pretty safe
> with threads.

I thought that vfork() only suspended the calling thread (not all the threads in the calling process).  Am I wrong about that?

We can't just vfork() and then exec immediately, because we need to do some file descriptor manipulation first.

When we run an external program, we want to set up the stdout, stdin, and stderr file descriptors and ensure that all the rest are closed.  If other threads are still running, they can still be opening files, and that would make it impossible for us to set up the state of the file descriptors the way we want before we exec.  This is the same sort of problem that I'm worrying about with posix_spawn() in http://trac.clozure.com/ccl/ticket/687.