[Openmcl-devel] Is setf atomic?

[Gary Byers]

On Tue, 6 Nov 2007, Pascal Costanza wrote:

> So to get this straight - are you saying that even a setf on a simple 
> variable, for example like (setf var 42) when var is not a symbol macro, is 
> potentially not thread-safe?

Yes, if "var" (let's call it *VAR*) is a special variable not bound
dynamically by any of the threads doing SETF on it, then things like

(progn
   (setf *var* 42)
   (= *var* 42))

might return false, and something like:

(loop
   (let* ((old-var *var*))
    (incf *var*)
    (unless (= *var* (1+ old-var))
      (beep))))

might beep.

Suppose that we were living in some imaginary world that satisfied
some nebulous, poorly-considered notion of atomicity.  (Many people
come up with nebulous, poorly-considered notions when they first
start to think about this.)  In that imaginary world, INCF locks
the bus, suspends the execution of all other threads, and otherwise
guarantees that an atomic read-modify-write cycle takes place.

Great, but as soon as the INCF finishes, we can't trust that the
value of *VAR* has anything to do with the value that INCF modified.

What we need to do is impose some conventions that define what
atomic access and modification of a shared resource (like the
value of *VAR*) mean; let's call the mechanism used to enforce
those conventions LOCKS and the conventions themselves a locking
protocol -- wait, that was all invented 50 or more years ago.
We haven't discovered anything new, and we should have been paying
more attention in CS200.

It's -sometimes- possible to do something that in some sense is
atomic/thread-safe without using locks.  (I just wrote something
this morning called CCL::ATOMIC-POP-UVECTOR-CELL; as the (obscure)
name implies, that's not very general but it does do a certain
flavor of POP atomically and avoids locking overhead; there are
a dozen or so ad hoc primitives scattered around the lisp, many
of them there because they're used by the code that actually
implements locks.)

There is a macro - CCL::CONDITIONAL-STORE - that mostly exists for
CLX's benefit and only knows how to recognize a few kinds of PLACE
forms and could be taught more (car and cdr being obvious candidates),
and it might be worth dressing this up more and documenting it so
that it could be used as a building-block for other things (like
the lock-free queues that Daniel wanted to implement.)  I'm all
in favor of this; it'd be good to not have to write ad hoc things
like CCL::ATOMIC-POP-UVECTOR-CELL if a macro or two could write
it for me, and real-world code needs to be able to do stuff like
this.

I'm not too enthused about the idea of falling down a rabbit hole,
deciding that everything's global and everything's shared and that
the only solution is to put a big lock around everything (though
I sometimes wonder whether or not it makes sense to put a big
lock around CLOS).

There is a reasonable argument that says that if parallelism is Good
and modification of global state (SETF) requires synchronization
(locks, test-and-set, store-conditional, atomic-swap) which inhibit
parallelism to some degree is Bad.  That's probably true (though
not entirely relevant for CL), and it suggests that avoiding
resource contention is a good thing (I have a feeling that that
was also covered in CS200 ...).  There's a big difference between:

(defun mangle-pathname (path)
   (setq *default-pathname-defaults* ...)
   ...)

and

(defun mangle-pathname-more-safely (path)
   (let* ((*default-pathname-defaults* ...))
     ...))

in that the former (likely) modifies some shared global state
and the latter creates a thread-private binding.


>
> Pascal
>
> On 6 Nov 2007, at 18:42, Gary Byers wrote:
>
>> If you're asking whether global variables and property lists are
>> potentially shared resources just like everything else or whether
>> they have some sort of magic dispensation ... well, there isn't
>> any magic dispensation.
>> 
>> There are certainly lots of low-level internal things in OpenMCL that
>> do atomic read-modify-write operations that're guaranteed (by the
>> hardware) to either succeed or detectably fail.  It's generally hard
>> to map things like SETF (whose specified behavior is about as far from
>> "atomic update" as one could imagine) onto these primitives, and it's
>> not clear that that'd be desirable: such an atomic operation is likely
>> to be an order of magnitude slower than a non-atomic operation in terms
>> of direct costs, and may have all kinds of indirect costs (serialization
>> on an machine that does out-of-order memory operations and/or execution.)
>> 
>> 
>> On Tue, 6 Nov 2007, Pascal Costanza wrote:
>> 
>>> Hi,
>>> 
>>> Are a setf on a simple variable, and a setf on a property list (i.e.,
>>> a (setf getf)), thread-safe (atomic) operations in OpenMCL?
>>> 
>>> 
>>> Thanks,
>>> Pascal
>>> 
>>> -- 
>>> Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
>>> Vrije Universiteit Brussel, Programming Technology Lab
>>> Pleinlaan 2, B-1050 Brussel, Belgium
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Openmcl-devel mailing list
>>> Openmcl-devel at clozure.com
>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>> 
>>> 
>
> -- 
> Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
> Vrije Universiteit Brussel, Programming Technology Lab
> Pleinlaan 2, B-1050 Brussel, Belgium
>
>
>
>