[Openmcl-devel] Managing multi-threaded access to list

[Joseph Oswald]

I have what might be a very general question about multi-processing.

I wish to use a list as a FIFO for multithreaded communication.

That is, I have one thread reading packets from the network, and when
suitable packets come in, that "producer" thread adds packets to the
list.

Other threads wish to consume packets off the list.

There are two possible "consumer" cases: "hang" or "no-hang",
corresponding to blocking until a packet appears on the list, or
returning nil immediately if no packet is available.

My current implementation currently looks like the following

(defparameter *packet-list-lock* (ccl::make-read-write-lock))
(defparameter *packet-list* nil)

(defun reading-packets ()
  "Function to read packets and putting them on *packet-list*"
  (loop
    (let ((pkt (read-packet-from-socket)))
     (when (acceptable-packet pkt)
        (ccl::with-write-lock (*packet-list-lock*)
          (setf *packet-list* (append *packet-list* (list pkt))))))))

(defun start-reader-process ()
   "Start a producer thread"
    (ccl:process-run-function "Network Reader Process" #'reading-packets))

(defun consume-packet-no-hang ()
  "Pick a packet off the packet list, or return nil if none is available"
  (ccl::with-write-lock (*packet-list-lock*)
    (pop *packet-list*)))

(defun data-available ()
  "Return T if there is a packet available, or NIL if not."
  (ccl::with-read-lock (*packet-list-lock*)
    (not (null *packet-list*))))

(defun consume-packet-hang ()
  "DON'T KNOW HOW THIS SHOULD WORK"
  (ccl::process-wait "waiting for packet" #'data-available)
;; OK, there was a packet, but now am I racing against other consumers???
  (consume-packet-no-hang)))

Here's my dilemma: how do I block until the list is non-empty, *and*
be sure to get the first element on the list when I return? It seems
to me that between the test and the pop my thread doesn't have the
lock; two threads could both be blocking for a packet to arrive, one
packet arrives, both threads see "data is now available", but only one
thread will get a packet, and the other one will get nil. Should I
simply loop until I actually am lucky enough to get the packet?

It feels to me that I am missing a basic building block, such as a
call looking like process-wait which could atomically test and return
the first element on the list if there is one, or keep waiting if
there isn't, but I don't know if there is a fundamental problem with
the scheduler trying to get the lock, or what I am looking for has a
different name.