[Openmcl-devel] File-length bug?

[Ron Garret]

On Aug 3, 2013, at 3:23 PM, Pascal J. Bourguignon wrote:

> Ron Garret <ron at flownet.com> writes:
> 
>> ? (setf path #P"~/Desktop/foo")
>> #P"/Users/ron/Desktop/foo"
>> ? (setf out (open path :direction :output :if-does-not-exist :create))
>> #<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("/Users/ron/Desktop/foo"/20 UTF-8) #x3020025DEA2D>
>> ? (setf in (open path :direction :input))
>> #<BASIC-FILE-CHARACTER-INPUT-STREAM ("/Users/ron/Desktop/foo"/21 UTF-8) #x3020025DD7DD>
>> ? (file-length in)
>> 0                              ; As expected
>> ? (listen in)
>> T                              ; Not as expected, but OK, using listen on file streams is a little hinky...
>> ? (print 123 out)
>> 123
>> ? (force-output out)
>> NIL
>> ? (file-length out)
>> 5                               ; As expected
>> ? (file-length in)
>> 0                               ; Not as expected
>> ? (file-position in 3)
>> 3
>> ? (file-position in)
>> 3
>> ? (file-length in)
>> 0
>> 
>> Is this a bug?
>> 
>> FYI, what I'm actually trying to do is to use a file as a FIFO, with
>> one process writing the file and another reading it (which is why I
>> can't use an :io stream).  Ideally what I'd like to do is have the
>> reader block when trying to read past the end of the file (like "tail
>> -f"), but a reliable way of polling to see if additional data has been
>> written would be enough.  It's a binary file, so I can't use
>> peek-char.  (I could use flexi-streams:peek-byte but I was hoping to
>> avoid adding a library dependency.)
> 
> I don't think you can do that conformingly.  Ie. ccl is not to be blamed
> for behaving like it does here.  (But this doesn't prevent it or any other
> implementation on POSIX system to get closer to the POSIX semantics).
> 
> Note that the default if-exist is :new-version, and you used
> :if-does-not-exist :create, therefore in both cases, you are writing a
> new file, and since it's possible to call (close stream :abort t) later,
> to undo the creation or new version, or  (close stream :abort nil) to
> commit the file system "transaction", an implementation even on POSIX
> system could (I'd dare say even should) set things up in such a way that
> before closing the created file or new version, opening it for reading
> should access the previous version (or signal a file does not exist
> error if it is to be created when (close :abort nil) is called).

It is easy to establish that this is not the problem:

? (setf path #P"~/Desktop/foo")
#P"/Users/ron/Desktop/foo"
? (setf out (open path :direction :output :if-does-not-exist :create))
#<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("/Users/ron/Desktop/foo"/12 UTF-8) #x30200125CEFD>
? (close out)
T
? (setf out (open path :direction :output :if-exists :append))
#<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("/Users/ron/Desktop/foo"/12 UTF-8) #x30200113DE0D>
? (setf in (open path :direction :input))
#<BASIC-FILE-CHARACTER-INPUT-STREAM ("/Users/ron/Desktop/foo"/13 UTF-8) #x30200113C52D>
? (file-length in)
0
? (print 123 out)
123
? (force-output out)
NIL
? (read in)
123
? (file-position in)
5
? (file-length in)
0
? (file-length out)
5
? (close out)
T
? (file-length in)
0
? (file-position in 20000)
20000

Leaving aside the question of whether or not it is possible to tail a file in portable CL (I don't actually care about portability, which is why I'm asking this question here rather than on CLL), it seems to me that (> (file-position f) (file-length f)) should never be true.

Just for the record, SBCL does this:

* (file-length in)
5
* (print 123 out)
123
* (force-output out)
NIL
* (file-length out)
10
* (file-length in)
10

CLisp does this:

[2]> (setf in (open "/Users/ron/Desktop/foo"))
** - Continuable Error
OPEN: #<OUTPUT BUFFERED FILE-STREAM CHARACTER #P"/Users/ron/Desktop/foo">
     already points to file "/Users/ron/Desktop/foo", opening the file again
     for :INPUT may produce unexpected results
Break 1 [5]> continue
#<INPUT BUFFERED FILE-STREAM CHARACTER #P"/Users/ron/Desktop/foo" @1>
[6]> (file-length in)
10
[7]> (print 123 out)
123
[8]> (force-output out)
NIL
[9]> (file-length in)
15

rg