[Openmcl-devel] ccl on ext4

[Tim Bradshaw]

On 12 Jul 2011, at 14:10, Gary Byers wrote:

> The Wikipedia article that Ron cites notes that a delayed allocation
> policy (as offered by ext4, ZFS, HFS+, ...) can increase the chance
> that on-disk data will be inconsistent in the event of a power loss
> (or that the inconsistencies will be more severe.)  I can't think of
> a filesystem that really guarantees disk consistency after #_write;
> there are typically all kinds of buffering and caching mechanisms
> involved.  (If disk consistency is important, you have to call #_fsync
> to defeat those mechanisms; FINISH-OUTPUT calls #_fsync in CCL.)

Based on some POSIX spec-reading (so only for Unixoid systems I suppose), I think that is correct: I think the two points where you know what is on disk are after successful return of fsync or close (and a very common problem is not checking that close has returned successfully).  There may be platform-specific options which make things more aggressive than that (for instance a lot of filesystems offer "direct IO" options for use with applications which want to manage their own caching and those incidentally can mean that when writes return data is (probably) on disk since there's no OS caching in the way).

To bring this back to CL, it's slightly annoying to me that FINISH-OUTPUT &c, as well as CLOSE seem to offer no standard way of detecting failure.  Granted there's often not much you can do on failures, but being able to at least say "something really bad happened" would be useful.  May be I am misreading the spec though.

> 
> The person who asked that question seemed to be satisfied by Matt's
> answer, so the fact that I still can't guess what the question means
> is probably not of general interest.

I think what it means is that a large number of people write to the particular characteristics of the filesystem they are using, rather than to the standard it purports to implement.  They then have bad experiences when their programs don't work very well when those characteristics change. The traditional example of that is running things over NFS on systems with lots of memory.

--tim