Re: [PATCH 0/9] overlay filesystem: request for inclusion (v17)

[Erez Zadok]

I tend to agree with Al's and Linus's POV regarding whiteouts. There are three general techniques to implementing whiteouts:

1. namespace: special file names, hard/symlinks, or special "hidden" dot files.
2. extended attributes.
3. DT_WHT dirent flags.
(there's actually a 4th method I've tried before that I won't discuss below: implementing your own data structures on a raw partition — way too cumbersome.)

The namespace techniques require lower file systems to support hard/symlinks and sometimes need long names. A plus: they work on most file systems (but not all).  But they cause all sorts of namespace ugliness, where you have to hide the special file names, avoid them, ensure atomic updates for ops that involve whiteouts.  It's all doable, as had been demonstrated by several implementations.  But it's still icky in terms of namespace pollution.

The extended attributes technique, I think, is better than the namespace one in that you don't pollute the namespace; plus, I think the EA technique minimizes atomicity issues that show up with the namespace method.  Yet, it still requires EA support in lower file systems, so it won't work unless lower file systems support xattr ops.  Plus it could fail for file systems that have limited xattr support (e.g., number of EAs per inode).

The DT_WHT technique is the cleanest in the long run, and the best of the three IMHO.  It's well understood and has been done in BSD a long time ago.  It doesn't have the namespace pollution as seen in technique #1 above.  And I believe it also minimize atomicity issues.  Plus you won't have issues running out of EAs.

A while back I've looked at the unionmount code for DT_WHT support for ext2/tmpfs, and it was small, clean, and mostly additive.  I even had a prototype port of unionfs using unionmount's DT_WHT support: it was relatively easy to port unionfs to use DT_WHT instead of namespace techniques. Plus, I was able to reduce the amount of code devoted to whiteout support by quite a bit.

So I think it'll be easy to port overlayfs to use DT_WHT.  Given that most people use unioning with ext* and tmpfs, minimal DT_WHT support in those would get most users happy initially. And we can then let other file systems support DT_WHT on their own, in whatever way they deem suitable (as Al suggested, this is really best deferred to the F/S to implement).

Lastly, what I'm not sure is what API to use for whiteouts: should every f/s implement some new methods to add/remove/query a whiteout, or should the upper f/s and VFS directly check DT_WHT flags with S_ISWHT.  The generic f/s methods may allow file systems to implement whiteouts in arbitrary ways, not necessarily as a dirent flag.

Cheers,
Erez.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/