Re: [ANNOUNCE] Btrfs: a copy on write, snapshotting FS

From: Chuck Lever
Date: Thu Jun 14 2007 - 14:21:54 EST

Hi Chris-

John Stoffel wrote:
As a user of Netapps, having quotas (if only for reporting purposes)
and some way to migrate non-used files to slower/cheaper storage would
be great.

Ie. being able to setup two pools, one being RAID6, the other being
RAID1, where all currently accessed files are in the RAID1 setup, but
if un-used get migrated to the RAID6 area.

And of course some way for efficient backups and more importantly
RESTORES of data which is segregated like this.

I like the way dump and restore was handled in AFS (and now ZFS and NetApp). There is a simple command to flatten a file system and send it to another system, which can receive it and re-expand it. The dump/restore process uses snapshots and can easily send incremental backups which are significantly smaller than 0-level. This is somewhat better than rsync, because you don't need checksums to discover what data has changed -- you already have the new data segregated into copied-on-write blocks.

NetApp happens to use the standard NDMP protocol for sending the flattened file system. NetApp uses it for synchronous replication, volume migration, and back up to nearline storage and tape. AFS used "vol dump" and "vol restore" for migration, replication, and back-up. ZFS has the "zfs send" and "zfs receive" commands that do basically the same (Eric Kustarz recently published a blog entry that described how these work). And of course, all file system objects are able to be sent this way: streams, xattrs, ACLs, and so on are all supported.

Note also that NFSv4 supports the idea of migrated or replicated file objects. All that is needed to support it is a mechanism on the servers to actually move the data. begin:vcard
fn:Chuck Lever
org:Oracle Corporation;Corporate Architecture: Linux Projects Group
adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA
email;internet:chuck dot lever at nospam oracle dot com
title:Principal Member of Staff
tel;work:+1 248 614 5091