Re: [GIT PULL] Btrfs pull request

[Andrea Gelmini]

2011/11/6 Chris Mason <chris.mason@xxxxxxxxxx>:

Hi Chris,
   and thanks a lot for your work.

> Arne Jansen and Jan Schmidt have improved the scrubber and provided
> utilities to walk btrfs' many backrefs. ÂThe scrubber is much faster
> thanks to extensive btree readahead and instead of just telling you a
> specific block is bad, it tells you which btree or which file was
> impacted by that bad block.

   Using your for-linus branch, on latest Linus' git tree, with latest
git tools,
   I've got this:

root@Q45:/home/gelma/dev/prg/btrfs# ./btrfs scrub start -Br /dev/md126
ERROR: scrubbing /dev/md126 failed for device id 1 (Cannot allocate memory)
scrub canceled for 11827b37-1ba0-4b3e-883d-2746987724ca
	scrub started at Sun Nov  6 20:23:46 2011 and was aborted after 0 seconds
	total bytes scrubbed: 0.00 with 0 errors
root@Q45:/home/gelma/dev/prg/btrfs# ./btrfs scrub start -Br /home/
ERROR: scrubbing /home/ failed for device id 1 (Cannot allocate memory)
scrub canceled for 11827b37-1ba0-4b3e-883d-2746987724ca
	scrub started at Sun Nov  6 20:25:01 2011 and was aborted after 0 seconds
	total bytes scrubbed: 0.00 with 0 errors

Thanks a lot for your time,
Andrea
>
> There are also progs updates to give you the same backref walking from
> the command line.
>
> SUSE and Fujitsu have a nice set of error handling fixes, and Li Zefan
> also closed out some problems in the mount -o autodefrag mode.
>
> I kicked in an array of backup tree roots. ÂIf a given mount fails to go
> through because a tree root is bad, you can mount -o recovery and it'll
> walk through the array and try older versions of the FS.
>
> I also spent a lot of time refining Fujitsu's log tree improvements.
> This code has been around for quite a while, and I really wanted to get
> it in this time. ÂBut yesterday I hit corruptions when I mixed heavy
> fsyncs with heavy snapshotting, and I wasn't able to fix things in time
> for this merge window.
>
> Josef Bacik (60) commits (+1847/-1238):
> Â ÂBtrfs: don't check bytes_pinned to determine if we should commit the transaction (+0/-11)
> Â ÂBtrfs: allow callers to specify if flushing can occur for btrfs_block_rsv_check (+10/-10)
> Â ÂBtrfs: be smarter about committing the transaction in reserve_metadata_bytes (+67/-19)
> Â ÂBtrfs: release metadata from global reserve if we have to fallback for unlink (+4/-1)
> Â ÂBtrfs: wait for ordered extents if we're in trouble when shrinking delalloc (+17/-8)
> Â ÂBtrfs: add a io_ctl struct and helpers for dealing with the space cache (+375/-318)
> Â ÂBtrfs: seperate out btrfs_block_rsv_check out into 2 different functions (+37/-24)
> Â ÂBtrfs: check the return value of filemap_write_and_wait in the space cache (+5/-2)
> Â ÂBtrfs: don't increase the block_rsv's size when emergency allocating space (+0/-3)
> Â ÂBtrfs: use the global reserve when truncating the free space cache inode (+17/-5)
> Â ÂBtrfs: make sure to unset trans->block_rsv before running delayed refs (+11/-0)
> Â ÂBtrfs: only reserve space in fallocate if we have to do a preallocate (+16/-6)
> Â ÂBtrfs: stop passing a trans handle all around the reservation code (+39/-43)
> Â ÂBtrfs: skip looking for delalloc if we don't have ->fill_delalloc (+5/-1)
> Â ÂBtrfs: if we have a lot of pinned space, commit the transaction (+15/-0)
> Â ÂBtrfs: release trans metadata bytes before flushing delayed refs (+3/-8)
> Â ÂBtrfs: make a delayed_block_rsv for the delayed item insertion (+13/-7)
> Â ÂBtrfs: ratelimit the generation printk for the free space cache (+7/-5)
> Â ÂBtrfs: use the global reserve as a backup for deleting inodes (+11/-1)
> Â ÂBtrfs: allow shrink_delalloc flush the needed reclaimed pages (+3/-2)
> Â ÂBtrfs: only inherit btrfs specific flags when creating files (+11/-6)
> Â ÂBtrfs: break out of orphan cleanup if we can't make progress (+11/-0)
> Â ÂBtrfs: move stuff around in btrfs_inode to get better packing (+3/-3)
> Â ÂBtrfs: reserve some space for an orphan item when unlinking (+8/-1)
> Â ÂBtrfs: wait for ordered extents if we didn't reclaim enough (+1/-1)
> Â ÂBtrfs: check unused against how much space we actually want (+1/-1)
> Â ÂBtrfs: put the block group cache after we commit the super (+3/-3)
> Â ÂBtrfs: inline checksums into the disk free space cache (+172/-68)
> Â ÂBtrfs: use the inode's mapping mask for allocating pages (+18/-6)
> Â ÂBtrfs: fix space leak when we fail to make an allocation (+14/-6)
> Â ÂBtrfs: don't skip writing out a empty block groups cache (+6/-4)
> Â ÂBtrfs: use the transactions block_rsv for the csum root (+10/-6)
> Â ÂBtrfs: fix call to btrfs_search_slot in free space cache (+1/-1)
> Â ÂBtrfs: use d_obtain_alias when mounting subvol/subvolid (+1/-24)
> Â ÂBtrfs: allow us to overcommit our enospc reservations (+88/-18)
> Â ÂBtrfs: don't get the block_rsv in btrfs_free_tree_block (+0/-4)
> Â ÂBtrfs: handle enospc accounting for free space inodes (+47/-23)
> Â ÂBtrfs: reduce the amount of space needed for truncates (+15/-4)
> Â ÂBtrfs: kill the orphan space calculation for snapshots (+0/-90)
> Â ÂBtrfs: use bytes_may_use for all ENOSPC reservations (+112/-82)
> Â ÂBtrfs: optimize how we account for space in truncate (+29/-29)
> Â ÂBtrfs: fix how we reserve space for deleting inodes (+38/-11)
> Â ÂBtrfs: don't flush the cache inode before writing it (+0/-4)
> Â ÂBtrfs: take overflow into account in reserving space (+1/-1)
> Â ÂBtrfs: don't try to commit in btrfs_block_rsv_check (+4/-25)
> Â ÂBtrfs: fix the amount of space reserved for unlink (+10/-1)
> Â ÂBtrfs: fix regression in re-setting a large xattr (+11/-0)
> Â ÂBtrfs: introduce mount option no_space_cache (+22/-10)
> Â ÂBtrfs: delay iput when deleting a block group (+1/-1)
> Â ÂBtrfs: kill btrfs_truncate_reserve_metadata (+0/-34)
> Â ÂBtrfs: fix how we mount subvol=<whatever> (+135/-64)
> Â ÂBtrfs: calculate checksum space correctly (+118/-8)
> Â ÂBtrfs: kill the durable block rsv stuff (+17/-101)
> Â ÂBtrfs: fix delayed insertion reservation (+49/-8)
> Â ÂBtrfs: fix orphan cleanup regression (+17/-19)
> Â ÂBtrfs: kill unused parts of block_rsv (+6/-22)
> Â ÂBtrfs: introduce convert_extent_bit (+190/-0)
> Â ÂBtrfs: set truncate block rsv's size (+2/-0)
> Â ÂBtrfs: kill reserved_bytes in inode (+0/-8)
> Â ÂBtrfs: stop using write_one_page (+20/-67)
>
> Jan Schmidt (13) commits (+1954/-303):
> Â Âbtrfs: new ioctls to do logical->inode and inode->path resolving (+162/-0)
> Â Âbtrfs scrub: add fixup code for errors on nodatasum files (+183/-6)
> Â Âbtrfs: integrating raid-repair and scrub-fixup-nodatasum (+67/-25)
> Â Âbtrfs: Moved repair code from inode.c to extent_io.c (+393/-159)
> Â Âbtrfs: added helper functions to iterate backrefs (+851/-1)
> Â Âbtrfs: btrfs_multi_bio replaced with btrfs_bio (+90/-78)
> Â Âbtrfs: Do not use bio->bi_bdev after submission (+1/-1)
> Â Âbtrfs: add mirror_num to extent_read_full_page (+6/-6)
> Â Âbtrfs scrub: print paths of corrupted files (+163/-6)
> Â Âbtrfs scrub: use int for mirror_num, not u64 (+4/-4)
> Â Âbtrfs scrub: bugfix: mirror_num off by one (+6/-6)
> Â Âbtrfs scrub: added unverified_errors (+26/-11)
> Â Âbtrfs: Put mirror_num in bi_bdev (+2/-0)
>
> Chris Mason (10) commits (+490/-67):
> Â ÂBtrfs: make sure to flush queued bios if write_cache_pages waits (+22/-10)
> Â ÂBtrfs: don't wait as long for more batches during SSD log commit (+2/-2)
> Â ÂBtrfs: fix extent_buffer leak in the metadata IO error handling (+1/-0)
> Â ÂBtrfs: ClearPageError during writepage and clean_tree_block (+10/-1)
> Â ÂBtrfs: make sure btrfs_remove_free_space doesn't leak EAGAIN (+3/-1)
> Â ÂBtrfs: fix the new inspection ioctls for 32 bit compat (+15/-16)
> Â ÂBtrfs: stop the readahead threads on failed mount (+1/-0)
> Â ÂBtrfs: fix extent pinning bugs in the tree log (+59/-8)
> Â ÂBtrfs: fix race during transaction joins (+8/-5)
> Â ÂBtrfs: add a log of past tree roots (+369/-24)
>
> Li Zefan (7) commits (+47/-24):
> Â ÂBtrfs: fix wrong max_to_defrag in btrfs_defrag_file() (+1/-1)
> Â ÂBtrfs: honor extent thresh during defragmentation (+26/-11)
> Â ÂBtrfs: remove BUG_ON() in compress_file_range() (+5/-1)
> Â ÂBtrfs: use i_size_read() in btrfs_defrag_file() (+4/-3)
> Â ÂBtrfs: fix defragmentation regression (+4/-2)
> Â ÂBtrfs: fix direct-io vs nodatacow (+1/-2)
> Â ÂBtrfs: fix array bound checking (+6/-4)
>
> Ilya Dryomov (6) commits (+22/-21):
> Â ÂBtrfs: pass the correct root to lookup_free_space_inode() (+1/-1)
> Â ÂBtrfs: rename btrfs_bio multi -> bbio for consistency (+15/-15)
> Â ÂBtrfs: fix a potential btrfs_bio leak on scrub fixups (+1/-0)
> Â ÂBtrfs: stop leaking btrfs_bios on readahead (+2/-0)
> Â ÂBtrfs: fix a bug when opening seed devices (+1/-1)
> Â ÂBtrfs: close all bdevs on mount failure (+2/-4)
>
> Arne Jansen (6) commits (+1130/-70):
> Â Âbtrfs: add an extra wait mode to read_extent_buffer_pages (+9/-6)
> Â Âbtrfs: initial readahead code and prototypes (+967/-1)
> Â Âbtrfs: add READAHEAD extent buffer flag (+35/-0)
> Â Âbtrfs: state information for readahead (+31/-0)
> Â Âbtrfs: use readahead API for scrub (+50/-62)
> Â Âbtrfs: hooks for readahead (+38/-1)
>
> David Sterba (3) commits (+105/-77):
> Â Âbtrfs: do not allow mounting non-subvolumes via subvol option (+19/-0)
> Â Âbtrfs: separate superblock items out of fs_info (+78/-76)
> Â Âbtrfs: ratelimit WARN_ON in use_block_rsv (+8/-1)
>
> Zheng Yan (1) commits (+4/-1):
> Â Âbtrfs: check file extent backref offset underflow
>
> Lukas Czerner (1) commits (+5/-1):
> Â Âbtrfs: return EINVAL if start > total_bytes in fitrim ioctl
>
> Daniel J Blueman (1) commits (+6/-2):
> Â Âbtrfs: fix oops on failure path
>
> Liu Bo (1) commits (+1/-1):
> Â ÂBtrfs: do not set EXTENT_DIRTY along with EXTENT_DELALLOC
>
> Diego Calleja (1) commits (+1/-3):
> Â Âbtrfs: fix memory leak in btrfs_defrag_file
>
> Jeff Liu (1) commits (+8/-2):
> Â Âbtrfs: trivial fix, a potential memory leak in btrfs_parse_early_options()
>
> Miao Xie (1) commits (+7/-0):
> Â ÂBtrfs: fix race between multi-task space allocation and caching space
>
> Tsutomu Itoh (1) commits (+7/-10):
> Â ÂBtrfs: fix return value of btrfs_get_acl()
>
> Total: (113) commits
>
> Âfs/btrfs/Makefile      |  Â3 +-
> Âfs/btrfs/acl.c       Â|  17 +-
> Âfs/btrfs/backref.c     Â| Â776 +++++++++++++++++++++++++++++++++++
> Âfs/btrfs/backref.h     Â|  62 +++
> Âfs/btrfs/btrfs_inode.h   Â|  17 +-
> Âfs/btrfs/compression.c   Â|  Â3 +-
> Âfs/btrfs/ctree.c      Â|  10 +-
> Âfs/btrfs/ctree.h      Â| Â198 ++++++++--
> Âfs/btrfs/delayed-inode.c  Â|  50 ++-
> Âfs/btrfs/disk-io.c     Â| Â434 +++++++++++++++++---
> Âfs/btrfs/disk-io.h     Â|  Â4 +-
> Âfs/btrfs/extent-tree.c   Â| Â848 +++++++++++++++++++++++---------------
> Âfs/btrfs/extent_io.c    Â| Â614 +++++++++++++++++++++++++++-
> Âfs/btrfs/extent_io.h    Â|  23 +-
> Âfs/btrfs/file-item.c    Â|  17 +-
> Âfs/btrfs/file.c       |  25 +-
> Âfs/btrfs/free-space-cache.c | Â926 +++++++++++++++++++++++++----------------
> Âfs/btrfs/inode-map.c    Â|  Â6 +-
> Âfs/btrfs/inode.c      Â| Â457 +++++++--------------
> Âfs/btrfs/ioctl.c      Â| Â227 +++++++++--
> Âfs/btrfs/ioctl.h      Â|  29 ++
> Âfs/btrfs/print-tree.c    |  Â8 +-
> Âfs/btrfs/reada.c      Â| Â951 +++++++++++++++++++++++++++++++++++++++++++
> Âfs/btrfs/relocation.c    |  24 +-
> Âfs/btrfs/scrub.c      Â| Â591 ++++++++++++++++++++++-----
> Âfs/btrfs/super.c      Â| Â298 +++++++++-----
> Âfs/btrfs/transaction.c   Â| Â146 +++----
> Âfs/btrfs/tree-log.c     |  19 +-
> Âfs/btrfs/volumes.c     Â| Â207 ++++++----
> Âfs/btrfs/volumes.h     Â|  18 +-
> Âfs/btrfs/xattr.c      Â|  11 +
> Â31 files changed, 5416 insertions(+), 1603 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at Âhttp://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/