[PATCH 00/26] ceph distributed file system client
From: Sage Weil
Date: Tue Mar 02 2010 - 19:53:43 EST
Hi everyone,
This is the latest patchset for the Ceph distributed file system client.
I've broken it down into individual files for ease of review.
Alternatively, the ceph-client.git tree includes a full history (from
mid-October). A shortlog for changes since the last time I sent this
out (during the 2.6.33 merge window) at the end of this email.
We've been hammering on this steadily for the past few weeks and are
pretty happy with the stability at this point. We would definitely
benefit from the broader testing that would come from inclusion in
2.6.34. If there are no major complaints, I will send this to Linus
again shortly.
Questions, comments, review, etc. are all welcome. :)
Thanks-
sage
Git tree:
git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus
Server-side git:
git://ceph.newdream.net/ceph.git
---
Documentation/filesystems/ceph.txt | 139 ++
Documentation/ioctl/ioctl-number.txt | 1 +
MAINTAINERS | 9 +
fs/Kconfig | 1 +
fs/Makefile | 1 +
fs/ceph/Kconfig | 27 +
fs/ceph/Makefile | 39 +
fs/ceph/README | 20 +
fs/ceph/addr.c | 1188 +++++++++++++
fs/ceph/armor.c | 99 ++
fs/ceph/auth.c | 257 +++
fs/ceph/auth.h | 84 +
fs/ceph/auth_none.c | 121 ++
fs/ceph/auth_none.h | 28 +
fs/ceph/auth_x.c | 656 ++++++++
fs/ceph/auth_x.h | 49 +
fs/ceph/auth_x_protocol.h | 90 +
fs/ceph/buffer.c | 78 +
fs/ceph/buffer.h | 39 +
fs/ceph/caps.c | 2925 ++++++++++++++++++++++++++++++++
fs/ceph/ceph_debug.h | 37 +
fs/ceph/ceph_frag.c | 21 +
fs/ceph/ceph_frag.h | 109 ++
fs/ceph/ceph_fs.c | 74 +
fs/ceph/ceph_fs.h | 651 ++++++++
fs/ceph/ceph_hash.c | 118 ++
fs/ceph/ceph_hash.h | 13 +
fs/ceph/ceph_strings.c | 176 ++
fs/ceph/crush/crush.c | 151 ++
fs/ceph/crush/crush.h | 180 ++
fs/ceph/crush/hash.c | 149 ++
fs/ceph/crush/hash.h | 17 +
fs/ceph/crush/mapper.c | 596 +++++++
fs/ceph/crush/mapper.h | 20 +
fs/ceph/crypto.c | 408 +++++
fs/ceph/crypto.h | 48 +
fs/ceph/debugfs.c | 483 ++++++
fs/ceph/decode.h | 194 +++
fs/ceph/dir.c | 1220 ++++++++++++++
fs/ceph/export.c | 223 +++
fs/ceph/file.c | 937 +++++++++++
fs/ceph/inode.c | 1750 ++++++++++++++++++++
fs/ceph/ioctl.c | 160 ++
fs/ceph/ioctl.h | 40 +
fs/ceph/mds_client.c | 3021 ++++++++++++++++++++++++++++++++++
fs/ceph/mds_client.h | 335 ++++
fs/ceph/mdsmap.c | 174 ++
fs/ceph/mdsmap.h | 54 +
fs/ceph/messenger.c | 2240 +++++++++++++++++++++++++
fs/ceph/messenger.h | 254 +++
fs/ceph/mon_client.c | 834 ++++++++++
fs/ceph/mon_client.h | 119 ++
fs/ceph/msgpool.c | 186 +++
fs/ceph/msgpool.h | 27 +
fs/ceph/msgr.h | 158 ++
fs/ceph/osd_client.c | 1537 +++++++++++++++++
fs/ceph/osd_client.h | 166 ++
fs/ceph/osdmap.c | 1019 ++++++++++++
fs/ceph/osdmap.h | 125 ++
fs/ceph/pagelist.c | 54 +
fs/ceph/pagelist.h | 54 +
fs/ceph/rados.h | 374 +++++
fs/ceph/snap.c | 904 ++++++++++
fs/ceph/super.c | 1030 ++++++++++++
fs/ceph/super.h | 900 ++++++++++
fs/ceph/types.h | 29 +
fs/ceph/xattr.c | 844 ++++++++++
67 files changed, 28064 insertions(+), 0 deletions(-)
Below is a shortlog of changes since v0.18, the last patchset sent out
to LKML/fsdevel for review. The main items are support for client and
server authentication (auth_x), some support for easing future protocol
changes, and a variety of bug fixes. The full history is available at
http://ceph.newdream.net/git/?p=ceph-client.git;a=shortlog;h=unstable
fs/ceph/Kconfig | 1 +
fs/ceph/Makefile | 4 +-
fs/ceph/addr.c | 153 ++++++++---
fs/ceph/armor.c | 99 +++++++
fs/ceph/auth.c | 70 ++++--
fs/ceph/auth.h | 7 +
fs/ceph/auth_none.c | 1 +
fs/ceph/auth_x.c | 656 +++++++++++++++++++++++++++++++++++++++++++++
fs/ceph/auth_x.h | 49 ++++
fs/ceph/auth_x_protocol.h | 90 ++++++
fs/ceph/buffer.c | 54 ++++-
fs/ceph/buffer.h | 30 +--
fs/ceph/caps.c | 310 +++++++++++++---------
fs/ceph/ceph_fs.h | 23 +-
fs/ceph/crypto.c | 408 ++++++++++++++++++++++++++++
fs/ceph/crypto.h | 48 ++++
fs/ceph/debugfs.c | 93 +++++--
fs/ceph/decode.h | 35 +++
fs/ceph/dir.c | 14 +-
fs/ceph/file.c | 95 +++++---
fs/ceph/inode.c | 188 +++++++++++---
fs/ceph/mds_client.c | 607 ++++++++++++++++++++++-------------------
fs/ceph/mds_client.h | 20 +-
fs/ceph/mdsmap.c | 4 +
fs/ceph/messenger.c | 507 ++++++++++++++++++++++-------------
fs/ceph/messenger.h | 37 ++--
fs/ceph/mon_client.c | 173 +++++++++---
fs/ceph/mon_client.h | 8 +-
fs/ceph/msgpool.c | 9 +-
fs/ceph/msgr.h | 23 +--
fs/ceph/osd_client.c | 525 ++++++++++++++++++++++++------------
fs/ceph/osd_client.h | 30 ++-
fs/ceph/osdmap.c | 223 +++++++++++----
fs/ceph/osdmap.h | 7 +-
fs/ceph/pagelist.c | 54 ++++
fs/ceph/pagelist.h | 54 ++++
fs/ceph/rados.h | 16 +-
fs/ceph/snap.c | 69 +++--
fs/ceph/super.c | 76 +++++-
fs/ceph/super.h | 47 ++--
fs/ceph/xattr.c | 8 +-
41 files changed, 3731 insertions(+), 1194 deletions(-)
Alexander Beregalov (1):
ceph: move dereference after NULL test
Julia Lawall (1):
ceph: remove duplicate variable initialization
Sage Weil (80):
ceph: use kref for ceph_buffer
ceph: simplify ceph_buffer interface
ceph: use kref for struct ceph_mds_request
ceph: use kref for ceph_osd_request
ceph: use kref for ceph_msg
ceph: do not feed bad device ids to crush
ceph: fix leak of monc mutex
ceph: carry explicit msg reference for currently sending message
ceph: plug msg leak in con_fault
ceph: detect lossy state of connection
ceph: don't save sent messages on lossy connections
ceph: hex dump corrupt server data to KERN_DEBUG
ceph: plug leak of incoming message during connection fault/close
ceph: make mds ops interruptible
ceph: include link to bdi in debugfs
ceph: ensure rename target dentry fails revalidation
ceph: do not drop lease during revalidate
ceph: fix error paths for corrupt osdmap messages
ceph: fix incremental osdmap pg_temp decoding bug
ceph: do not touch_caps while iterating over caps list
ceph: only unregister registered bdi
ceph: unregister canceled/timed out osd requests
ceph: use connection mutex to protect read and write stages
ceph: control access to page vector for incoming data
ceph: more informative msgpool errors
ceph: include transaction id in ceph_msg_header (protocol change)
ceph: add feature bits to connection handshake (protocol change)
ceph: support ceph_pagelist for message payload
ceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change)
ceph: remove unused erank field
ceph: display pgid in debugfs osd request dump
ceph: mark MDS CREATE as a write op
ceph: properly handle aborted mds requests
ceph: precede encoded ceph_pg_pool struct with version
ceph: include type in ceph_entity_addr, filepath
ceph: release all pages after successful osd write response
ceph: buffer decoding helpers
ceph: aes crypto and base64 encode/decode helpers
ceph: allow renewal of auth credentials
ceph: add struct version to auth encoding
ceph: add support for auth_x authentication protocol
ceph: add uid field to ceph_pg_pool
ceph: cap revocation fixes
ceph: do not retain caps that are being revoked
ceph: fix sync read eof check deadlock
ceph: cleanup async writeback, truncation, invalidate helpers
ceph: invalidate pages even if truncate is pending
ceph: remove bogus invalidate_mapping_pages
ceph: fix msgr to keep sent messages until acked
ceph: reset osd connections after fault
ceph: allow connection to be reopened by fault callback
ceph: cancel delayed work when closing connection
ceph: use rbtree for mds requests
ceph: use rbtree for snap_realms
ceph: use rbtree for mon statfs requests
ceph: fix authentication races, auth_none oops
ceph: clean up readdir caps reservation
ceph: fix iterate_caps removal race
ceph: fix memory leak when destroying osdmap with pg_temp mappings
ceph: use rbtree for pg pools; decode new osdmap format
ceph: v0.19 release
ceph: fix typo in ceph_queue_writeback debug output
ceph: fix check for invalidate_mapping_pages success
ceph: fix up unexpected message handling
ceph: fix comments, locking in destroy_inode
ceph: drop messages on unregistered mds sessions; cleanup
ceph: fix client_request_forward decoding
ceph: invalidate_authorizer without con->mutex held
ceph: fix connection fault STANDBY check
ceph: remove fragile __map_osds optimization
ceph: remove bogus mds forward warning
ceph: reset bits on connection close
ceph: use single osd op reply msg
ceph: fix snaptrace decoding on cap migration between mds
ceph: reset front len on return to msgpool; BUG on mismatched front iov
ceph: set osd request message front length correctly
ceph: return EBADF if waiting for caps on closed file
ceph: fix osdmap decoding when pools include (removed) snaps
ceph: include migrating caps in issued set
ceph: fix flush_dirty_caps race with caps migration
Yehuda Sadeh (23):
ceph: fix msgpool reservation leak
ceph: remove unaccessible code
ceph: writepage grabs and releases inode
ceph: writeback congestion control
ceph: fix copy_user_to_page_vector()
ceph: change dentry offset and position after splice_dentry
ceph: allocate middle of message before stating to read
ceph: refactor messages data section allocation
ceph: alloc message data pages and check if tid exists
ceph: keep reserved replies on the request structure
ceph: remove unreachable code
ceph: always send truncation info with read and write osd ops
ceph: remove unused variable
ceph: put unused osd connections on lru
ceph: fix short synchronous reads
ceph: refactor ceph_write_begin, fix ceph_page_mkwrite
ceph: fix truncation when not holding caps
ceph: sync read/write considers page cache
ceph: remove page upon writeback completion if lost cache cap
ceph: don't truncate dirty pages in invalidate work thread
ceph: cleanup redundant code in handle_cap_grant
ceph: don't clobber write return value when using O_SYNC
ceph: reset osd after relevant messages timed out
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/