[patch] cache pipe buf page address for non-highmem arch

[Ken Chen]

It is really sad that we always call kmap and friends for every pipe
buffer page on 64-bit arch that doesn't use HIGHMEM, or on
configuration that doesn't turn on HIGHMEM.

The effect of calling kmap* is visible in the execution profile when
pipe code is being stressed.  It is especially true on amd's x86-64
platform where kmap() has to traverse through numa node index
calculation in order to convert struct page * to kernel virtual
address.  It is fairly pointless to perform that calculation repeatly
on system with no highmem (i.e., 64-bit arch like x86-64).  This patch
caches kernel pipe buffer page's kernel vaddr to speed up pipe buffer
mapping functions.

There is another suboptimal block in pipe_read() where wake_up is
called twice.  I think it was an oversight since in pipe_write(), it
looks like it is doing the right thing.

Signed-off-by: Ken Chen <kenchen@xxxxxxxxxx>


--- linus-2.6.git/fs/pipe.c.orig	2007-03-01 12:41:06.000000000 -0800
+++ linus-2.6.git/fs/pipe.c	2007-03-22 16:28:03.000000000 -0700
@@ -20,6 +20,22 @@

#include <asm/uaccess.h>
#include <asm/ioctls.h>
+#include <asm/kmap_types.h>
+
+#ifdef CONFIG_HIGHMEM
+#define pipe_kmap		kmap
+#define pipe_kmap_atomic	kmap_atomic
+#else	/* CONFIG_HIGHMEM */
+static inline void *pipe_kmap(struct page *page)
+{
+	return (void *) page->private;
+}
+static inline void *pipe_kmap_atomic(struct page *page, enum km_type type)
+{
+	pagefault_disable();
+	return pipe_kmap(page);
+}
+#endif

/*
 * We use a start+len construction, which provides full use of the
@@ -169,10 +185,10 @@ void *generic_pipe_buf_map(struct pipe_i
{
	if (atomic) {
		buf->flags |= PIPE_BUF_FLAG_ATOMIC;
-		return kmap_atomic(buf->page, KM_USER0);
+		return pipe_kmap_atomic(buf->page, KM_USER0);
	}

-	return kmap(buf->page);
+	return pipe_kmap(buf->page);
}

void generic_pipe_buf_unmap(struct pipe_inode_info *pipe,
@@ -316,6 +332,7 @@ redo:
		if (do_wakeup) {
			wake_up_interruptible_sync(&pipe->wait);
 			kill_fasync(&pipe->fasync_writers, SIGIO, POLL_OUT);
+			do_wakeup = 0;
		}
		pipe_wait(pipe);
	}
@@ -423,6 +440,8 @@ redo1:
					ret = ret ? : -ENOMEM;
					break;
				}
+				page->private = (unsigned long)
+							page_address(page);
				pipe->tmp_page = page;
			}
			/* Always wake up, even if the copy fails. Otherwise
@@ -438,9 +457,9 @@ redo1:
			iov_fault_in_pages_read(iov, chars);
redo2:
			if (atomic)
-				src = kmap_atomic(page, KM_USER0);
+				src = pipe_kmap_atomic(page, KM_USER0);
			else
-				src = kmap(page);
+				src = pipe_kmap(page);

			error = pipe_iov_copy_from_user(src, iov, chars,
							atomic);
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/