[PATCH 11/11] list_lru: don't lock during add/del if unnecessary

[Dave Chinner]

From: Dave Chinner <dchinner@xxxxxxxxxx>

We only add an item to the LRU is it is empty, and we only remove
them if they are on the LRU. We can check these conditions without
holding the node lock and avoid grabbing the lock if they evaluate
as not needing a list operation.

We can do this without concern, because we already have the
situation that two concurrent callers to list_lru_add/del() for the
same object will race on the node lock and so the result of the
concurrent operations on the same object has always been always
undefined.

We still do the check once we have the lock, so even if we are
racing and decide we need the node lock to do the operation, then
the behaviour under the lock will be unchanged.

This significantly reduces the per-node lock traffic on workloads
where we aren't lazy about adding and removing objects from the LRU
lists.

Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx>
---
 mm/list_lru.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/mm/list_lru.c b/mm/list_lru.c
index 9aadb6c..5207ae2 100644
--- a/mm/list_lru.c
+++ b/mm/list_lru.c
@@ -15,6 +15,9 @@ bool list_lru_add(struct list_lru *lru, struct list_head *item)
 	int nid = page_to_nid(virt_to_page(item));
 	struct list_lru_node *nlru = &lru->node[nid];
 
+	if (!list_empty(item))
+		return false;
+
 	spin_lock(&nlru->lock);
 	WARN_ON_ONCE(nlru->nr_items < 0);
 	if (list_empty(item)) {
@@ -34,6 +37,9 @@ bool list_lru_del(struct list_lru *lru, struct list_head *item)
 	int nid = page_to_nid(virt_to_page(item));
 	struct list_lru_node *nlru = &lru->node[nid];
 
+	if (list_empty(item))
+		return false;
+
 	spin_lock(&nlru->lock);
 	if (!list_empty(item)) {
 		list_del_init(item);
-- 
1.8.3.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/