[PATCH v9 0/5] lib,kprobes: kretprobe scalability improvement

From: wuqiang.matt
Date: Tue Sep 05 2023 - 12:41:23 EST


This patch series introduces a scalable and lockless ring-array based
object pool and replaces the original freelist (a LIFO queue based on
singly linked list) to improve scalability of kretprobed routines.

v9:
1) objpool: raw_local_irq_save/restore added to prevent interruption

To avoid possible ABA issues, we must ensure objpool_try_add_slot
and objpool_try_add_slot are uninterruptible. If these operations
are blocked or interrupted in the middle, other cores could overrun
the same slot's ages[] of uint32, then after resuming back, the
interrupted pop() or push() could see same value of ages[], which
is a typical ABA problem though the possibility is small.

The pair of pop()/push() costs about 8.53 cpu cycles, measured
by IACA (Intel Architecture Code Analyzer). That is, on a 4Ghz
core dedicated for pop() & push(), theoretically it would only
need 8.53 seconds to overflow a 32bit value. Testings upon Intel
i7-10700 (2.90GHz) cost 71.88 seconds to overrun a 32bit integer.

2) codes improvements: thanks to Masami for the thorough inspection

v8:
1) objpool: refcount added for objpool lifecycle management

wuqiang.matt (5):
lib: objpool added: ring-array based lockless MPMC
lib: objpool test module added
kprobes: kretprobe scalability improvement with objpool
kprobes: freelist.h removed
MAINTAINERS: objpool added

MAINTAINERS | 7 +
include/linux/freelist.h | 129 --------
include/linux/kprobes.h | 11 +-
include/linux/objpool.h | 174 ++++++++++
include/linux/rethook.h | 16 +-
kernel/kprobes.c | 93 +++---
kernel/trace/fprobe.c | 32 +-
kernel/trace/rethook.c | 90 +++--
lib/Kconfig.debug | 11 +
lib/Makefile | 4 +-
lib/objpool.c | 338 +++++++++++++++++++
lib/test_objpool.c | 689 +++++++++++++++++++++++++++++++++++++++
12 files changed, 1320 insertions(+), 274 deletions(-)
delete mode 100644 include/linux/freelist.h
create mode 100644 include/linux/objpool.h
create mode 100644 lib/objpool.c
create mode 100644 lib/test_objpool.c

--
2.40.1