Hi,
Several places in the kernel run holding the global kernel lock when it
isn't needed. This usually occurs when where is a function which can be
called via many different paths; some holding the lock, others not.
Now, if a function can block (and hence drop the kernel lock) the caller
cannot rely on the kernel lock for its own integerity. Such a functon
_may_ be a candidate for dropping the lock (if it is held) on entry, and
retaken on exit. This improves SMP scalability.
A good example of this is generic_make_request(). This function can be
arrived at by several different paths, some of which will be holding the
lock, and some not. It (or, rather the functions it can call) may block
and a caller has no control over this. generic_make_request() does not
need the kernel lock itself, and any functions it calls which do require
the lock should be taking it (as they cannot guarantee it is already
held). This makes it a good candidate for dropping the lock early (rather
than only dropping when blocking).
Dropping the kernel lock, even for a short period, helps
scalability. Note, if the lock is held when an interrupt arrives, the
interrupt handler runs holding the lock and so do any bottom-half handlers
run on the back of the interrupt. So, less time it is held, the smaller
the chance of being interrupted while holding it, the better the
scalability.
As the current nfsd code isn't threaded, it runs holding the kernel
lock. Any reduction in holding the lock helps nfs server scalability.
The current macros used to drop and retake the kernel lock in
schedule() cannot be used in many cases as they do not nest.
The attached patch (against 2.4.1-pre10) defines two new macros for x86
(save_and_drop_kernel_lock(x) and restore_kernel_lock(x)) and a new
declaration macro (DECLARE_KERNEL_LOCK_COUNTER(x)). These allow "pushing"
and "poping" of the kernel lock.
The patch also contains some examples of usage (although the one in
nfsd/vfs.c could be done with an unlock_kernel()/lock_kernel() pair).
If the idea is acceptable, I'll tidy up the patch and add the necessary
macros for other archs.
My questions are;
a) Does this make sense?
b) Is it acceptable in the kernel?
c) Any better suggestions for the macro names?
I'd be interested in any results from those applying this patch
and running benchmarks on SMP boxes - mainly filesystem benchmarks.
I admit this is not the cleanest of ideas.
Mark
This archive was generated by hypermail 2b29 : Wed Jan 31 2001 - 21:00:21 EST