Re: [PATCHSET] workqueue: reimplement CPU hotplug to keep idle workers

[Rafael J. Wysocki]

On Tuesday, July 17, 2012, Tejun Heo wrote:
> Currently, workqueue destroys all workers for offline CPUs unless
> there are lingering work items.  On some platforms, CPU hotplugs are
> used as part of runtime powersaving and thus can be very frequent.
> The overhead of destroying and re-creating workers on each cycle can
> be noticeable.
> 
> In addition, being different from the usual mode of opration,
> kill-all-idle involves separate driving logic, which currently is
> implemented across trustee_thread() and CPU hotplug notifier.  This is
> rather complex and ugly.
> 
> This patchset updates workqueue CPU hotplug path such that a
> disassociated global_cwq (the per-cpu worker pool) is run as if it's
> running as an unbound one (WQ_UNBOUND) - executing work items ASAP
> without concurrency management while following the usual worker
> management based on idle timer.
> 
> While this makes rebinding somewhat more complicated, as it has to be
> able to rebind idle workers too, it allows overall hotplug path to be
> much simpler.  A disassociating global_cwq simply becomes an unbound
> one at the end of CPU_DOWN_PREPARE and makes use of the same logic
> already existing for WQ_UNBOUND.  While re-associating, it switches
> back to a bound one.  Except during the transitions themselves, the
> usual worker management is always in effect making trustee
> unnecessary.
> 
>  0001-workqueue-perform-cpu-down-operations-from-low-prior.patch
>  0002-workqueue-drop-CPU_DYING-notifier-operation.patch
>  0003-workqueue-ROGUE-workers-are-UNBOUND-workers.patch
>  0004-workqueue-use-mutex-for-global_cwq-manager-exclusion.patch
>  0005-workqueue-drop-bind-from-create_worker.patch
>  0006-workqueue-reimplement-CPU-online-rebinding-to-handle.patch
>  0007-workqueue-don-t-butcher-idle-workers-on-an-offline-C.patch
>  0008-workqueue-remove-CPU-offline-trustee.patch
>  0009-workqueue-simplify-CPU-hotplug-code.patch
> 
> 0001 is fix for an unlikely race condition.  0002-0005 prepare for CPU
> hotplug reimplementation.  0006 reimplements re-binding such that it
> can handle idle workers too.  0007 drops killing idle workers of
> offline CPUs.  0008 replaces trustee with the usual management logic.
> 0009 refactors hotplug path a bit to clean it up.
> 
> This patchset is on top of "workqueue: reimplement high priority using
> a separate worker pool" patchset[1] and available on the following git
> branch.
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git review-wq-hotplug
> 
>  include/linux/cpu.h |    5 
>  kernel/workqueue.c  |  738 ++++++++++++++++++++--------------------------------
>  2 files changed, 296 insertions(+), 447 deletions(-)

I like these changes.

I suppose you're going to push them through your tree, right?  I that case
please free to add my Acked-by to them.

Thanks,
Rafael
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/