Re: [PATCH] memcg: lock-free clear page writeback (Was Re: [PATCH04/10] memcg: disable local interrupts in lock_page_cgroup()

[KAMEZAWA Hiroyuki]

On Fri, 8 Oct 2010 08:35:30 +0900
Minchan Kim <minchan.kim@xxxxxxxxx> wrote:

> Hi Kame,
> 
> On Thu, Oct 7, 2010 at 3:24 PM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@xxxxxxxxxxxxxx> wrote:
> > Greg, I think clear_page_writeback() will not require _any_ locks with this patch.
> > But set_page_writeback() requires it...
> > (Maybe adding a special function for clear_page_writeback() is better rather than
> > Âadding some complex to switch() in update_page_stat())
> >
> > ==
> > From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
> >
> > Now, at page information accounting, we do lock_page_cgroup() if pc->mem_cgroup
> > points to a cgroup where someone is moving charges from.
> >
> > At supporing dirty-page accounting, one of troubles is writeback bit.
> > In general, writeback can be cleared via IRQ context. To update writeback bit
> > with lock_page_cgroup() in safe way, we'll have to disable IRQ.
> > ....or do something.
> >
> > This patch waits for completion of writeback under lock_page() and do
> > lock_page_cgroup() in safe way. (We never got end_io via IRQ context.)
> >
> > By this, writeback-accounting will never see race with account_move() and
> > it can trust pc->mem_cgroup always _without_ any lock.
> >
> > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
> > ---
> > Âmm/memcontrol.c | Â 18 ++++++++++++++++++
> > Â1 file changed, 18 insertions(+)
> >
> > Index: mmotm-0928/mm/memcontrol.c
> > ===================================================================
> > --- mmotm-0928.orig/mm/memcontrol.c
> > +++ mmotm-0928/mm/memcontrol.c
> > @@ -2183,17 +2183,35 @@ static void __mem_cgroup_move_account(st
> > Â/*
> > Â* check whether the @pc is valid for moving account and call
> > Â* __mem_cgroup_move_account()
> > + * Don't call this under pte_lock etc...we'll do lock_page() and wait for
> > + * the end of I/O.
> > Â*/
> > Âstatic int mem_cgroup_move_account(struct page_cgroup *pc,
> > Â Â Â Â Â Â Â Âstruct mem_cgroup *from, struct mem_cgroup *to, bool uncharge)
> > Â{
> > Â Â Â Âint ret = -EINVAL;
> > +
> > + Â Â Â /*
> > + Â Â Â Â* We move severl flags and accounting information here. So we need to
> > + Â Â Â Â* avoid the races with update_stat routines. For most of routines,
> > + Â Â Â Â* lock_page_cgroup() is enough for avoiding race. But we need to take
> > + Â Â Â Â* care of IRQ context. If flag updates comes from IRQ context, This
> > + Â Â Â Â* "move account" will be racy (and cause deadlock in lock_page_cgroup())
> > + Â Â Â Â*
> > + Â Â Â Â* Now, the only race we have is Writeback flag. We wait for it cleared
> > + Â Â Â Â* before starting our jobs.
> > + Â Â Â Â*/
> > +
> > + Â Â Â lock_page(pc->page);
> > + Â Â Â wait_on_page_writeback(pc->page);
> > +
> > Â Â Â Âlock_page_cgroup(pc);
> > Â Â Â Âif (PageCgroupUsed(pc) && pc->mem_cgroup == from) {
> > Â Â Â Â Â Â Â Â__mem_cgroup_move_account(pc, from, to, uncharge);
> > Â Â Â Â Â Â Â Âret = 0;
> > Â Â Â Â}
> > Â Â Â Âunlock_page_cgroup(pc);
> > + Â Â Â unlock_page(pc->page);
> > Â Â Â Â/*
> > Â Â Â Â * check events
> > Â Â Â Â */
> >
> >
> 
> Looks good to me.
> But let me ask a question.
> Why do only move_account need this logic?

Because charge/uncharge (add/remove to radix-tree or swapcache)
never happens while a page is PG_writeback.

> Is deadlock candidate is only this place?
yes.

> How about mem_cgroup_prepare_migration?
> 
> unmap_and_move
> lock_page
> mem_cgroup_prepare_migration
> lock_page_cgroup
> ...
> softirq happen
> lock_page_cgroup
> 
> 
Nice cactch. I'll move prepare_migraon after wait_on_page_writeback()

> If race happens only where move_account and writeback, please describe
> it as comment.
> It would help to review the code in future.
> 

Sure, updates are necessary.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/