Re: [PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock

[Daisuke Nishimura]

On Wed, 14 Apr 2010 21:48:25 -0700, Greg Thelen <gthelen@xxxxxxxxxx> wrote:
> On Wed, Apr 14, 2010 at 7:40 PM, Daisuke Nishimura
> <nishimura@xxxxxxxxxxxxxxxxx> wrote:
> > On Wed, 14 Apr 2010 13:14:07 -0700, Greg Thelen <gthelen@xxxxxxxxxx> wrote:
> >> Vivek Goyal <vgoyal@xxxxxxxxxx> writes:
> >>
> >> > On Tue, Apr 13, 2010 at 11:55:12PM -0700, Greg Thelen wrote:
> >> >> On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> wrote:
> >> >> > On Fri, 19 Mar 2010 08:10:39 +0530
> >> >> > Balbir Singh <balbir@xxxxxxxxxxxxxxxxxx> wrote:
> >> >> >
> >> >> >> * KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2010-03-19 10:23:32]:
> >> >> >>
> >> >> >> > On Thu, 18 Mar 2010 21:58:55 +0530
> >> >> >> > Balbir Singh <balbir@xxxxxxxxxxxxxxxxxx> wrote:
> >> >> >> >
> >> >> >> > > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2010-03-18 13:35:27]:
> >> >> >> >
> >> >> >> > > > Then, no probelm. It's ok to add mem_cgroup_udpate_stat() indpendent from
> >> >> >> > > > mem_cgroup_update_file_mapped(). The look may be messy but it's not your
> >> >> >> > > > fault. But please write "why add new function" to patch description.
> >> >> >> > > >
> >> >> >> > > > I'm sorry for wasting your time.
> >> >> >> > >
> >> >> >> > > Do we need to go down this route? We could check the stat and do the
> >> >> >> > > correct thing. In case of FILE_MAPPED, always grab page_cgroup_lock
> >> >> >> > > and for others potentially look at trylock. It is OK for different
> >> >> >> > > stats to be protected via different locks.
> >> >> >> > >
> >> >> >> >
> >> >> >> > I _don't_ want to see a mixture of spinlock and trylock in a function.
> >> >> >> >
> >> >> >>
> >> >> >> A well documented well written function can help. The other thing is to
> >> >> >> of-course solve this correctly by introducing different locking around
> >> >> >> the statistics. Are you suggesting the later?
> >> >> >>
> >> >> >
> >> >> > No. As I wrote.
> >> >> > Â Â Â Â- don't modify codes around FILE_MAPPED in this series.
> >> >> > Â Â Â Â- add a new functions for new statistics
> >> >> > Then,
> >> >> > Â Â Â Â- think about clean up later, after we confirm all things work as expected.
> >> >>
> >> >> I have ported Andrea Righi's memcg dirty page accounting patches to latest
> >> >> mmtom-2010-04-05-16-09. ÂIn doing so I have to address this locking issue. ÂDoes
> >> >> the following look good? ÂI will (of course) submit the entire patch for review,
> >> >> but I wanted make sure I was aiming in the right direction.
> >> >>
> >> >> void mem_cgroup_update_page_stat(struct page *page,
> >> >> Â Â Â Â Â Â Â Â Â Âenum mem_cgroup_write_page_stat_item idx, bool charge)
> >> >> {
> >> >> Â Âstatic int seq;
> >> >> Â Âstruct page_cgroup *pc;
> >> >>
> >> >> Â Âif (mem_cgroup_disabled())
> >> >> Â Â Â Â Â Âreturn;
> >> >> Â Âpc = lookup_page_cgroup(page);
> >> >> Â Âif (!pc || mem_cgroup_is_root(pc->mem_cgroup))
> >> >> Â Â Â Â Â Âreturn;
> >> >>
> >> >> Â Â/*
> >> >> Â Â * This routine does not disable irq when updating stats. ÂSo it is
> >> >> Â Â * possible that a stat update from within interrupt routine, could
> >> >> Â Â * deadlock. ÂUse trylock_page_cgroup() to avoid such deadlock. ÂThis
> >> >> Â Â * makes the memcg counters fuzzy. ÂMore complicated, or lower
> >> >> Â Â * performing locking solutions avoid this fuzziness, but are not
> >> >> Â Â * currently needed.
> >> >> Â Â */
> >> >> Â Âif (irqs_disabled()) {
> >> > Â Â Â Â Â Â ^^^^^^^^^
> >> > Or may be in_interrupt()?
> >>
> >> Good catch. ÂI will replace irqs_disabled() with in_interrupt().
> >>
> > I think you should check both. __remove_from_page_cache(), which will update
> > DIRTY, is called with irq disabled(iow, under mapping->tree_lock) but not in
> > interrupt context.
> 
> The only reason to use trylock in this case is to prevent deadlock
> when running in a context that may have preempted or interrupted a
> routine that already holds the bit locked.  In the
> __remove_from_page_cache() irqs are disabled, but that does not imply
> that a routine holding the spinlock has been preempted.  When the bit
> is locked, preemption is disabled.  The only way to interrupt a holder
> of the bit for an interrupt to occur (I/O, timer, etc).  So I think
> that in_interrupt() is sufficient.  Am I missing something?
> 
IIUC, it's would be enough to prevent deadlock where one CPU tries to acquire
the same page cgroup lock. But there is still some possibility where 2 CPUs
can cause dead lock each other(please see the commit e767e056).
IOW, my point is "don't call lock_page_cgroup() under mapping->tree_lock". 

Thanks,
Daisuke Nishimura.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/