Re: [PATCHv4] memcg: reclaim memory from node in round-robin

[KAMEZAWA Hiroyuki]

On Wed, 4 May 2011 14:26:23 -0700
Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:

> On Thu, 28 Apr 2011 10:49:12 +0900
> KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> wrote:
> 
> > On Thu, 28 Apr 2011 10:37:05 +0900
> > Daisuke Nishimura <nishimura@xxxxxxxxxxxxxxxxx> wrote:
> > > > +	if (time_after(mem->next_scan_node_update, jiffies))
> > > > +		return;
> > > > +
> > > Shouldn't it be time_before() or time_after(jiffies, next_scan_node_update) ?
> > > 
> > > Looks good to me, otherwise.
> > > 
> > 
> > time_after(a, b) returns true when a is after b.....you're right.
> > ==
> > Now, memory cgroup's direct reclaim frees memory from the current node.
> > But this has some troubles. In usual, when a set of threads works in
> > cooperative way, they are tend to on the same node. So, if they hit
> > limits under memcg, it will reclaim memory from themselves, it may be
> > active working set.
> > 
> > For example, assume 2 node system which has Node 0 and Node 1
> > and a memcg which has 1G limit. After some work, file cacne remains and
> > and usages are
> >    Node 0:  1M
> >    Node 1:  998M.
> > 
> > and run an application on Node 0, it will eats its foot before freeing
> > unnecessary file caches.
> > 
> > This patch adds round-robin for NUMA and adds equal pressure to each
> > node. When using cpuset's spread memory feature, this will work very well.
> > 
> > But yes, better algorithm is appreciated.
> 
> That ten-second thing is a gruesome and ghastly hack, but didn't even
> get a mention in the patch description?
> 
> Talk to us about it.  Why is it there?  What are the implications of
> getting it wrong?  What alternatives are there? 
> 

Ah, sorry I couldn't think of fix to that levet, I posted.

> It would be much better to work out the optimum time at which to rotate
> the index via some deterministic means.
> 
> If we can't think of a way of doing that then we should at least pace
> the rotation frequency via something saner than wall-time.  Such as
> number-of-pages-scanned.
> 


What I think now is using reclaim_stat or usigng some fairness based on
the ratio of inactive file caches. We can calculate the total sum of
recalaim_stat which gives us a scan_ratio for a whole memcg. And we can
calculate LRU rotate/scan ratio per node. If rotate/scan ratio is small,
it will be a good candidate of reclaim target. Hmm,

  - check which memory(anon or file) should be scanned.
    (If file is too small, rotate/scan ratio of file is meaningless.)
  - check rotate/scan ratio of each nodes.
  - calculate weights for each nodes (by some logic ?)
  - give a fair scan w.r.t node's weight.

Hmm, I'll have a study on this.

Thanks.
-Kame













--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/