Re: Memory overcommit

[KAMEZAWA Hiroyuki]

On Tue, 3 Nov 2009 12:49:52 -0800 (PST)
David Rientjes <rientjes@xxxxxxxxxx> wrote:

> On Fri, 30 Oct 2009, KAMEZAWA Hiroyuki wrote:
> 
> > > >    - The kernel can't know the program is bad or not. just guess it.
> > > 
> > > Totally irrelevant, given your fourth point about /proc/pid/oom_adj.  We 
> > > can tell the kernel what we'd like the oom killer behavior should be if 
> > > the situation arises.
> > > 
> > 
> > My point is that the server cannot distinguish memory leak from intentional
> > memory usage. No other than that.
> > 
> 
> That's a different point.  Today, we can influence the badness score of 
> any user thread to prioritize oom killing from userspace and that can be 
> done regardless of whether there's a memory leaker, a fork bomber, etc.  
> The priority based oom killing is important to production scenarios and 
> cannot be replaced by a heuristic that works everytime if it cannot be 
> influenced by userspace.
> 
I don't removed oom_adj...

> A spike in memory consumption when a process is initially forked would be 
> defined as a memory leaker in your quiet_time model.
> 
I'll rewrite or drop quiet_time.

> > In this summer, at lunch with a daily linux user, I was said
> > "you, enterprise guys, don't consider desktop or laptop problem at all."
> > yes, I use only servers. My customer uses server, too. My first priority
> > is always on server users.
> > But, for this time, I wrote reply to Vedran and try to fix desktop problem.
> > Even if current logic works well for servers, "KDE/GNOME is killed" problem
> > seems to be serious. And this may be a problem for EMBEDED people, I guess.
> > 
> 
> You argued before that the problem wasn't specific to X (after I said you 
> could protect it very trivially with /proc/pid/oom_adj set to 
> OOM_DISABLE), but that's now your reasoning for rewriting the oom killer 
> heuristics?
> 
One of reasons. My cusotomers always suffers from "OOM-RANDOM-KILLER".
Why I mentioned about "lunch" is for saying that "I'm not working _only_
for servers."
ok ?


> > I can say the same thing to total_vm size. total_vm size doesn't include any
> > good information for oom situation. And tweaking based on that not-useful
> > parameter will make things worse.
> > 
> 
> Tweaking on the heuristic will probably make it more convoluted and 
> overall worse, I agree.  But it's a more stable baseline than rss from 
> which we can set oom killing priorities from userspace.

- "rss < total_vm_size" always.
- oom_adj culculation is quite strong.
- total_vm of processes which maps hugetlb is very big ....but killing them
  is no help for usual oom.

I recommend you to add "stable baseline" knob for user space, as I wrote.
My patch 6 adds stable baseline bonus as 50% of vm size if run_time is enough
large.

If users can estimate how their process uses memory, it will be good thing.
I'll add some other than oom_adj (I don't say I'll drop oom_adj).

Thanks,
-Kame






--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/