[mmotm][PATCH 0/4] per-process OOM kill v3

From: KOSAKI Motohiro
Date: Wed Aug 26 2009 - 05:32:57 EST

since v2
- rebase to latest mmotm
- fixed strstrip abuse
- oom_adjust_write() use strict_strtol() instead simple_strtol()
- remove unnecessary signal lock (pointed by Oleg)

The commit 2ff05b2b (oom: move oom_adj value) move oom_adj value to mm_struct.
It is very good first step for sanitize OOM.

However Paul Menage reported the commit makes regression to his job scheduler.
Current OOM logic can kill OOM_DISABLED process.

Why? His program has the code of similar to the following.

set_oom_adj(OOM_DISABLE); /* The job scheduler never killed by oom */
if (vfork() == 0) {
set_oom_adj(0); /* Invoked child can be killed */

vfork() parent and child are shared the same mm_struct. then above set_oom_adj(0) doesn't
only change oom_adj for vfork() child, it's also change oom_adj for vfork() parent.
Then, vfork() parent (job scheduler) lost OOM immune and it was killed.

Actually, fork-setting-exec idiom is very frequently used in userland program. We must
not break this assumption.

This patch moves oom_adj to signal_struct instead mm_struct. signal_struct is
shared by thread but isn't shared vfork.

Sorting out OOM requirements:
- select_bad_process() must select killable process.
otherwise OOM might makes following livelock.
1. select_bad_process() select unkillable process
2. oom_kill_process() do no-op and return.
3. exit out_of_memory and makes next OOM soon. then, goto 1 again.
- vfork parent and child must not shared oom_adj.

My proposal
- oom_adj become per-process property. it have been documented long time.
but the implementaion was not correct.
- oom_score also become per-process property. it makes oom logic simpler and faster.
- remove bogus vfork() parent killing logic

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/