This was on my TODO list for a while and it turns out someone already fixed the
armwaving overcommit mode for the same problem. It is easy to get into a
situation where you have no overcommit and nothing can be done because there is
no memory to clean up the stable but non-useful state of the machine.
The fix is trivial and duplicated from the armwaving overcommit code path.
The last 3% of the memory can be claimed by root processes only. It isn't a
cure but it does seem to solve the real world problems - at least providing
you have enough memory for 3% to be useful 8).
--- security/commoncap.c~ 2004-08-31 15:27:46.777504736 +0100
+++ security/commoncap.c 2004-08-31 15:27:46.778504584 +0100
@@ -357,6 +357,11 @@
allowed = (totalram_pages - hugetlb_total_pages())
* sysctl_overcommit_ratio / 100;
+ /*
+ * Leave the last 3% for root
+ */
+ if (!capable(CAP_SYS_ADMIN))
+ allowed -= allowed / 32;
allowed += total_swap_pages;
if (atomic_read(&vm_committed_space) < allowed)