This sounds like a memory leak somewhere: Something eats more and more
memory, until only little memory is left to the applications. I don't
know whether there are already ideas about searching memory leaks in the
kernel; a good start would be watching /proc/slabinfo. If you see that some
pool continually grows, you know at least what kind of object consumes all
the memory. Finding out where it was allocated is left as an exercise :-)
Regards,
Martin