[Problem] Constantly swapping-in swaped-out pages (2.0 & 2.1)

Simon Kirby (sim@netnation.com)
Tue, 25 Aug 1998 00:54:56 -0700 (PDT)


Peter and I were discussing on linuxnet a problem with the swapper that
we have bothed noticed in the output of "vmstat". It appears that when
an executable that continues to execute (say, for example,
for(;;sleep(1)); ) is paged out, it will show up as being paged back in
constantly once every second and is never actually taken back up off the
swap, even when there is a lot of free memory available.

When swap is disabled and reenabled, the machine sits normally and there
are no more swap-ins or swap-outs and no more disk access (previously
there would be a few interrupts on the disk controller every time the
loop executed and the "swaped-in" would show a value, indicating that it
was actually being paged in from disk).

This problem seems to be present in 2.0.33, 2.0.35, 2.1.117, and probably
many other kernels.

>From IRC:

---
* peter wonders why this system is so actively doing swap ins all the time.
* peter ponders
<Simon> without doing swapouts?
 <peter> Simon: Yup.
<Simon> that happens here, too.
<Simon> 2.0 and 2.1 :)
<Simon>  procs                  memory    swap        io    system         cpu
<Simon>  r b w  swpd  free  buff cache  si  so   bi   bo   in   cs  us  sy  id
<Simon>  0 0 0  2720  4404 35208 175504  20   0    5    0  151   60   2   3  95
<Simon>  1 0 0  2720  4368 35208 175536   0   0    0    0  156   91   1   3  96
<Simon>  0 0 0  2720  4596 35208 175504  20   0    5   35  161   66   1   8  91
<Simon> etc
 <peter> Simon: Any particular reason?
<Simon> not sure
 <peter>  3 0 0  1016  3728 15248 11692 245   0   61    3  612  618   6  27  66
<Simon> it "swaps in" but the "swpd" doesn't change.
 <peter> Simon: Yeah, that would mean there are swap outs. Which there are none.
 <MrCurious> simon: maybe it only changes when the swaped in part gets "dirtied"
 as intil that point, it doesnt need to re-swap it in order to discard it
<Simon> ahh
<Simon> shouldn't it be pulled out of swap if it gets used that much, though?
<Simon> obviously, with 175 megs of cache, there is enough room :)
 <peter> Simon: Yes, and something else gets swapped out instead. So you should
 see the swap out counter >0.
<Simon> but swapoff -a && swapon -a  followed by watching vmstat for a while
        shows that there's no more swapping
<Simon> so it should be taken out of swap..or is it actually not reading the
        disk?
 <peter>  6 0 0     0  1076 14092 11464   0   0    0    4  550  495   8  37  56
 <peter> You're right.
 <peter> That would mean the swap code is broken?
<Simon> *shrug* :)
<Simon> lemme check some other servers
<Simon> yup, the same
<Simon>  3 0 0  1424 109464 12724 123120  20   0    5    0  473   64  56  14  30
<Simon>  3 0 0  1424 109428 12724 123120  20   0    5    0  289   62  58  14  28
<Simon>  4 0 0  1424 108412 12724 123120  20   0    5    0  215   57  58  12  30
<Simon>  3 0 0  1424 109396 12724 123120  20   0    5    0  109   63  59  12  29
<Simon> constantly swapping 20 every second
<Simon> lemme see if the aic is interrupting
<Simon> yup, a few interrupts every second
<Simon> hrmm..should post this to linux-kernel
 <peter> Yeah.
 <peter> What kernel do you run? 2.0.35?
<Simon> the machine was sitting there paging in my "jobspawnd" binary which has a main loop of
        sleep(1); <bunch of checks>
 <peter> jobspawnd?
<Simon> and the scsi device had a few interrupts every second when it was going (I straced it),
        indicating that was what was happening
 <peter> Does that fork() many times?
<Simon> so it's broken :)
<Simon> it fork()s if there's room for another job...
<Simon> but in its life it forks millions of times :)
<Simon> it's the heart of a web stats server
 <peter> Simon: Same here, I'm running a proxy firewall that forks() a process for every
 connection.
<Simon> 2.1 on both of these servers right now, but I've noticed it with 2.0 also I think
 <peter> I'm running 2.0.35.
 <peter> Maybe it has been in Linux for a long time.
---

Any ideas?

Please CC back to me any replies. :)

Simon-

| Simon Kirby | Systems Administration | | mailto:sim@netnation.com | NetNation Communications | | http://www.netnation.com/ | Tech: (604) 684-6892 |

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html