Re: All the problems with 2.2.8/2.3.x and bdflush/update

Zack Weinberg (zack@rabi.columbia.edu)
Sat, 15 May 1999 16:30:42 -0400


On Sat, 15 May 1999 20:19:58 +0200 (CEST), Andrea Arcangeli wrote:
>On Sat, 15 May 1999, Zack Weinberg wrote:
>
>>Here's a scenario for you: I do some operation which generates a lot
>>of dirty buffers, but not enough to trigger bdflush due to low
>>memory. The machine then sits completely idle for several hours. If
>>update is not running, will those dirty buffers be written back?
>
>No, but where is the problem? ;)
>
>If you don't want to go in too much big troubles with a crash or a
>power-loss, then simply run update.

I get the feeling we are talking past each other. From my point of
view, it is entirely too easy to forget to run update. You then have
no safety net for disk operations. Putting update into the kernel
guarantees you don't make this mistake. The patch makes the kernel
_smaller_, frees a task slot, and is a wash performance-wise. What's
not to like?

Have you ever booted with init=/bin/sh to run fsck by hand, rebooted
immediately after it got done, and discovered that NONE of fsck's
writes hit the disk? I have. (It was an old fsck, I think it calls
sync now. Still.)

>>>Think if bdflush was writing the last bdflush_param.ndirty buffer. You'll
>>>issue a sleep_on() but you'll get a wakeup without waiting that some more
>>>buffer is been synced back to disk. And my way will avoid many task switch
>>>ping-pong.
>>
>>If bdflush is already active, you probably wanted to be woken up at
>>that point anyway. I think.
>
>If bdflush is active you want to sleep and wait of I/O completation if
>there are too many dirty buffers.

If bdflush wakes you up, it thinks there aren't too many dirty
buffers. Of course, it's probably wrong. Which is an argument for
doing it your way.

Here's an interesting statistic: I had a different version of my
patch that wrote max(bdf_prm.b_un.ndirty, nr_buffers*bdf_prm.b_un.nfract/100)
buffers, instead of checking every ndirty buffers whether the dirty
list had dropped below the required fraction. I expected this to be
equivalent except for code appearance.

In reality, doing it this way you gain 50-75K/sec on every bonnie
operation except block writes, which lose 500K/sec! I am not certain
what's doing this, but I think the problem is that bonnie keeps
generating dirty buffers while bdflush is writing. With the
every-ndirty check, bdflush will keep writing; otherwise it goes to
sleep again and has to be reawakened.

>>lose on read-mod-write, and a biggish improvement on block input. The
>>combination performs about the same as andrea1 by itself.
>
>I run some bonnie here too and I changed some worthwhile thing since
>2.2.8_andrea1. I have a fully-featured buffer patch almost ready against
>2.2.9. I am finishing to fix a memleak (just a minor issue ;), then I'll
>post it here.

I'm using 2.3 only, now; I don't think Linus wants either of our
patches in 2.2 after the 2.2.8 fiasco.

>BTW my code you merged into your -z patch had a minor issue (not really a
>bug). Just fixed here though. Wait the new patch then feel free to copy my
>code into your patch again (or take a more polite option: provide patches
>incremental with my pending buffer patch ;).

I'm going to keep pulling chunks out of your code, not to be rude but
because I think your buffer patch is trying to do too many things at
once. I don't want to try to sort out which part of it is responsible
for performance deltas in either direction.

If you issue a patch versus 2.3.x that just does flushtime and
tasks start I/O directly, I will work relative to that.

zw

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/