Re: [resent PATCH] Re: very slow parallel read performance

From: Rik van Riel (riel@conectiva.com.br)
Date: Fri Aug 24 2001 - 18:03:26 EST


On Fri, 24 Aug 2001, Daniel Phillips wrote:

> > Actually, no. FIFO would be ok if you had ONE readahead
> > stream going on, but when you have multiple readahead
> > streams going on you want to evict the data each of the
> > streams has already used, and not all the readahead data
> > which happened to be read in first.
>
> We will be fine up until the point that the set of all readahead
> fills the entire cache, then we will start dropping *some* of
> the readahead. This will degrade gracefully: if the set of
> readahead is twice as large as cache then half the readahead
> will be dropped. We will drop the readahead in coherent chunks
> so that it can be re-read in one disk seek. This is not such
> bad behaviour.

The problem is that it WON'T degrade gracefully. Suppose we
have 5 readahead streams, A B C D and E, and we can store
4 readahead windows in RAM without problems. A page which has
not yet been read is marked with a capital letter, a page
which has already been read is marked with a small letter.

The queue looks like this, with new pages being added to the
front and old pages being dropped off the right side:
        AAaaBBbbCCccDDdd

With the current use-once thing, we will end up dropping ALL
pages from file D, even the ones we are about to use (DDdd).

With drop-behind we'll drop four pages we have already used,
without affecting the pages we are about to use (dcba).

> That said, I think I might be able to come up with something
> that uses specific knowledge about readahead to squeeze a little
> better performance out of your example case without breaking
> loads that are already working pretty well. It will require
> another lru list - this is not something we want to do right
> now, don't you agree?

Ummm, if you're still busy trying to come up with the idea,
how do you already know your future idea will require an extra
LRU list? ;)

cheers,

Rik

--
IA64: a worthy successor to the i860.

http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Aug 31 2001 - 21:00:15 EST