Re: rm a_large_file takes too long under linux-2.2.1 (also unSTOPable)

Zygo Blaxell (uixjjji1@umail.furryterror.org)
15 Feb 1999 12:31:41 -0500


In article <19990211134714Z92298-868+257@mea.tmt.tele.fi>,
Matti Aarnio <matti.aarnio@sonera.fi> wrote:
>Sang Kang <kernel@mocha.sarang.net> wrote:
> The criteria is not 'rm', like has been said here
> earlier, the criteria is loosing the last reference
> to the given file.

Correct.

> Lets say, we have a large directory with lots of files
> to clean up, if each reference loss would cause a new
> thread to be created without bounds, we would soon have
> thousands of threads doing the deletes.

Correct. That would be dumb, so we don't do it.

> Even if such a thing would give fast response speeds,
> there must be some limit at how many of them can be
> running at the same time.

Also correct.

> As the cleanups of small files would happen faster than
> cleanups of large files, thus presuming the count of large
> files to be deleted to be fairly small, one can guess that
> a limit around 10-20 deleter threads would be all what the
> doctor ordered. If the limit is exceeded, deleter blocks
> until the count comes down again (and then starts a new
> thread, and increments the count..)

Here I disagree. You can serialize these quite effectively. Have one
thread per filesystem (anything more will just make the disk heads thrash)
plus a request queue of a few inodes' worth on each of those threads.
When the last reference to a file is lost, send a message to the cleanup
thread and if there is still free space in that thread's queue, unblock
the unlink() call, otherwise block the unlink() call (you'd have to wait
anyway).

So it's obviously possible to get this in kernel space. Now answer the
question "Why?". I think everyone else who replied to this did that quite
effectively--it's kernel bloat that can't already be done in user space.

-- 
Zygo Blaxell, Linux Engineer, Corel Corporation, zygob@corel.ca (work),
zblaxell@furryterror.org (play).  It's my opinion, I tell you! Mine! All MINE!
Size of 'diff -Nurw [...] winehq corel' as of Mon Feb 15 12:14:00 EST 1999
Lines/files:  In 17182 / 210, Out 6461 / 77, Both 20891 / 257

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/