Re: A problem about DIRECT IO on ext3

From: Erik Mouw
Date: Mon Aug 29 2005 - 08:30:18 EST


On Mon, Aug 29, 2005 at 08:15:43PM +0800, colin wrote:
> I wrote a simple program to test direct io, and found that there are some
> strange behaviors of it on "ext3".
> My simple program is below. Assume that the executable file name is
> "directio". If I do the following:
> 1. cp directio aaa
> 2. ./directio directio aaa
>
> The size of aaa is about the same with directio. This is wrong.

No, it's right, but it's not what you expected.

> It should be 3 times the size of directio because there are 2 write
> operations and one lseek to the file end.

I suggest to strace() your program to see what happens.

> If the second file is not opened with "O_DIRECT", the result is correct.
>
> What's the problem of direct io? I found that if I remove the instruction of
> lseek, the result is correct.

There are four prerequisites for direct IO:
- the file needs to be opened with O_DIRECT
- the buffer needs to be page aligned (hint: use getpagesize() instead
of assuming that a page is 4k
- reads and writes need to happen *in* multiples of the soft block size
- reads and writes need to happen *at* multiples of the soft block size

> Is there any problem of lseek when doing direct io on ext3?
> My platform is 2.6.11.

There is no problem.

> Regards,
> Colin
>
> #define _GNU_SOURCE
>
> #include <stdio.h>
> #include <fcntl.h>
> #include <sys/stat.h>
> #include <sys/types.h>
> #include <stdlib.h>

Compile your program with -Wall, you're missing quite some include
files over here.

> int main(int argc, char **argv) {
>
> int fd1, fd2, count;
> char *ptr1, *ptr2;
>
> if(argc == 3) {
> fd1 = open(argv[1], O_RDONLY | O_DIRECT, S_IRWXU);
> fd2 = open(argv[2], O_RDWR | O_CREAT | O_DIRECT);
> } else {
> printf("Error syntax\n");
> exit(1);
> }
> printf("%d\n", lseek(fd2, 0, SEEK_END));

Make that lseek(fd2, 4 * 4096, SEEK_SET);

> ptr1 = malloc(4096 + 4096-1);
> ptr2 = (void*)((int)ptr1 - (int)ptr1 % 4096 + 4096);

Use memalign() or posix_memalign().

> do {
> count = read(fd1, ptr2, 4096);
> if(!count)
> break;

And what happens when count < 0 ?

> write(fd2, ptr2, 4096);
> write(fd2, ptr2, 4096);

Check return values.

> } while(count > 0);
>
> free(ptr1);
> close(fd1);
> close(fd2);

return 0;

> }

With the changes, the result is:

erik@arthur:/tmp > ls -l directio aaa
-rwxr-xr-x 1 erik erik 49152 2005-08-29 15:26 aaa*
-rwxr-xr-x 1 erik erik 12628 2005-08-29 15:26 directio*


Erik

--
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/