Re: REAL reason for 2GB file size limit (was: Ok, making ready...)

Theodore Y. Ts'o (tytso@mit.edu)
Fri, 24 Dec 1999 12:55:03 -0500


Date: Fri, 24 Dec 1999 01:02:15 -0700
From: Chuck Phillips <cdp@peakpeak.com>

NIT: The REAL limitation for POSIX file sizes on 32-bit architectures isn't
*directly* from POSIX, but rather, indirectly from ISO/ANSI C (which is
assumed by POSIX).

o In POSIX, the lseek() file offset is defined as off_t. There's nothing
to prevent a 32-bit POSIX-compliant implementation from typedef'ing off_t
as a 64-bit signed "long long".

True, but at the moment we're using off_t as the type in the Linux
system calls. This will get fixed in Linux 2.4. Earlier libc's also
used a 32-bit off_t, and changing this would have been painful.
Fortunately the move to glibc solved this problem for us; or rather, the
pain in moving to glibc also gave us the benefit of a 64-bit interface
between user mode programs and the C library.

o Similarly, ISO/ANSI C's fsetpos()/fgetpos() can be fixed by typedef'ing
fpos_t to be a 64-bit long long.

o The problem lies with ISO/ANSI C's fseek()/ftell(), which use a "long"
for the offset. Why fpos_t wasn't used (consistently!) is beyond me, but
hindsight is 20/20. AFAIK, these are the only two functions that *must*
break if >2GB files are permitted with 32-bit longs. Other functions
*can* break (like the present case), but aren't *required* to break by
the POSIX+C standards.

Actually, ISO/ANSI can be worked around, because ANSI isn't as strict
about error conditions as POSIX is. It would be OK from an ANSI point
of view of an attempt to fopen a > 2GB file returned an E2BIG error
message --- or for ftell/fseek to retun E2BIG.

It's unfortunate that fseek/ftell used "long" explicitly. I suspect
that at the time, the ANSI committee didn't consider the problem of file
sizes greater than what "long" could express.

- Ted

P.S. It's worth noting that fpos_t doesn't have to be a scalar value;
it could be a structure and still conform to ISO/ANSI C.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/