Re: PATCH: Raw device IO for 2.1.131
Richard Gooch (rgooch@atnf.csiro.au)
Sun, 13 Dec 1998 18:00:08 +1100
Alan Cox writes:
> > - real work. If zero-copy makes a real difference, you'd better buy a new
> > computer, because your memory subsystem is too slow. Either you're IO
>
> Every computer on the planet has a memory subsystem that is too slow, thats
> why we have cache. Zero copy makes a difference on real world networking.
> Real world file and web serving is about blasting arbitary blocks of data
> from memory to and from networks. Disk doesnt come into it that much. Sendfile
> may be able to be extended to cover some of this, but its only half the needed
> API - you need a way to do writev()/readv() type operations with mixed
> kernel and user sourced data. Gigabit cards can checksum that sort of stuff
> quite happily. Larry seems to have the right idea with his API proposals here.
>
> Think not just about httpd/ftpd which sendfile nicely conquers but also about
> NFS/Samba/NCPfs/Netatalk file sharing stuff.
>
> > So I want to be able to do things like frame capture (into a _file_, not a
>
> Frame capture into a file is uninteresting for serious video people because
> a) we have this toy computer 2Gig file limit[1] and b) performance is a colossal
> issue. a) is easy to work around [apple worked around it nicely with Qt,
> intel forgot] b) is a problem. Video people really don't care about throwing
> an 80Gig FCAL array at a video problem. No file system who cares - saves fscking
>
> [1] to them anyway.
>
> > buffer cache behaviour under certain circumstances. But basically I still
> > see absolutely no reason for supporting raw devices per se.
>
> We need memory locking for stuff like video capture DMA. The demonstration
> HDTV chipsets want capture to DMA 1600x1200x24bit data to memory targets.
> The existing bttv sick 'vmalloc and look the other way' approach isnt as
> good as locking pages for this (Disk I/O issues aside)
>
> Similarly scanners really want to be working direct scatter gather to big
> DMA buffers. Its fairly crazy to then go copying those to user space.
I think Linus' point may be that the extra time spent copying data
around is not that significant. With sendfd(2) or similar, we cover a
whole class of data transfer operations. It seems the majority of the
remaining applications are like what you described above: DMA to user
space memory.
What I'd like to know is why the transfer to user space memory is
deemed so performance critical. Most likely, if you are going to *any*
processing of that data, the CPU and memory IO requirements will be
far greater than the kernel->userspace copy. Is that extra copy really
going to matter? As Linus said, maybe it's time to buy a new computer
if you're running that close to the wire.
On the other hand, if you want to copy from the video capture card
directly into a file, then you would use copyfd(2), in which case
there is no need for kernel->userspace->kernel transfers. Writing file
headers is not a problem, since you just have the following in
userspace:
for (count = 0; count < num_frames; ++count)
{
write_header (fd_file, count);
wait_for_capture (fd_device);
sendfd (fd_file, fd_device, image_size);
}
Linus: have I correctly interpreted your point?
Regards,
Richard....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/