Re: dynamic partitions

Dan Merillat (dan@merillat.org)
Mon, 19 May 1997 12:03:31 -0400


Erik Corry writes:
> Jan Kratochvil wrote:
> > P.S.: When we're at another large memory copying, will someone the write pa
ge
> > flipping support so that during CD burn the data will no longer be copied
> > 6 times? Count with me:
> >
> > [disk DMA->kernel buffer]
> > 1. buffer -> userland mkisofs
> > 2. mkisofs -> kernel pipe buffer
> > 3. kernel pipe buffer -> userland buffering tool
> > [>10MB buffer to deal with starvations during compiles]
> > 4. userland buffering tool -> kernel pipe buffer
> > 5. kernel pipe buffer -> userland cdrecord buffer
> > 6. userland cdrecord buffer -> scsi layer buffer
> > [scsi layer->scsi controller DMA]
>
> To get this to work better, you could:
> 1) Use a ring buffer in shared memory instead of pipes for the
> "userland buffering tool".

> Then you get:
>
> 1. DMA raw partition -> ring buffer in shared memory.
> 2. DMA ring buffer in shared memory -> SCSI card with CD writer.
>

It's not a raw partition: mkisofs generates the RO iso9660 from a existing
(usually ext2) filesystem, and outputs it where you want it to go.

So really, when creating a "new" CD, you can cut down on the number of
copies by

1) increasing the userland cdwrite buffer to 10mb (kills 2 right there)
2a) making a cdwriter module instead of requiring a seperate program
or
2b) moving cdwrite into mkisofs... (probably the better solution)

If you did those two, the copies end up:

[disk DMA->kernel buffer]
1. buffer -> userland mkisofs
2. mkisofs -> scsi layer buffer
[scsi layer->scsi controller DMA]

Which is about as quick a copy as you can get when modifying the data
userland...

Of course, this will take a 10mb ring buffer in mkisofs (though since
only one process, you may not need 10mb)

> This is assuming that data gets copied verbatim from the disk
> to the CDROM.

It usually never does... not block for block, anyway.

> Of course, the ideal would be to DMA directly from the disk
> to the CD across the PCI bus. Hey they might be on the same
> SCSI controller, then the data never has to leave the
> controller... :-).

That is what a cd duplicator is for. ;-)

The ideal would be some way to say "write blocks 2 4 1 9 5 7 3" on
disk 1 to the CD burner, in that order, mixed in with some userland
blocks.... this would keep 99% of diskcopy disk->disk

Pointless for CD burns, if mkisofs could directally handle the burning
of the CD it would fix that problem... however, how fast could we make cp
disk1 -> disk2 transfers if we played games? Perhaps a copy(fd, fd, size)
syscall? I.E. I don't care what the data is, move it from file1 to file2
as efficiently as possible. That would allow the scsi layer to move
disk->disk without even hitting the PCI bus...

More useful on network sockets... read() a header from the incoming
socket, write() a new header on the outgoing socket, then copy() the rest
of the packet down the line. And, if you are interested in the data,
copy_and_read(fd,fd,buf,size) pageflips/copies the data into buf once it
is sent on. It'd be nice for high-speed network links (> 100bt) to
eliminate lots of memory copying to->from userspace.

;-)

Of course, the same effect can be achieved by pageflipping things
into user buffers... flip it in on read(), modify in userland,
write() pulls directly from userspace and returns control when it's done.
Yes, copy() was a joke, but eliminating memory copies is a goal greatly
to be desired... and flip in/write from userland would speed things
up a lot. (Granted, userspace would have to page align buffers...
does malloc() align now or not?)

--Dan