Re: advice requested re: huge files and/or disk seeking

From: Rogier Wolff (R.E.Wolff@BitWizard.nl)
Date: Tue Feb 01 2000 - 05:33:21 EST


Paul Barton-Davis wrote:
> i am working on a multitrack hard-disk recorder for ALSA. when i say
> multitrack, i don't mean "record one track or two, then add
> another". i mean "record 24+ tracks of 24bit audio at 96kHz sample
> rate at the same time".

Hmm. I'm working for the guys who sell the $25000 - $50000 machines
that do this... ;-)

> i face a problem or two. a 4GB file size limit means that i could
> never fit more than about 7-3/4 minutes of audio into any one
> file. thats bad, very bad.

Right.

> instead, and also prompted by the fact that the audio interface (the
> RME Digi9652, a pure digital interface with 26 input channels and 26
> output channels) does not interleave its data, i have taken the
> obvious approach of storing each track in its own file.

OK.
 
> this works fine for recording, but playback is *hard* to do if we want
> low latency operation (where low latency means queing less than, oh,
> 20ms worth of data in the h/w buffer). why is it hard ? because seek
> times on modern disks are at least 4.5ms, and when we have to seek
> across 24+ files, which happens from time to time as the buffer cache
> does it stuff, we lose bigtime.

Count on seek times of 16ms. Things may be a bit better, but not by
much.

Your files are going to be hopelessly scattered anyway: when you're
recording 24 files at 96kHz and writing those files sequentially, they
will get interleaved all over the place....

You said hardware buffer. That doesn't mean you can't have a software
buffer.

If a user hits "rewind", you tell all the buffer programs to do the
rewind. So after 16ms the first has the first part of the buffer
loaded. After another 16ms the second has its buffer loaded.

In the mean while, you should asynchronously be playing those tracks
that happen to be already in the buffer. Prevent transients, and the
human won't notice. Have a button that says: "final" which will mean
that you wait for completion.

A human who hears: "Hey Where did the trumpet in the intro go?", will
hit rewind again, and this time it will be there, and the human will
blame himself....

> one option is to pray for a large file size limit, and then interleave
> the data. last thing i knew, this was still an unincorporated kernel
> patch. bad choice from my perspective, if thats still true.

This is NOT an option: the mixer will have to be able to offset the
different tracks.

Suppose you find out that the trompetist was 30m away, so was playing
100ms too late. You will want to be able to mix him in with the rest
of the instruments at the right moment. So you should be able to run
different tracks at different offsets anyway. And having the tracks
separate is only useful if THAT's what you want to do with it...

So separating the tracks is good.

> i can check this fairly easily, but i wanted to put the problem to the
> brilliant minds here, and see if anyone has any thoughts. the goal is
> to get "instant" response to "tape transport" controls - some clicks
> on the rewind button, and when they let go, the time until we're ready
> to start playing again is less than typical human physical response
> times. in that time, we (may) have to seek and load in about 1-3secs
> worth of data from the N files used to hold the audio.

The question is: How much seeking can you tolerate before you don't
meet the throughput requirements anymore.

This can be calculated pretty easily:

In a second you need 24bit / (8bits/byte) * 96KHz * 24channels = 6.9Mb
of data. With a 20Mb per second disk, you need 0.35 seconds of
disk-time to read this. So you have 0.65 seconds of "seek-time" to do
the seeks. So you can allow about 40 seeks per second. This means that
per channel you need to issue a read of about 0.6 seconds or 172k to
achieve this performance. If you'd read about 250k at a time, you have
some margin left.

So if you want the "perfect" playback, you need 24*250k = 6Mb of data,
and 24 seeks. That takes 6/20 + 24*0.016 = 684ms. Clicking the rewind
button takes 300ms, so this is just a teeny bit slower.

I'm working on the video part of the machine. I'll be playing along
the video that goes along with the sound. Seems to be working. :-) It
is THEIR responsibility to configure the machine that my driver
achieves the 3MB per second of video throughput alongside with the
7 or 14Mb (for the 48 channel model) of audio throughput.

Oh, one more tip: Have a commandline backend: don't do anything in the
user-interface program.

I have an "mp3 box": a Linux machine with a large harddisk that plays
my music for me. I very much like "kjukebox", but quits as soon as I
kill the userinterface by turning off my workstation.

It should have been designed with a backend that schedules the songs,
and a userinterface that interacts with that. The userinterface can
then be used to tell it: "play this and that", and then the backend
plays it wether the frontend is present or not.

                                        Roger.

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
 "I didn't say it was your fault. I said I was going to blame it on you."

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Mon Feb 07 2000 - 21:00:06 EST