> Is this a known limitation of the current (2.1.16) linux kernel, i.e. that
> no tape I/O which involves buffers or record sizes greater than 128KB in
> size is supported?? If so, I think that is too low of a limit, and far
> too restrictive, and I would like some advice on how to increase this
> limit. Is this limit in any way related to the (undocumented) fixed con-
> stant called NR_MEM_LISTS which is defined at or near line 37 of the
> kernel file `mm/page-alloc.c'? It seems to me that it may indeed be.
> (I note that that symbol, NR_MEM_LISTS, is defined to `6' and that
> 4KB << 6 == 128KB.)
>
This is a known limitation (and documented in
linux/drivers/scsi/README.st): the tape driver allocates a buffer that is
contiguous in the physical memory and the largest block that can be
allocated is currently 256 kB for Alphas and 128 kB for the other
architectures.
> Anyway, another equally serious problem I have found is that even when
> I defined the `buf_size' constant in the program below to just (1 << 17),
> I *still* get the same errno (EOVERFLOW) from the `st' SCSI tape driver
> in some cases... although not right after a reboot. It seems that after
> running the system for awhile (and opening several xterm windows) the
> kernel is no longer willing or able to supply a 128KB I/O buffer to the
> SCSI tape driver.
>
The default buffer size is defined by ST_BUFFER_BLOCKS in
linux/drivers/scsi/st_options.h and in the distribution kernel it is 32
kB (this can be changed also at kernel boot). The buffers the driver
allocates are usually of this size. When the driver has allocated a buffer
of this size, it does not free it because it may not be able to reallocate
it later. Two buffers are usually allocated at boot time.
However, if the default buffer is too small for some operation, the driver
tries to allocate a larger temporary buffer that is deallocated after use.
This explains why your 128 kB read requests don't fail right after boot
but may fail later.
> I do not understand enough linux kernel internals to be sure that this is
> what is happening, but that is my (educated?) guess.
>
> Can anyone confirm or refute this guess? (please?)
>
> Anyway, assuming that this indeed what is happening, can anyone advise me
> on what I might do to my kernel if I wish to insure that memory (for an
> I/O buffer) will *always* be made available to the SCSI tape driver when
> it requests it... at least as long as the request could in fact be satisfied
> (without overwriting the kernel itself)?
>
If you want to use larger blocks than 32 kB, you should either edit
st_options.h and recompile the kernel or use the boot definition
st=128,126,n where n is the number of buffers to allocate (this assumes
you want the maximum to be 128 kB). Use the value one for n to save some
memory if you don't use more than one drive concurrently.
> To clarify, I think that if the SCSI tape driver requests some pages for
> use as an I/O buffer, this request should always succeed (assuming that
> there is enough physical memory on the system, minus the kernel of course)
> even if it means that one or more user processes have to be swapped out in
> order to make enough free pages to satisfy the request. In other words,
> I think that drivers should have be given priority access to memory pages
> above that afforded to mere user processes.
>
I would like this, too, but this needs some changes in the kernel memory
allocator. However, as you can see above, tape blocks up to 128 kB can be
reliably read and I think a larger block size is not usually necessary (I
know one case). A common rule is not to use larger blocks than 64 kB if
you want to read the tape with other computers :-)
Kai