Re: NE2000 slow interrupts safe?

Richard B. Johnson (root@chaos.analogic.com)
Sun, 19 Apr 1998 12:42:24 -0400 (EDT)


On Sun, 19 Apr 1998, Richard Gooch wrote:

> Richard B. Johnson writes:
> > On Fri, 17 Apr 1998, Richard Gooch wrote:
> >
> > > Following up on this again, this time with tests using 2.1.90
> > > (previous tests were with 2.0.33).
> > > Rogier Wolff writes:
> > > > Richard Gooch wrote:
> > > >
> > > > > On my tests I noticed quite a spread of times. When I send short
> > > > > packets, then yes, the times are around a few hundred
> > > > > microseconds. However, for longer packets, I was getting up to 2
> > > > > milliseconds. And my measurements were consistent with the results
> > > > > other were getting with pSOS. I first instrumented almost the whole
> > > > > interrupt handler, then later just instrumented the block transfer
> > > > > routine and got the same results.
> > > >
> > > > Rule-of-thumb calculation: ISA bus cycle costs 1 us (one microsecond).
> > > > 1500 bytes -> 750 transfers (Ne2000 is 16bit card) -> 750 us.
> > > >
> > > > I'd say 2000 us is quite long, 300 us is a pretty fast.
> > > >
> > > > I guess that you can have ne2000's that are slower than others. Maybe
> > > > 300 us was a PCI card?
> > >
> > > The test was done with an ISA card. I've re-run the tests on a Pentium
> > > 200 MMX with an ISA NE2000 card (el cheapo). The 8390 interrupt
> > > service routine often takes 4 ms. The ne_block_input() function takes
> > > around 1.1 ms. This is when I'm sending a megabyte of data to the
> > > machine using TCP, so the EtherNet packets should be close to 1500
> > > bytes. The lower times would be due to shorter received packets. This
> > > machine is running 2.1.90.
> > >
> > > I've done similar tests on a dual PPro with a PCI NE2000 (also an el
> > > cheapo), and I get similar times for ne_block_input(). However the
> > > whole 8390 interrupt routine has gone down to around 2 ms. This
> > > machine is 2.1.91.
> > >
> > > On a P90 system with an ISA SMC-ULTRA I recall getting up to 2 ms per
> > > interrupt. That machine was running 2.0.33.
> > >
> >
> > I think you are measuring the time, not only for your interrupt, but
> > also for all the interrupts that happened during your ISR because the
> > interrupts were enabled.
>
> Sigh. I don't think this is a significant factor, however, I've just
> run another series of tests where I disable interrupts in
> ne_block_input() and I get times ranging from 0.5 to 1 millisecond
> (for largish packets). This is with my PCI NE2000 card on a dual PPro.

I mentioned before that your 'printer' port can be a valuable tool.
using PIOW, run 32k WORDS out your printer port (both data and control
are adjacent so you won't dork anything). Measure the time with the
cycle counter or use a scope to watch the burst-length.

You can get 32k words of anything by setting DS = CS and ESI = 0x00001000
The kernel can read its own code without an exception, and there is
at least 32k words available.

This will tell you if your bus is throwing in a lot of 'wait states'.

Then, just read and throw away the byte at the NE* status port,
immediately followed by running the same 32k WORDS out the remote-DMA
port of the chip. This will not hang the chip if you read the status
port first (interrupts MUST be disabled to prevent the chip from
interrupting and having the ISR do something while the DMA is happening).

The data will be harmlessly 'lost', but the chip will be forced to
read the data.

If the time is different, you have a very bad board.

This will test the bus interface state-machine.

PIO runs at the same speed as 'real' DMA. The only disadvantage is
that the CPU is tied up doing it. If the DMA controller did it, the
CPU could do other things then get interrupted when the DMA reached
terminal count. Note that the DMA controller could be used to write to
and read from the Ethernet chip. This is not done because no other port
of the chip can be read or written while the remote-DMA is in progress.
The Chip's bus interface state-machine does not support this and it will
hang the bus forever.

>
> > A ISR needs so many CPU cycles to complete. If the ISR gives away its
> > CPU cycles by enabling interrupts, it will take much longer to complete
> > than expected. The idea of enabling interrupts during an ISR is, in
> > principle, not a sound engineering technique and, if absolutely necessary,
> > should be justified on the basis of some special circumstances.
>
> If I have an ISR taking 1 millisecond (NE2000) and another ISR that
> comes every 1 millisecond, I think it's pretty obvious that enabling
> other interrupts during the NE2000 ISR is a damn good idea.

If the NE2 ISR takes 1 millisecond, there is a problem. In a real-mode
application, using 2 SNICS (NE2s), toggling with overlapping packets,
we obtain and MUST maintain a continuous data flow of 1.75 megabytes
per second on a RF downlink that uses these Ethernet controllers as
modulator/demodulators for our CAT Scanner marketed by Philips. The
packets arrive at 750 microsecond intervals and overlap to get nearly
twice the data throughput of a single ethernet link. The packets are
1440 WORDS (2880 bytes) in length. The ISR to handle TX/RX interrupts
gets control 1440 times per second. At the same time a trigger generator
interrupts 1440 times per second, plus there is a 'regular' Ethernet board
that interrupts whenever it feels like it. It's used for network
communication with a Sun Workstation used for the user interface. In
addition, there are interrupts from IIC Controllers and RS-232C, plus
control push-buttons, etc.

In no ISRs are interrupts ever enabled, and we don't ever lose any
interrupts.

The machine first ran on a 486/40, then a 486/DX-66. Now a 90 MHz
Pentimu since you can't get those devices any more.
[SNIPPED]
>
> Once again, no argument, but I don't see what this has to do with my
> problem either. The NE2000 isn't polling, it's sitting in a tight
> loop, in particular:
> insw(NE_BASE + NE_DATAPORT,buf,count>>1);
>
No, it's not polling. It should run at hardware speeds. You should
get out/in one WORD every 300 nanoseconds.

> I don't call that polling! What bothers me is that my colleague is
> reporting under 140 microseconds for his pSOS ISR, now that he's coded
> some of it in assembler and increased the ISA clock to 12 or 16
> MHz. And this is on a 486 DX2-66! I would expect that my PCI card with
> a PPro would be able to match that, but apparently not.
> This is something I'd like to track down.

Here is the MACRO I use to read these controllers in ping/pong. I
leave a bit of history in the header so you can understand the development
that occurred.

Modified just about everything because it was found that the SNIC
chip would seize the bus if I checked any status register during
certain (unknown) critical periods. This meant that an attempt to
determine if it was busy so I could fill the output buffer, would
sometimes halt the machine.

The "fix" was to update the chip's internal output buffer only
after a transmit packet interrupt. This would guarantee that it
would not be busy. Unfortunately, since an interrupt may occur
at any time, other portions of code might not have completed
updating the data to be transmitted before the chip interrupted
and transmitted it. This required additional redesign so that
the transmitted buffer can be marked "complete".

Modified 22-DEC-1992 Richard B. Johnson

Changed the method of maintaining the SNIC boundary pointer. It
is now permanently set to zero with the Page Start Address set at
one. This will force this pointer to always be less than the current
address.

Modified 28-DEC-1992 Richard B. Johnson

Removed above modification and fixed a bug in the initialization
that kept the data configuration register in the loopback mode.

Modified 31-JAN-1993 Richard B. Johnson
Added support to queue the error-logger if some new errors are sent
from the DEC card in the Disc Subsystem

Changed code to accommodate new structure length on the returned
data because of the redesign of the DEC card

Modified 04-FEB-1993 Richard B. Johnson

Changed down-link interrupt service routine to use a MACRO which
is optimized for speed. Modifying MACRO READ_LINK now changes the
code in both downlink channels.

Modified the method of handling the SNIC page stop parameter. How
uses a mask so a compare and jump on condition is not necessary.
this saves 12 processor clocks.

Modified 07-FEB-1993 Richard B. Johnson

Added support to extract Auxiliary DAS data from the downlink
packets.

Modified 19-FEB-1993 Richard B. Johnson

Added support to update the position system record number from
within the transmitter interrupt service routine.

Modified 21-MAY-1993 Richard B. Johnson
Added support for returned Disc Subsystem errors.

Modified 02-MAR-1995 Richard B. Johnson
Added support for conditional compilation to detect sequence number
errors
*

INT_USE EQU IRQ9 ; Interrupt we are using
BASE EQU 200H ; Base address of the board
DNLINK0 EQU BASE ; Downlink port
DNLINK1 EQU BASE + 10H ; Downlink port
UPLINK EQU BASE + 20H ; Uplink port
UPDATA EQU BASE + 32H ; Data port for uplink
ADD_LEN EQU SIZE DISC_DEST ; Bytes in an ethernet address
WRAP EQU 0FFFH ; Max offset in Global Table
SNIC_ADDS EQU 04H ; Two words
MAXBUF EQU 0000011111111110B ; Ethernet limit mask
MAXALO EQU MAXBUF + SNIC_ADDS ; Data to allocate for buffer
_PSTART EQU 1 ; Starting page address
_PSTOP EQU 20H ; End of RAM 32
_PMASK EQU _PSTOP - 1 ; Mask for boundry pointer
_BNDRY EQU _PSTART ; Boundary pointer
_CURR EQU _PSTART ; Current starting page
_NEXT_PKT EQU _PSTART ; Next packet address
UL$FAIL0 EQU 00000001B ; Uplink failed
DL$FAIL0 EQU 00000010B ; Downlink 0 failed
DL$FAIL1 EQU 00000100B ; Downlink 1 failed
SN$DMABT EQU 00001000B ; DMA aborted
LN$SEQNM EQU 00010000B ; Sequence number error
KEY_LEN EQU 12 ; Length of key-word data
;
OUT_SNC MACRO ADDR, VALUE
MOV DX,BX ; Get base address
IF VALUE ; If not zero
MOV AL,VALUE ; Number into AL
ELSE
XOR AL,AL ; Clear AL
ENDIF
IF ADDR ; If not zero
ADD DX,ADDR ; Add in offset address
ENDIF
OUT DX,AL ; Output to port
ENDM
;-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
;
; Used to read Serial Network Controller ports.
;
IN_SNC MACRO ADDR
MOV DX,BX ; Get base address
ADD DX,ADDR ; Add in offset address
IN AL,DX ; Get port data
ENDM
;-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
; This reads and filters data from the downlink Serial Network
; Controllers. N is the channel, 0 or 1, and M is the other, i.e.,
; 1 or 0.
;
READ_LINK MACRO N
LOCAL DMA_OK, STA_OK, RX_BAD, MAXBF, IMAGE, BNDRY, BNDOK, JUNK
;
MOV AL,RD_SEL&N ; Select device 'N'
MOV DX,BASE.RDPCA ; Remote DMA path control port
OUT DX,AL ; Select it
;
; To avoid hanging the system, we must check if the previous remote
; DMA operation has completed. If it has not completed, we will
; abort it.
;
MOV DX,DNLINK&N.CMD ; Pick up command register
IN AL,DX ; See if DMA is complete
FLUSH ; Chip-to-chip select time bug
AND AL,CR_RD2 ; Check abort/complete DMA
JNZ DMA_OK ; It has completed
MOV AL,CR_RD2 ; Abort complete DMA
OUT DX,AL ; Tell controller
FLUSH ; Chip-to-chip select time bug
MOV DX,BASE.RREAD ; Remote read port
IN AX,DX ; Reset PRQ (Note 1, 10.0, pp 1-17)
;
MOV AL,(NOT RD_SEL&N) ; Select device NOT 'N'
MOV DX,BASE.RDPCA ; Remote DMA path control port
OUT DX,AL ; Select it
NOT AL ; Toggle the bit
OUT DX,AL ; Select it
OR BYTE PTR LNK_FAIL, SN$DMABT ; Show DMA was aborted
;
; DMA problem has been fixed. How check the interrupt status register
; and handle either the error or the new data in the SNIC.
;
DMA_OK: MOV DX,DNLINK&N.ISR ; Downlink interrupt status port
IN AL,DX ; Get status
OUT DX,AL ; Clear status register
TEST AL,IS_PRX ; Was packet received?
JZ RX_BAD ; No
;
; Data was received, now remove our private data from the SNIC then
; send the image data to the FIFO.
;
MOV BYTE PTR _chmon&N,TRUE ; Show channel is alive
ADD WORD PTR _recvs&N.LW.LO,1 ; Increment receiver count
ADC WORD PTR _recvs&N.LW.HI,0 ; Take care of overflow
;
MOV DX,DNLINK&N.RSR ; Get downlink receive status
IN AL,DX ; Get status
MOV CL,AL ; Save receive status
;
XOR AL,AL ; Clear low byte
MOV AH,BYTE PTR NPK&N ; Address of current packet
MOV WORD PTR CURPK,AX ; Save current packet address
MOV DX,DNLINK&N.RSAR0 ; Remote start address port
OUT DX,AX ; Set it
FLUSH ; Chip-to-chip select time bug
;
MOV AX,4 ; Two words
MOV DX,DNLINK&N.RBCR0 ; Remote byte count register
OUT DX,AX ; Set remote byte-count
FLUSH ; Chip-to-chip select time bug
;
MOV AL,CR_GET ; Remote read
MOV DX,DNLINK&N.CMD ; Pick up command register
OUT DX,AL ; Set remote read
FLUSH ; Chip-to-chip select time bug
;
MOV DX,BASE.RREAD ; Remote read port
IN AX,DX ; Get first word (status)
CMP CL,AL ; Check the status
JZ STA_OK ; Status was okay
IN AX,DX ; Get next word to complete DMA
;
RX_BAD: OR BYTE PTR LNK_FAIL, DL$FAIL&N ; Show downlink failure
MOV DX,DS ; Save segment
LDS DI,DWORD PTR DGROUP:_history ; Get History structure
ADD WORD PTR [DI.DNLNK&N&_ERRORSL],1 ; Accumulate errors
ADC WORD PTR [DI.DNLNK&N&_ERRORSH],0 ; Propagate overflow
MOV DS,DX ; Restore segment
RET
;
STA_OK: MOV BYTE PTR NPK&N,AH ; Set next packet address
IN AX,DX ; Get byte count
AND AX,MAXBUF ; Mask byte count limit
MOV WORD PTR _dlink_len&N,AX ; Save the length
SUB AX,4 ; What we have already read
MOV CX,AX ; Set byte count in counter register
MOV DX,DNLINK&N.RBCR0 ; Remote byte count register
OUT DX,AX ; Set byte count
FLUSH ; Chip-to-chip select time bug
;
MOV DX,DNLINK&N.CMD ; Pick up command register
MOV AL,CR_GET ; Remote read
OUT DX,AL ; Set remote read
FLUSH ; Chip-to-chip select time bug
;
MOV DI,OFFSET DGROUP:_dnlink_buf&N ; Where to put the data
MOV AX,CX ; Save the byte count
MOV CX,(DASM_AUX SHR 1) ; Up to, but not including (WORDS)
MOV DX,BASE.RREAD ; Remote read port (where to get data)
REP INSW ; Read the data
MOV CX,AX ; Restore byte count
SUB CX,DASM_AUX ; CX = rest of the data
; Check data type
CMP BYTE PTR _dnlink_buf&N.DISC_MSG_TYP,DISC_NORM
JZ IMAGE ; It's image data
SHR CX,1 ; Make a word count
REP INSW ; Read the rest of the data
JMP BNDRY ; Do the boundary pointer

JUNK: MOV AL,CR_RD2 ; Abort/complete remote DMA
MOV DX,DNLINK&N.CMD ; Pick up command register
OUT DX,AL ; Tell controller
FLUSH ; Chip-to-chip select time bug
MOV DX,BASE.RREAD ; Remote read port
IN AX,DX ; Reset PRQ (Note 1, 10.0, pp 1-17)
JMP SHORT BNDRY ; Do the boundary pointer
;
; The data is image data. Extract the Auxiliary DAS data that we need.
; Note that we do not use a multiply within the ISR. That takes 21
; clocks. Instead, we index into a table of fixed offsets.
;
IMAGE: TEST CX,0FF00H ; Make sure it's real
JZ JUNK ; It's junk
IN AX,DX ; Get REV_NUMBER
IN AX,DX ; Get KEY_WORD
AND AX,7 ; Mask off junk
MOV CX,BX ; Save nonvolatile register
MOV BX,OFFSET DGROUP:KEY_INDEX ; Set offset (2 clocks)
XLAT ; Get offset (4 clocks)
MOV BX,CX ; Restore old BX register
MOV DI,OFFSET DGROUP:_dasd ; Where the data starts
ADD DI,AX ; New offset
MOV CX,(KEY_LEN SHR 1) ; Number of WORDS per record
REP INSW ; Get the AUX data
;
; Aux data has been received. We have to abort/complete the remote
; DMA operation so we can start again for the FIFO
;
MOV AL,CR_RD2 ; Abort/complete remote DMA
MOV DX,DNLINK&N.CMD ; Pick up command register
OUT DX,AL ; Tell controller
FLUSH ; Chip-to-chip select time bug
MOV DX,BASE.RREAD ; Remote read port
IN AX,DX ; Reset PRQ (Note 1, 10.0, pp 1-17)
;
; Now set up remote DMA so FIFO can get its data.
;
MOV AX,WORD PTR CURPK ; Get current packet address
ADD AX,DASM_AUX ; Set up new offset
MOV DX,DNLINK&N.RSAR0 ; Remote start address port
OUT DX,AX ; Set it
FLUSH ; Chip-to-chip select time bug
;
MOV AX,WORD PTR _dlink_len&N ; Get the length
SUB AX,DASM_AUX ; Get rid of stuff we don't want
MOV DX,DNLINK&N.RBCR0 ; Remote byte count register
OUT DX,AX ; Set byte count
FLUSH ; Chip-to-chip select time bug
;
MOV AL,CR_GET ; Remote read
MOV DX,DNLINK&N.CMD ; Pick up command register
OUT DX,AL ; Set remote read
FLUSH ; Chip-to-chip select time bug
;
MOV AL,(NOT RD_SEL&N) ; Select alternate device, signal FIFO
MOV DX,BASE.RDPCA ; Remote DMA path control
OUT DX,AL ; Select it
;
; FIFO has been signaled. Now check for any change in the error status
; word and queue the error logger if it has changed since the last time
; it was read.
;
CMP BYTE PTR _ctx.EBLOCK.DISC_HEAD.DISC_MSG_TYP,DISC_NORM
JNZ BNDRY ; Not normal control messages
MOV AX,WORD PTR _dnlink_buf&N.DASM_STAT
XOR DX,DX ; Clear the mask
CMP AX,1 ; Sets CY if it's 0
SBB DX,0 ; Makes DX = 0FFFFH if CY is set
AND WORD PTR _ctx.EBLOCK.DISC_HVPS_VOLT,DX
AND WORD PTR _ctx.EBLOCK.DISC_XRAY_FCUR,DX
OR AX,AX ; Any errors?
JZ BNDRY ; No
OR WORD PTR LAST_STAT,AX ; Set new status
OR WORD PTR _ctx.ENQUE_LOW, TASK0C ; Queue the error logger
;
; Set the boundary pointer for the next packet reception. Note that
; this uses auto-wrap of the ring buffer. We are using page 0 of
; the SNIC and not throwing it away as the documentation advises.
;
BNDRY: MOV AL,BYTE PTR NPK&N ; Address of next packet
DEC AL ; BNDRY = next_pkt - 1
AND AL,_PMASK ; Auto wrap
MOV DX,DNLINK&N.BNDRY ; Get boundary pointer register
OUT DX,AL ; Set new value
;
; Record any new Disc RF Link errors that have been sent on the
; downlink packet.
;
MOV AL,BYTE PTR _dnlink_buf&N.DISC_MSG_ERR
MOV AH,AL ; Copy error count
SUB AL,BYTE PTR _last_dsc ; Get last error count
MOV BYTE PTR _last_dsc,AH ; Set new value
XOR AH,AH ; Clear high byte
;
MOV DX,DS ; Save segment
LDS DI,DWORD PTR DGROUP:_history ; Get History structure
ADD WORD PTR [DI.DISCLK_ERRORSL],AX ; Accumulate errors
ADC WORD PTR [DI.DISCLK_ERRORSL],0 ; Propagate overflow
MOV DS,DX ; Restore segment
;
ENDM

Cheers,
Dick Johnson
***** FILE SYSTEM MODIFIED *****
Penguin : Linux version 2.1.92 on an i586 machine (66.15 BogoMips).
Warning : It's hard to remain at the trailing edge of technology.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu