Re: problems doing direct dma from a pci device to pci-e device

[Mark Hounschell]

Mark Hounschell wrote:
> Alan Cox wrote:
>>> I now have an AM3 based DFI DK 790FXB-M3H5 motherboard. This board has 3 regular
>>> PCI slots and 3 PCI-E (16x) slots. I also have a PCI-E (x4) version of the VMIC-5565
>>> reflective memory  card in one of the PCI-E slots and our gpiohsd card in one of the regular
>>> PCI slots. All on the motherboard. No expansion slots being used. However I cannot get
>>> data from our gpiohsd into the PCI-E VMIC-5565 cards memory. I can certainly get the data there
>>> from a userland buffer, no problem. Just not from one card to the other directly. Oh and when
>>> I put the regular PCI version of the VMIC into one of the regular PCI slots everything works
>>> as expected. They are then both on the same PCI bus and no bridges are involved though.
>> Have you verified with the vendor that such DMA works properly ? There is
>> a long history of there being boards where some device to device DMA
>> exploded or vanished. The arrival of PCI capture cards doing direct to
>> video DMA cleaned the world up (eg the BT848) but I wouldn't be suprised
>> if this recurred somewhere since they were popular and nobody really
>> noticed as they didn't run such an unusual config.
>>
> 
> I have not made an inquiry with the vendor yet. I'm pretty sure I have done 
> this with this very card in another mother board. I guess I need to dig that one 
> up and verify it before I go much further.  
> 
> But it's much like you describe above. The data seems to just vanish. I can't
> see any way to use my v-metro to find out where it's going either.
> 
>> Also does the board have a true IOMMU in the PCI-E side of the system ?
>> It's not a chipset I know.
>>
> 
> That I don't know for sure but I thought they all did??
> 

A quick readup on the IOMMU makes me think that if it does use one, that could certainly
cause my problem. I'm using bus addresses plugged into page tables that live in the gpiohsd card
obtained from virt2bus. They do match what lcpsi says they are. But it would appear from what
I read, that an IOMMU, could still be causing me grief if there is one being used. 
   
> I'll verify that the VMIC card is in fact capable in that other MB before
> I do anything else.

I still haven't done this yet. Maybe the MB this all worked on didn't have an IOMMU though?

I'll investigate futher.

Regards
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/