Re: [PATCH] intel-iommu: Default to non-coherent for domainsunattached to iommus

[David Woodhouse]

On Fri, 2011-11-11 at 16:51 -0800, Chris Wright wrote:
> * Roland Dreier (roland@xxxxxxxxxxxxxxx) wrote:
> > On Fri, Nov 11, 2011 at 4:37 PM, David Woodhouse <dwmw2@xxxxxxxxxxxxx> wrote:
> > > This brain-damage only affects the first chipsets
> > > from before we worked out that cache incoherency was a *really* f*cking
> > > stupid idea, doesn't it?
> > 
> > As we talked about at KS, I have some Westmere EP (ie latest
> > and greatest server platform) systems where the BIOS exposes
> > an option that allows choosing VT-d coherency on or off, and
> > defaults it to "off".
> 
> That's just more brain damage AFAICT.  Esp. if you do performance
> testing (and choose not to use passthrough mode)... have and it's
> quite measurable.  I switched default to on long time ago, w/out
> issue.
> 
> > What is the "official" Intel line on coherency with Westmere and
> > Tylersburg -- because as I also mentioned, I was seeing some
> > problems with VT-d and the default "coherency off" setting that
> > looked like the IOMMU HW is getting stale PTEs (ie a missing
> > or not working cache flush).
> 
> That sounds like sw bugs more than official recommendation issue.

Although the cache-flushing has been tested on the original chipsets
fairly well, and it's one of the parts I've mostly rewritten when doing
performance work since I inherited the code, so that might not be my
first suspicion. I would be more inclined to suspect that there's some
chipset buffering that we aren't correctly flushing (which might in
itself be a hardware issue, since the way to flush the cache is supposed
to be well-defined).

-- 
dwmw2

Attachment: smime.p7s
Description: S/MIME cryptographic signature