Re: [PATCH] pci: pciehp update the slot bridge res to get bigrange for pcie devices

[Bjorn Helgaas]

On Thu, 2009-10-29 at 12:28 -0700, Eric W. Biederman wrote:
> Bjorn Helgaas <bjorn.helgaas@xxxxxx> writes:
> 
> > On Thu, 2009-10-29 at 08:13 -0700, Eric W. Biederman wrote:
> >> Bjorn Helgaas <bjorn.helgaas@xxxxxx> writes:
> >> 
> >> > On Thu, 2009-10-29 at 01:16 -0700, Eric W. Biederman wrote:
> >> >> Yinghai Lu <yinghai@xxxxxxxxxx> writes:
> >> >> >
> >> >> > after closing look up the code, it looks it will not break your setup.
> >> >> >
> >> >> > 1. before the patches:
> >> >> > a. when master card is inserted, all bridge in that card will get assigned with min_size
> >> >> > b. when new cards is inserted to those slots in master card, will get assigned in the bridge size.
> >> >> >
> >> >> > 2. after the patches: v5
> >> >> > a. booted up, all leaf bridge mmio get clearred.
> >> >> > b. when master card is inserted, all bridge in that card will get assigned with min_size, and master bridge will be sum of them
> >> >> > c. when new cards is inserted to those slots in master card, will get assigned in the bridge size.
> >> >> >
> >> >> > can you check those two patches in your setup to verify it?
> >> >> 
> >> >> I have a much simpler case I will break, as I tried something similar by accident.
> >> >> 
> >> >> AMD cpu MCP55 with one pcie port setup as hotplug.
> >> >> The system only has 2GB of RAM.  So plenty of space for pcie devices.
> >> >> 
> >> >> If the firmware assigns nothing and linux at boot time assigns the pci mmio space:
> >> >> Reads from the bar of the hotplugged device work
> >> >> Writes to the bar of the hotplugged device, cause further writes to go to lala land.
> >> >> 
> >> >> So I had to have the firmware make the assignment, because only it knows the
> >> >> details of the hidden AMD bar registers for each hypertransport chain etc.
> >> >
> >> > Do you mean you had to have firmware program a hot-added device, or just
> >> > that firmware had to program the apertures of the root port that was
> >> > present at boot, even though it had no devices below it?
> >> 
> >> Firmware had to program the apertures of the root port that was present
> >> at boot, even though it had no devices below it.
> >> 
> >> > Firmware normally supplies ACPI _CRS information that tells us how it
> >> > programmed the host bridge windows.  On x86, Linux normally ignores that
> >> > and just assumes a range based on memory size.  If we paid attention to
> >> > it (as with "pci=use_crs"), it's likely that we could do a better job of
> >> > doing this setup.
> >> >
> >> > Or, of course, we could add a Linux driver that knows about "the hidden
> >> > AMD bar registers."  But I think that should be a last resort, for when
> >> > firmware supplied incorrect _CRS information.
> >> 
> >> In this case there was no ACPI, and even if there was correct _CRS information
> >> it would have said only those addresses routed to bars/apertures on the
> >> root bridge was routed to the MCP55.  So while it looked like we had gobs
> >> of unallocated space we could use.  In practice we did not.
> >
> > I know this is a hypothetical case since you don't have ACPI, but I'm
> > curious about this.
> >
> > I assume the magic AMD BARs only affect the host bridge, and that the
> > downstream root ports look like standard PCI-to-PCI bridges.  If that's
> > the case, and if we have correct descriptions of the host bridge
> > apertures, Linux should theoretically be able to do as well as firmware.
> >
> > But you seem to be suggesting that even with a correct host bridge
> > description, there's space that *looks* available but is not.  I don't
> > understand how this can be.
> 
> What I meant was simply that not all of the non-memory space was
> routed down the hypertransport chain to the mcp55.  If you have an
> accurate description of that you should be fine.

OK, yep, that makes perfect sense.  That's a good example of why I
believe we should start paying attention to the root bridge _CRS,
because that's exactly what it would tell us.

Bjorn


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/