Re: Regression from 2.6.26: Hibernation (possibly suspend) brokenon Toshiba R500 (bisected)

From: Linus Torvalds
Date: Thu Dec 04 2008 - 18:23:59 EST

On Thu, 4 Dec 2008, Rafael J. Wysocki wrote:
> That said the "don't bother allocating bridging windows for transparent
> bridges" patch resulted in the following layout on my box (from /proc/iomem):
> 88000000-8807ffff : 0000:00:02.1
> 88080000-88083fff : 0000:00:1b.0
> 88080000-88083fff : ICH HD audio
> 88084000-88087fff : 0000:03:0b.1
> 88088000-88088fff : 0000:03:0b.0
> 88088000-88088fff : yenta_socket
> 88089000-880897ff : 0000:03:0b.1
> 88089000-880897ff : firewire_ohci
> 88089800-880898ff : 0000:03:0b.3
> 88089800-880898ff : mmc0
> 8808a000-8808afff : Intel Flush Page
> 8c000000-8fffffff : PCI CardBus 0000:04
> 90000000-93ffffff : PCI CardBus 0000:04
> while my "don't allocate bridging windows for cardbus bridges behind
> transparent bridges" patch I've just sent (appended for easier reference)
> results in the layout:
> 88000000-880fffff : PCI Bus 0000:03
> 88000000-88003fff : 0000:03:0b.1
> 88004000-88004fff : 0000:03:0b.0
> 88004000-88004fff : yenta_socket
> 88005000-880057ff : 0000:03:0b.1
> 88005000-880057ff : firewire_ohci
> 88005800-880058ff : 0000:03:0b.3
> 88005800-880058ff : mmc0
> 88100000-8817ffff : 0000:00:02.1
> 88180000-88183fff : 0000:00:1b.0
> 88180000-88183fff : ICH HD audio
> 88184000-88184fff : Intel Flush Page
> 8c000000-8fffffff : PCI CardBus 0000:04
> 90000000-93ffffff : PCI CardBus 0000:04

Well, this happens because in the second case, you still will allocate a
non-prefetchable window due to the _other_ devices behind the bridge (ie
the firewire and mmc device.

So with your "ignore cardbus device resources", you still do end up with a
bridge window allocated, but now it will depend entirely on what other
devices exist on that PCI bus. Which is why I really dislike that patch,
because it really makes no sense at all.

And once you've allocated the PCI bridge window, then the alignment
requirements are that you end up having at least 1MB of window, and now
because a window exists, and will fit, it will then happily put the yenta
control mappings into that window even though it didn't size it for them.


Also, notice how it doesn't put the actual cardbus bridge windows
themselves into that PCI bridge window, because they won't fit. So these

8c000000-8fffffff : PCI CardBus 0000:04
90000000-93ffffff : PCI CardBus 0000:04

are outside the window, even though that bus is topologically inside the
same bus, and they both end up depending on the fact that the PCI bus
controller is transparent.

So the above results are fairly easy to explain.

The _ordering_ difference comes simply from the allocation order. We'll do
bus allocations first (if we do them), which is why _if_ we allocate a
window for that PCI bus 03 (ie the one that is bridged to by 0:1e.0, that
transparent bridge), then we'll end up allocating it first. So that's why
the PCI bridge window shows up at 0x88000000, if it shows up at all
(because that's the starting address for PCI allocations).

Then, we'll do the regular devices in the order we found them, so then
we'll allocate the resources for device 0:02.1 and then 0:1b.0. Now, _if_
we did a bus window allocation, they'll end up being after the bus window
we allocated. Otherwise they'll end up being the first allocations.

So then, by the time we actually get to bus#3, and devices 3:0b.*, where
they end up is going to depend on whether we allocated that bus window (in
which case they'll preferentially get allocated inside the window - ie
starting at 0x88000000), or they'll just get allocated inside the parent
(root) PCI bus. In the latter case they'll be allocated after the 0:02.1
etc devices.

So the differences in ordering really do make sense, and are a direct
consequence of whether we decided to need to allocate a bridging window
for that PCI-PCI bridge at 0:1e.0.

Also note that _if_ PCI bus #3 had had other devices with prefetchable
memory resources, then we'd have allocated a prefetchable window for
those even with your patch, and then we'd have been back to the original
allocation again (although sizing might cause changes, since your patch
will make us ignore the cardbus controlle for sizing).

> where devices behind the transparent bridge (PCI Bus 0000:03) are located
> _before_ ICH HD audio in the memory address space, and this one appears to
> work. So there _may_ be an effect of the layout too.

I do agree that your patch will affect layout. I just don't think it makes
any real amount of sense because of how it essentially does so "randomly"
depending on what other devices you'd have behind the transparent bridge.

Which is why I think we should either not size transparent bridges at all,
or we should size _all_ the devices behind them.

Oh, btw, I thought your and Frans' laptops were identical? They don't seem
to be: in Frans' case, the firmware does seem to set up one memory window,
so he gets that one even with my "don't size anything" patch, just because
we'll still honor allocations done by firmware.

> Well, given that both affected boxes have the same chipset (945GM), I seriously
> suspect a nastiness in that chipset we're not aware of. Especially that
> the problem is not reproducible without snd_hda_intel (at least on my box).

Well, the counter-argument to that is that the 945GM should be a _very_
common chipset. So I'd not expect a chipset nastiness.

I'd be more wary about the BIOS doing something odd. Or an ACPI thing.

> I've already checked the resume ordering of PCI devices on my box and it
> is the following:
> [snip]
> So, snd_hda_intel resumes before all of the bridges and the layout of devices
> _behind_ the transparent bridge shouldn't affect it at all.

Yes. And in the case of Frans' machine, the e1000e controller was before
all the bridges too.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at