Re: [PATCH] PCI: pciehp: Prevent child devices from doing RPM on PCIe Link Down

[Kai-Heng Feng]

Hi Lukas and Ricky,

On Thu, Oct 19, 2023 at 5:35 PM Lukas Wunner <lukas@xxxxxxxxx> wrote:
>
> On Thu, Oct 19, 2023 at 01:49:50AM +0000, Ricky WU wrote:
> > [    0.267813] pci 0000:00:1c.0: [8086:a33c] type 01 class 0x060400
>
> Cannon Lake PCH Root Port
>
> > [    0.275241] pci 0000:01:00.0: [10ec:5261] type 00 class 0xff0000
> > [    0.275315] pci 0000:01:00.0: reg 0x10: [mem 0xa3b00000-0xa3b00fff]
> > [    0.275782] pci 0000:01:00.0: supports D1 D2
> > [    0.275784] pci 0000:01:00.0: PME# supported from D1 D2 D3hot D3cold
> > [    0.276490] pci 0000:00:1c.0: PCI bridge to [bus 01]
>
> Device below Root Port is initially a Realtek RTS5261 card reader.
>
> > [    0.395968] pcieport 0000:00:1c.0: PME: Signaling with IRQ 122
> > [    0.396009] pcieport 0000:00:1c.0: pciehp: Slot #8 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl+ IbPresDis- LLActRep+
>
> Root Port is hotplug-capable.
>
> > [   43.180701] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Down
> > [   43.180709] pcieport 0000:00:1c.0: pciehp: Slot(8): Card not present
> > [   44.403768] pcieport 0000:00:1c.0: pciehp: Slot(8): Card present
> > [   44.403772] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Up
> > [   44.540631] pci 0000:01:00.0: [15b7:5007] type 00 class 0x010802
>
> Card reader is unplugged and replaced by SanDisk SN530 NVMe SSD.
>
> > [   51.682628] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Down
> > [   51.716800] nvme0n1: detected capacity change from 495050752 to 0
> > [   51.793382] pcieport 0000:00:1c.0: pciehp: Slot(8): Card present
> > [   51.793392] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Up
> > [   51.928633] pci 0000:01:00.0: [10ec:5261] type 00 class 0xff0000
>
> NVMe SSD replaced by the card reader again.
>
> > [   54.872928] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Down
> > [   56.146581] pcieport 0000:00:1c.0: pciehp: Slot(8): Card present
> > [   56.146584] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Up
> > [   56.284632] pci 0000:01:00.0: [15b7:5007] type 00 class 0x010802
>
> Card reader replaced by NVMe SSD, second time.
>
> > [   60.635845] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Down
> > [   60.676842] nvme0n1: detected capacity change from 495050752 to 0
> > [   60.748953] pcieport 0000:00:1c.0: pciehp: Slot(8): Card present
> > [   60.748958] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Up
> > [   60.884619] pci 0000:01:00.0: [10ec:5261] type 00 class 0xff0000
>
> NVMe SSD replaced by the card reader, second time.
>
> > [   63.898861] pcieport 0000:00:1c.0: pciehp: Slot(8): Link Down
> > [   63.912118] BUG: unable to handle page fault for address: ffffb24d403e5010
>
> Card reader replaced with NVMe SSD, third time.
>
> So it took three tries to reproduce the page fault.
>
> Thanks for the log, the issue is a little less murky now.
> But it's still unclear what the root cause is and thus
> what the proper solution is.  I think this needs more
> in-depth debugging, see my previous e-mail.
>
> Hope that helps!  Thanks,

I think the following approach should cover all the cases?
Ricky, can you please give it a try?

diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
index d749ea8250d6..907d60587227 100644
--- a/drivers/pci/remove.c
+++ b/drivers/pci/remove.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/pci.h>
 #include <linux/module.h>
+#include <linux/pm_runtime.h>
 #include "pci.h"

 static void pci_free_resources(struct pci_dev *dev)
@@ -89,6 +90,8 @@ static void pci_remove_bus_device(struct pci_dev *dev)
        struct pci_bus *bus = dev->subordinate;
        struct pci_dev *child, *tmp;

+       pm_runtime_barrier(&dev->dev);
+
        if (bus) {
                list_for_each_entry_safe(child, tmp,
                                         &bus->devices, bus_list)


>
> Lukas