Re: [PATCH 2.6.26] PCI: refuse to re-add a device to a bus upon pci_scan_child_bus()

From: Eran Liberty
Date: Tue Jul 22 2008 - 09:18:01 EST


WOW, sorry for the html tags (though I had this Thunderbird under control). Here is the same mail without html tags (hopeully)


Matthew,
You seem to have a finer grasp of the subject then I do, please correct/educate me on any of the points I raise in the following lines.

Matthew Wilcox wrote:

On Tue, Jul 22, 2008 at 11:21:06AM +0300, eran liberty wrote:
I think this is your real problem, that you're rescanning the entire
bus. I don't think that's the route we'd recommend taking.
My stating point was that I have loaded a new design into a
programmable device which sits on the pci device. The new design can
implement numerous pci devices or non at all. I can think of an easy
way (or clean one) to scan only the programmable device. Scanning the
whole bus seemed reasonable.
That's what pci_scan_slot() is for. It scans the first function at the
device number, then (if the header indicates it's a multifunction
device) scans the other functions associated with that device. eg you
could call pci_scan_slot(bus, 0x30) and it will create function 06.0
(and potentially 06.1, 06.2, ...)
You presumably already have the devfn for the existing device since
you're able to call pci_remove_bus_device().
Each slot represent a single device which can have more then one function. pci_scan_slot is aimed for scanning these multiple functions.
I, on the other hand, have programmable device on the pci bus which is, for the sake of this discussion, a complete black box.
This black box up on loading can implement more then one device, which can have more then one function each.
So as far as I see it, now I need to scan all slots on the bus.

But to be honest, upon looking a way to make my device work I dismissed the "pci_scan_slot()" option as It did not reach the "fixup_resource ()" part.

Why don't you call pci_scan_slot() instead? You won't get the benefit of
pcibios_fixup_bus(), but I'm not convinced that's safe to call on a bus
that's already been scanned.
As said its not exactly a slot its more like a regular pci device that
someone suddenly welded into the pci bus. Its not a hotplug as well,
and I do not want to give up on the pcibios_fixup_bus()

Why not? What architecture are you using? What does
pcibios_fixup_bus() do for you?
I work with ARCH=powerpc. pcibios_fixup_bus() will deal with all the resource bars allocation.
I needed Linux to renegotiate the resources bars on the PCI devices.

(as a side-note, I'd like to reimplement the pcibios_fixup_*() routines;
I think a lot of what they do can be done more generically these days.
It'll take a while and isn't high on my priority list).
If I can lend a hand there, let me know and I will try to squeeze it in somewhere.

As it is, with my patch applied i successfully go over the bus and
remove my own devices before I reprogram the
programmable device.

while ((dev = pci_get_device(PCI_VENDOR_ID_MYCOMP,PCI_DEVICE_ID_MYDEV,NULL))
!= NULL) {
pci_remove_bus_device(dev);
pci_dev_put(dev);
}

Load a new design into it.

Then scan the entire bus and add the newly discovered devices.

bus = null;
while ((bus = pci_find_next_bus(bus)) != NULL) {
pci_scan_child_bus(bus);
pci_bus_assign_resources(bus);
pci_bus_add_devices(bus);
}

As seen here, this sequence of instructions seems very intuitive. It
will fail without the patch upon pci_bus_add_devices().

Seems utterly unintuitive to me. You're doing a lot of unnecessary work
here, and if you have two cards in your machine, you'll take away both
of them when you reload either of them.
Hmmm, I do want to remove all the devices that are implemented by the programmable unit which is reloaded.
I have not considered the possibility of having more then one programmable unit.
I guess that the removing part can be more fine tuned as the need arises.

What you should do is cache the pci_bus and the devfn at startup:

static struct pci_bus *my_bus;
static int my_devfn;

struct pci_dev *dev = pci_get_device(PCI_VENDOR_ID_MYCOMP,
PCI_DEVICE_ID_MYDEV, NULL);
if (!dev)
return -ENODEV;
my_bus = dev->bus;
my_devfn = dev->devfn;
pci_dev_put(dev);

when you want to remove it:

for (func = 0; func < 8; func++)
struct pci_dev *dev = pci_get_slot(my_bus, my_devfn + func);
if (!dev)
continue;
pci_remove_bus_device(dev);
pci_dev_put(dev);
}

when you want to rescan it:

pci_scan_slot(my_bus, my_devfn);

(this only handles one programmable card. The basic idea could be
extended to handle multiple cards if you need to do that).
I think there is a hidden assumption in this code, again please correct me if I missed the point.
This code assumes that the devices which will re-appear after the programmable unit is loaded has the same devfn and bus as the devices which were present before the reload.
This assumption might be wrong.

For example, I have a basic programmable image which has no pci devices at all.
upon unloading I do not remove any device (as non are present) and up on reloading I suddenly have two. What is their bus? their devfn?

Ultimately I would have expected to find a "int pci_scan_bus(struct pci_bus *bus );" the "pci_scan_child_bus ()" was the closest to the mark

Liberty

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/