Re: [bisected][resend] pnp: Huge number of "io resource overlap" messages

Previous thread: Feature Request by l5ynlwlcyku9kvaqc2jf.j.HadVabVobs on Tuesday, September 9, 2008 - 2:49 am. (1 message)

Next thread: [GIT PULL] 2.6.27-rc5 updates for s390 by Martin Schwidefsky on Tuesday, September 9, 2008 - 3:51 am. (1 message)
From: Frans Pop
Date: Tuesday, September 9, 2008 - 3:50 am

The only thing I used to get for pnp 00:08 on my Toshiba Satellite A40
up to 2.6.26 was this single line:
pnp 00:08: can't add resource for IO 0xa8-0xa9

During bisecting I have found that fairly early in the 2.6.27 cycle this
was "fixed" and that message disappeared. The commit that changed this was:
commit aee3ad815dd291a7193ab01da0f1a30c84d00061
Author: Bjorn Helgaas <bjorn.helgaas@hp.com>
Date:   Fri Jun 27 16:56:57 2008 -0600
    PNP: replace pnp_resource_table with dynamically allocated resources

Was it expected that that change could lead to a message disappearing?
I don't really read that from the commit description.


So far for the intro, now the issue (regression?) that prompted this mail.

Now with 2.6.27-rc4/5 I'm suddenly getting a total of 78 (!) warnings
about "io resource overlaps" for pnp 00:08 in my dmesg, even though
AFAIK those have never caused any trouble.

Bisection identified this commit as the cause:
commit 999ed65ad12e374d7445fbc13f5a1d146ae4b0da
Author: Rene Herman <rene.herman@gmail.com>
Date:   Fri Jul 25 19:44:47 2008 -0700
    pnp: have quirk_system_pci_resources() include io resources

The PCI devices mentioned in the messages (1f.5 and 1f.6) are the ICH4
AC'97 audio controller and AC'97 (software) modem. At least sound works
fine _without_ this change; I don't really use the softmodem.
I'm not completely sure what pnp 00:08 is. Seems to have the "system"
driver.

Anyway, I wonder if this patch is really desirable as a general check.

Full dmesg and kernel config attached.

Cheers,
FJP

From: Rene Herman
Date: Tuesday, September 9, 2008 - 4:22 am

It's not just a check, and not just general...

Generally, you need it -- if PnP grabs an I/O resource, PCI can no 
longer do so (making the driver fail) which is the same problem that 
quirk_system_pci_resources() upto that point solved for mem resources only.

And specifically, I definitely need it to not have my soundcard driver 
crap out due to PnPACPI grabbing a range that overlaps with its BAR.

I don't know why your 1f.5 and 1f.6 are grabbing the "motherboard I/O 
ports" 0x00-0xff (with your BIOS also advertising those ports through 
ACPI) but obviously, 78 messages are not something to put up with.

Bjorn might have something more to say about the general setup of things 
here but at this point and for now it might make most sense to just go 
ahead and do our doings without noting that we do. Ie, just delete the 
message...

Rene.
From: Bjorn Helgaas
Date: Tuesday, September 9, 2008 - 8:30 am

2.6.26 supported up to 40 I/O resources for PNP devices.  If a
device had more than 40, we complained once and silently ignored

Yep.  This commit removed the fixed limit (40), so we shouldn't see
messages like that any more.  The commit log mentioned these:

        pnpacpi: exceeded the max number of IO resources
        00:01: too many I/O port resources


78 messages is certainly intolerable.  If the PNP quirk ignored
PNP resources that had already been disabled, I think we'd generate

pnp 00:08 is a "motherboard" device the describes a lot of the
legacy hardware (DMA controllers, timers, keyboard, etc) on the
system board.  The "system" driver reserves those resources to
prevent us from placing anything else on top of them.

What's curious about this to me is that those BARs just look wrong:

  0000:00:1f.5 BAR 0 (0x0-0xff)
  0000:00:1f.5 BAR 1 (0x0-0x3f)
  0000:00:1f.6 BAR 0 (0x0-0xff)
  0000:00:1f.6 BAR 1 (0x0-0x7f)

Those seem like they'd be right on top of all those motherboard devices
(and each other).

I have some similar devices, with I/O BARs of the same sizes as
yours, but they have reasonable values and /proc/iomem and
/proc/ioports show sensible things:

  00:1e.2 Multimedia audio controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) AC'97 Audio Controller (rev 03)
        Region 0: I/O ports at 3100 [size=256]
        Region 1: I/O ports at 3200 [size=64]
        Region 2: Memory at c8c01000 (32-bit, non-prefetchable) [size=512]
        Region 3: Memory at c8c02000 (32-bit, non-prefetchable) [size=256]

  00:1e.3 Modem: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) AC'97 Modem Controller (rev 03) (prog-if 00 [Generic])
        Region 0: I/O ports at 3400 [size=256]
        Region 1: I/O ports at 3500 [size=128]

  $ cat /proc/ioports 
  ...
  3100-31ff : 0000:00:1e.2
    3100-31ff : Intel ICH6
  3200-323f : 0000:00:1e.2
    3200-323f : Intel ICH6
  3400-34ff : 0000:00:1e.3
  3500-357f : 0000:00:1e.3

  $ cat ...
From: Frans Pop
Date: Tuesday, September 9, 2008 - 9:26 am

Thanks for your quick replies Bjorn and Rene!


Right, that explains. Thanks.
The old message text led me to think that only that particular resource 
was problematic, while actually it was the listed one and all following. 

Thanks for the background info and data for comparison.
Here's what I have (full info attached). AFAICT it looks equally sane and 
consistent.

00:1f.5 Multimedia audio controller: Intel Corporation 82801DB/DBL/DBM 
(ICH4/ICH4-L/ICH4-M) AC'97 Audio Controller [8086:24c5] (rev 03)
        Region 0: I/O ports at 1000 [size=256]
        Region 1: I/O ports at 1880 [size=64]
        Region 2: Memory at 28080800 (32-bit, non-prefetchable) [size=512]
        Region 3: Memory at 28080a00 (32-bit, non-prefetchable) [size=256]

00:1f.6 Modem: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) 
AC'97 Modem Controller [8086:24c6] (rev 03)
        Region 0: I/O ports at 1400 [size=256]
        Region 1: I/O ports at 1800 [size=128]

/proc/ioports:
1000-10ff : 0000:00:1f.5
  1000-10ff : Intel 82801DB-ICH4
1400-14ff : 0000:00:1f.6
  1400-14ff : Intel 82801DB-ICH4 Modem
1800-187f : 0000:00:1f.6
  1800-187f : Intel 82801DB-ICH4 Modem
1880-18bf : 0000:00:1f.5
  1880-18bf : Intel 82801DB-ICH4

/proc/iomem:
28080800-280809ff : 0000:00:1f.5
  28080800-280809ff : Intel 82801DB-ICH4
28080a00-28080aff : 0000:00:1f.5
  28080a00-28080aff : Intel 82801DB-ICH4

Cheers,
FJP

From: Bjorn Helgaas
Date: Tuesday, September 9, 2008 - 10:40 am

Yup, this all looks reasonable to me, too.  But these regions must be
different than they were when the PNP quirk ran.  I wonder if the BIOS
left them unprogrammed, we ran the PNP quirk and discovered all these
"conflicts," then PCI came along and assigned resources.

Your dmesg shows power state changes for the PCI devices.  Maybe
the BIOS left them in D3 and unprogrammed:

  Intel ICH Modem 0000:00:1f.6: power state changed by ACPI to D0
  Intel ICH 0000:00:1f.5: power state changed by ACPI to D0
  Intel ICH 0000:00:1f.5: enabling device (0000 -> 0003)

If the PCI device isn't fully initialized, it doesn't seem right to
check it for resource conflicts.  But I don't know how to tell that.

--

From: Rene Herman
Date: Tuesday, September 9, 2008 - 11:31 am

His pci_resource_start() values are 0. How about just checking for that?

Rene.
From: Bjorn Helgaas
Date: Wednesday, September 17, 2008 - 10:10 pm

Frans, can you test Rene's patch?  I think it will solve the problem
you're seeing, and if so, we should put it in for 2.6.27.  But I'd like
to have your "Tested-by" first.

(Jesse, Rene's patch just skips the PNP/PCI resource conflict check
for things with pci_resource_start() == 0.)

Thanks,
  Bjorn
--

From: Frans Pop
Date: Saturday, September 20, 2008 - 4:49 pm

Tested against current git (v2.6.27-rc6-158-g9824b8f) and looks good.
Attached the patch with Rene's Signed-off and my Tested-by for 
convenience.

I had not tested earlier as you said you wanted to better understand the 
cause first. Did you get anything more about why things happen as they do 
from the info I sent?

Attached also a dmesg diff for boots with the two different BIOS settings 
with this kernel (with Rene's patch having filtered out the "io resource 
overlap" messages).
It still clearly shows the difference in how some devices get enabled and 
how their resources get assigned.

Cheers,
FJP

From: Bjorn Helgaas
Date: Saturday, September 20, 2008 - 4:56 pm

Thanks for testing this.

We're looking at some other issues in the same area, or at least,
where the fix might be in the same area:
  http://bugzilla.kernel.org/show_bug.cgi?id=10231
  http://bugzilla.kernel.org/show_bug.cgi?id=9904

I am still not 100% comfortable with this because I think we really
want to know whether the BAR value is zero, not whether the CPU
address is zero, and pci_resource_start() gives us the CPU address.

Bus and CPU addresses are currently identical on x86, but I expect
that will change someday.  They're already different on ia64 and
some other architectures.

Bottom line, I think we should tweak the patch to check the BAR
address before we put it in.

Bjorn

--

From: Bjorn Helgaas
Date: Friday, September 26, 2008 - 2:40 pm

http://bugzilla.kernel.org/show_bug.cgi?id=11550


The problem seems to be that Frans has some PCI devices that are not
configured by the BIOS, and their BARs contain zero.  A PNP quirk
checks for overlaps of PCI devices and PNP devices, and those zero-
valued BARs of course conflict with the PNP motherboard devices that
describe legacy hardware.

Here's another approach based on section 3.5 of the PCI Firmware spec.
It says:

  Since not all devices may be configured prior to the operating
  system handoff, the operating system needs to know whether a
  specific BAR register has been configured by firmware. The operating
  system makes the determination by checking the I/O Enable, and
  Memory Enable bits in the device's command register, and Expansion
  ROM BAR enable bits. If the enable bit is set, then the corresponding
  resource register has been configured.

So instead of checking whether the BAR contains zero, the patch below
checks the I/O, Mem, and ROM BAR enable bits to determine whether a
BAR is enabled.

Frans, I'm sorry to trouble you again, but could you test this and
make sure it takes care of the "resource overlap" messages you saw?

diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 1a5fc83..26195c3 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -26,6 +26,28 @@
 #include "pci.h"
 
 
+int pci_resource_enabled(struct pci_dev *dev, int bar)
+{
+	u16 command = 0;
+	u32 addr = 0;
+
+	pci_read_config_word(dev, PCI_COMMAND, &command);
+
+	if (pci_resource_flags(dev, bar) & IORESOURCE_IO)
+		return command & PCI_COMMAND_IO;
+
+	if (command & PCI_COMMAND_MEMORY) {
+		if (bar == PCI_ROM_RESOURCE) {
+			pci_read_config_dword(dev, dev->rom_base_reg, &addr);
+			return addr & PCI_ROM_ADDRESS_ENABLE;
+		}
+
+		return 1;
+	}
+
+	return 0;
+}
+
 void pci_update_resource(struct pci_dev *dev, struct resource *res, int resno)
 {
 	struct pci_bus_region region;
diff --git a/drivers/pnp/quirks.c ...
From: Frans Pop
Date: Saturday, September 27, 2008 - 8:16 am

No problem at all. Works correctly (applied on top of current git).
I don't see any unexpected changes in the dmesg output, so:

Tested-by: Frans Pop <elendil@planet.nl>

Cheers,
FJP
--

From: Ingo Molnar
Date: Saturday, September 27, 2008 - 1:53 pm

cool! Looks like a pretty significant fix, for all sorts of legacy 
devices. Worth backporting?

	Ingo
--

From: Frans Pop
Date: Wednesday, March 4, 2009 - 1:17 pm

Sorry for having to revive this old thread. In November 2008 I reported
that this issue had been solved for me as a result of 1f98757776ea, but I
now find that was due to faulty testing. (I suspect that changing the BIOS
setting that affects this issue on my Toshiba laptop only takes effect
after a cold boot, not a normal reboot.)

The problem was that with the BIOS setting for "Device config" set to
"Setup by OS", I get 78 messages like:
    pnp 00:08: io resource (0x2e-0x2f) overlaps 0000:00:1f.5 BAR 0 (0x0-0xff), disabling
    pnp 00:08: io resource (0x2e-0x2f) overlaps 0000:00:1f.6 BAR 0 (0x0-0xff), disabling

If the BIOS setting is set to "All Devices", the problem does not occur.

The origin of these messages was bisected to:
commit aee3ad815dd291a7193ab01da0f1a30c84d00061
Author: Bjorn Helgaas <bjorn.helgaas@hp.com>
Date:   Fri Jun 27 16:56:57 2008 -0600
    PNP: replace pnp_resource_table with dynamically allocated resources


Below the then proposed patch from Bjorn, rediffed against 2.6.29-rc7.
I've verified that the patch still solves the issue for me. Attached
dmesg output for 2.6.29-rc7 without and with the patch.

Bjorn, could you please consider this patch for inclusion again?

Original thread: http://marc.info/?l=linux-kernel&m=122095745403793&w=4

TIA and sorry for the confusion,
FJP


diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 32e8d88..e63f800 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -26,6 +26,28 @@
 #include "pci.h"
 
 
+int pci_resource_enabled(struct pci_dev *dev, int bar)
+{
+	u16 command = 0;
+	u32 addr = 0;
+
+	pci_read_config_word(dev, PCI_COMMAND, &command);
+
+	if (pci_resource_flags(dev, bar) & IORESOURCE_IO)
+		return command & PCI_COMMAND_IO;
+
+	if (command & PCI_COMMAND_MEMORY) {
+		if (bar == PCI_ROM_RESOURCE) {
+			pci_read_config_dword(dev, dev->rom_base_reg, &addr);
+			return addr & PCI_ROM_ADDRESS_ENABLE;
+		}
+
+		return 1;
+	}
+
+	return 0;
+}
+
 void ...
From: Bjorn Helgaas
Date: Wednesday, March 4, 2009 - 2:53 pm

Seems like we do need something, but this patch is kind of a klunky
approach, so I'd like to come up with a better proposal.  I don't
have any better ideas yet, though.



--

From: Jesse Barnes
Date: Thursday, March 19, 2009 - 7:07 pm

On Wed, 4 Mar 2009 14:53:51 -0700

Patch actually seems pretty reasonable to me, though like we discussed
at kernel summit last year, there are places where a 0 resource is
assumed to mean "not assigned".  And clearly we need to do something


-- 
Jesse Barnes, Intel Open Source Technology Center
--

From: Bjorn Helgaas
Date: Monday, March 23, 2009 - 8:46 am

IIRC, Linus complained that it was ugly and slow to do all those
config space reads, and I have to agree with him.  I'd like it
better if we had some sort of pci_dev "enabled" flag or if we could
make it so the pci_dev resources were invalid when the device is
disabled.



--

From: Frans Pop
Date: Wednesday, September 10, 2008 - 12:39 am

If the approach suggested by Rene feasible or would you still like me to 
get this info? In the last case I'll need some pointers where exactly to 
look.

Cheers,
FJP
--

From: Bjorn Helgaas
Date: Wednesday, September 10, 2008 - 2:34 pm

Checking pci_resource_start() for zero would certainly work
in your particular case.  But I don't know enough about PCI to
know whether that's always safe.

Matthew Wilcox suggested that the BARs may be zero because the BIOS
has "disabled" those two devices.  Are there any BIOS setup options
related to them?  I know you need to use at least the sound device,
so I'm not suggesting that Linux should leave it disabled;  I'm just
trying to learn more about the situation.

If I were trying to figure out where we assign resources, I'd probably
boot with "pci=earlydump" and sprinkle calls to early_dump_pci_device()
in pcibios_resource_survey(), and drill down from there.

Bjorn
--

From: Frans Pop
Date: Thursday, September 11, 2008 - 9:58 am

The BIOS setup has no options to enable/disable these devices.

The only option I can see that could be relevant is:
=CONFIGURATION=
Device Config. = [Setup by OS | All Devices]

This was set to "Setup by OS".

[me reboots with that option changed]

Hmm. If I change that option to "All devices" the "io resource overlap"

Attached are two logs with PCI debugging enabled and extra "early dumps"
included in the following places (changes in includes omitted):

+++ b/arch/x86/pci/i386.c
@@ -31,6 +31,7 @@
@@ -224,9 +225,11 @@ static int __init pcibios_assign_resources(void)
 void __init pcibios_resource_survey(void)
 {
        DBG("PCI: Allocating resources\n");
+       early_dump_pci_devices();
        pcibios_allocate_bus_resources(&pci_root_buses);
        pcibios_allocate_resources(0);
        pcibios_allocate_resources(1);
+       early_dump_pci_devices();
 }

+++ b/sound/pci/intel8x0.c
@@ -41,6 +41,7 @@
@@ -2799,6 +2800,7 @@ static int __devinit snd_intel8x0_create(struct snd_card *card,

        if ((err = pci_enable_device(pci)) < 0)
                return err;
+       early_dump_pci_devices();

        chip = kzalloc(sizeof(*chip), GFP_KERNEL);
        if (chip == NULL) {

The "Setup by OS" log does show the following _after_ the PNP quirks:
pci 0000:00:1f.5: BAR 2: got res [0x2c080800-0x2c0809ff] bus [0x2c080800-0x2c0809ff] flags 0x20200
pci 0000:00:1f.5: BAR 2: moved to bus [0x2c080800-0x2c0809ff] flags 0x20200
pci 0000:00:1f.5: BAR 0: got res [0x1000-0x10ff] bus [0x1000-0x10ff] flags 0x20101
pci 0000:00:1f.5: BAR 0: moved to bus [0x1000-0x10ff] flags 0x20101
pci 0000:00:1f.5: BAR 3: got res [0x2c080a00-0x2c080aff] bus [0x2c080a00-0x2c080aff] flags 0x20200
pci 0000:00:1f.5: BAR 3: moved to bus [0x2c080a00-0x2c080aff] flags 0x20200
pci 0000:00:1f.6: BAR 0: got res [0x1400-0x14ff] bus [0x1400-0x14ff] flags 0x20101
pci 0000:00:1f.6: BAR 0: moved to bus [0x1400-0x14ff] flags 0x20101
pci 0000:00:1f.6: BAR 1: got res [0x1800-0x187f] bus [0x1800-0x187f] flags ...
From: Frans Pop
Date: Friday, November 7, 2008 - 2:51 am

(Info below was also posted in reply to the announcement mail for .28-rc3 
(http://lkml.org/lkml/2008/11/3/201); reposting to the original thread as
there was no response yet.)


It appears there's been a fundamental change between .28-rc2 and .28-rc3
that fixes this issue. I no longer get the "io resource overlap" messages
when the BIOS is set to let PCI devices be "Set up by OS".

I now get the exact same /proc/io{mem,ports} as with the BIOS set to
"initialize all PCI devices". It very much looks as if the two devices that
caused the messages now get activated much earlier.

This shows best in a diff between the dmesgs for .28-rc2 and .28-rc3. Here are
the relevant bits for the two devices:
# I now immediately get good resources assigned:
-pci 0000:00:1f.5: reg 10 io port: [0x00-0xff]
-pci 0000:00:1f.5: reg 14 io port: [0x00-0x3f]
-pci 0000:00:1f.5: reg 18 32bit mmio: [0x000000-0x0001ff]
-pci 0000:00:1f.5: reg 1c 32bit mmio: [0x000000-0x0000ff]
+pci 0000:00:1f.5: reg 10 io port: [0xbe00-0xbeff]
+pci 0000:00:1f.5: reg 14 io port: [0xbdc0-0xbdff]
+pci 0000:00:1f.5: reg 18 32bit mmio: [0xcfdffe00-0xcfdfffff]
+pci 0000:00:1f.5: reg 1c 32bit mmio: [0xcfdffd00-0xcfdffdff]
 pci 0000:00:1f.5: PME# supported from D0 D3hot D3cold
 pci 0000:00:1f.5: PME# disabled
-pci 0000:00:1f.6: reg 10 io port: [0x00-0xff]
-pci 0000:00:1f.6: reg 14 io port: [0x00-0x7f]
+pci 0000:00:1f.6: reg 10 io port: [0xba00-0xbaff]
+pci 0000:00:1f.6: reg 14 io port: [0xb980-0xb9ff]
 pci 0000:00:1f.6: PME# supported from D0 D3hot D3cold
 pci 0000:00:1f.6: PME# disabled
[...]
# The block of 78 "io resource overlap" messages are now gone:
-pnp 00:08: io resource (0x2e-0x2f) overlaps 0000:00:1f.5 BAR 0 (0x0-0xff), disabling
-pnp 00:08: io resource (0x62-0x62) overlaps 0000:00:1f.5 BAR 0 (0x0-0xff), disabling
[75 similar messages omitted]
-pnp 00:08: io resource (0x72-0x77) overlaps 0000:00:1f.6 BAR 1 (0x0-0x7f), disabling
[... much later]
# Apparently there is no longer any need to enable the device later ...
From: Ingo Molnar
Date: Friday, November 7, 2008 - 3:00 am

Correct.

	Ingo
--

Previous thread: Feature Request by l5ynlwlcyku9kvaqc2jf.j.HadVabVobs on Tuesday, September 9, 2008 - 2:49 am. (1 message)

Next thread: [GIT PULL] 2.6.27-rc5 updates for s390 by Martin Schwidefsky on Tuesday, September 9, 2008 - 3:51 am. (1 message)