Home » Mailing list archives » linux-kernel » 2010 » June » 21

[PATCH RESEND] (revised) Calgary: increase max PHB number

Previous thread: [PATCH] x86/mrst/pci: avoid enabling intx for msi capable devices by Jacob Pan on Monday, June 21, 2010 - 11:30 am. (3 messages)

Next thread: [PATCH 0/2] cfq: fixes to bring cfq in line with deadline performance for mid- to high-end storage by Jeff Moyer on Monday, June 21, 2010 - 12:49 pm. (19 messages)
[view original message]
From: Corinna Schultz
Subject: [PATCH RESEND] (revised) Calgary: increase max PHB number
Date: Monday, June 21, 2010 - 11:41 am

There was a small difference in 35-rc3 that prevented this patch from  
applying, so I regenerated it.

===============================

Newer systems (x3950M2) can have 48 PHBs per chassis and 8 chassis, so  
bump the
limits up and provide an explanation of the requirements for each class.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Acked-by: Muli Ben-Yehuda <muli@il.ibm.com>

--- a/arch/x86/kernel/pci_calgary_64.c     2010-06-21 10:57:15.000000000 -0700
+++ b/arch/x86/kernel/pci_calgary_64.c    2010-06-21 11:01:56.000000000 -0700
@@ -103,10 +103,15 @@
  #define PMR_SOFTSTOPFAULT      0x40000000
  #define PMR_HARDSTOP           0x20000000

-#define MAX_NUM_OF_PHBS                8 /* how many PHBs in total? */
-#define MAX_NUM_CHASSIS                8 /* max number of chassis */
-/* MAX_PHB_BUS_NUM is the maximal possible dev->bus->number */
-#define MAX_PHB_BUS_NUM                (MAX_NUM_OF_PHBS *  
MAX_NUM_CHASSIS * 2)
+/*
+ * The maximum PHB bus number.
+ * x3950M2 (rare): 8 chassis, 48 PHBs per chassis = 384
+ * x3950M2: 4 chassis, 48 PHBs per chassis        = 192
+ * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
+ * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
+ */
+#define MAX_PHB_BUS_NUM       384
+
  #define PHBS_PER_CALGARY       4

  /* register offsets in Calgary's internal register space */





--


[ message continues ]
[view original message]
From: Darrick J. Wong
Subject: Re: [PATCH RESEND] (revised) Calgary: increase max PHB number
Date: Thursday, June 24, 2010 - 2:26 pm

Newer systems (x3950M2) can have 48 PHBs per chassis and 8 chassis, so bump the
limits up and provide an explanation of the requirements for each class.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Acked-by: Muli Ben-Yehuda <muli@il.ibm.com>

diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index fb99f7e..967c646 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -103,11 +103,14 @@ int use_calgary __read_mostly = 0;
 #define PMR_SOFTSTOPFAULT	0x40000000
 #define PMR_HARDSTOP		0x20000000
 
-#define MAX_NUM_OF_PHBS		8 /* how many PHBs in total? */
-#define MAX_NUM_CHASSIS		8 /* max number of chassis */
-/* MAX_PHB_BUS_NUM is the maximal possible dev->bus->number */
-#define MAX_PHB_BUS_NUM		(MAX_NUM_OF_PHBS * MAX_NUM_CHASSIS * 2)
-#define PHBS_PER_CALGARY	4
+/*
+ * The maximum PHB bus number.
+ * x3950M2 (rare): 8 chassis, 48 PHBs per chassis = 384
+ * x3950M2: 4 chassis, 48 PHBs per chassis        = 192
+ * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
+ * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
+ */
+#define MAX_PHB_BUS_NUM		384
 
 /* register offsets in Calgary's internal register space */
 static const unsigned long tar_offsets[] = {
--


[ message continues ]
[view original message]
From: Ingo Molnar
Subject: Re: [PATCH RESEND] (revised) Calgary: increase max PHB number
Date: Friday, June 25, 2010 - 7:14 am

Doesnt build:

arch/x86/kernel/pci-calgary_64.c: In function ?calgary_locate_bbars?:
arch/x86/kernel/pci-calgary_64.c:1107: error: ?PHBS_PER_CALGARY? undeclared (first use in this function)
arch/x86/kernel/pci-calgary_64.c:1107: error: (Each undeclared identifier is reported only once
arch/x86/kernel/pci-calgary_64.c:1107: error: for each function it appears in.)

i have added back the PHBS_PER_CALGARY line - that is still needed, right?

Thanks,

	Ingo
--


[ message continues ]
[view original message]
From: tip-bot for Darrick J. Wong
Subject: [tip:x86/urgent] x86, Calgary: Increase max PHB number
Date: Friday, June 25, 2010 - 8:29 am

Commit-ID:  499a00e92dd9a75395081f595e681629eb1eebad
Gitweb:     http://git.kernel.org/tip/499a00e92dd9a75395081f595e681629eb1eebad
Author:     Darrick J. Wong <djwong@us.ibm.com>
AuthorDate: Thu, 24 Jun 2010 14:26:47 -0700
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Fri, 25 Jun 2010 16:14:58 +0200

x86, Calgary: Increase max PHB number

Newer systems (x3950M2) can have 48 PHBs per chassis and 8
chassis, so bump the limits up and provide an explanation
of the requirements for each class.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Acked-by: Muli Ben-Yehuda <muli@il.ibm.com>
Cc: Corinna Schultz <cschultz@linux.vnet.ibm.com>
Cc: <stable@kernel.org>
LKML-Reference: <20100624212647.GI15515@tux1.beaverton.ibm.com>
[ v2: Fixed build bug, added back PHBS_PER_CALGARY == 4 ]
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 arch/x86/kernel/pci-calgary_64.c |   15 ++++++++++-----
 1 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index fb99f7e..0b96b55 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -103,11 +103,16 @@ int use_calgary __read_mostly = 0;
 #define PMR_SOFTSTOPFAULT	0x40000000
 #define PMR_HARDSTOP		0x20000000
 
-#define MAX_NUM_OF_PHBS		8 /* how many PHBs in total? */
-#define MAX_NUM_CHASSIS		8 /* max number of chassis */
-/* MAX_PHB_BUS_NUM is the maximal possible dev->bus->number */
-#define MAX_PHB_BUS_NUM		(MAX_NUM_OF_PHBS * MAX_NUM_CHASSIS * 2)
-#define PHBS_PER_CALGARY	4
+/*
+ * The maximum PHB bus number.
+ * x3950M2 (rare): 8 chassis, 48 PHBs per chassis = 384
+ * x3950M2: 4 chassis, 48 PHBs per chassis        = 192
+ * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
+ * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
+ */
+#define MAX_PHB_BUS_NUM		384
+
+#define PHBS_PER_CALGARY	  4
 
 /* register offsets in Calgary's internal register space */
 static const unsigned long tar_offsets[] = {
--


[ message continues ]
[view original message]
From: Andrew Morton
Subject: Re: [tip:x86/urgent] x86, Calgary: Increase max PHB number
Date: Tuesday, June 29, 2010 - 3:51 pm

On Fri, 25 Jun 2010 15:29:50 GMT

arch/x86/kernel/pci-calgary_64.c: In function 'calgary_init_one':
arch/x86/kernel/pci-calgary_64.c:1059: warning: comparison is always false due to limited range of data type

from

        BUG_ON(dev->bus->number >= MAX_PHB_BUS_NUM);

with

http://userweb.kernel.org/~akpm/stuff/config-akpm2.txt
--


[ message continues ]
[view original message]
From: H. Peter Anvin
Subject: Re: [tip:x86/urgent] x86, Calgary: Increase max PHB number
Date: Wednesday, June 30, 2010 - 2:31 pm

This comes from:

/*
 * The maximum PHB bus number.
 * x3950M2 (rare): 8 chassis, 48 PHBs per chassis = 384
 * x3950M2: 4 chassis, 48 PHBs per chassis        = 192
 * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
 * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
 */
#define MAX_PHB_BUS_NUM		384

Clearly there can't be 384 busses with standard PCI numbering (bus
numbers are 8 bits).  That means either that the number 384 is just
wrong, or it means that there are multiple PCI domains involved, and
that the BUG_ON() should be something else.

Furthermore, in get_tce_space_from_tar() we have:

	for (bus = 0; bus < MAX_PHB_BUS_NUM; bus++) {
		struct calgary_bus_info *info = &bus_info[bus];
		unsigned short pci_device;
		u32 val;

		val = read_pci_config(bus, 0, 0, 0);
		pci_device = (val & 0xFFFF0000) >> 16;

... which assumes the bus is a PCI bus number, no domain involved.

Does this mean the limit should be 256 (in which case we can just drop
the BUG_ON()), or is there support for domains which should be in this
code but isn't?

	-hpa
--


[ message continues ]
[view original message]
From: Darrick J. Wong
Subject: Re: [tip:x86/urgent] x86, Calgary: Increase max PHB number
Date: Wednesday, June 30, 2010 - 2:45 pm

Yes, it should be 256.  Some time ago there was talk of building 8-node
x3950M2s with PCI domain support, but the final word now is that nobody will
ever have more than 4 nodes.  The BUG_ON can be dropped, too.  I'll send a
patch shortly that incorporates both updates.

(Honestly I sort of wonder if the Calgary code should be removed entirely,
but that's a whole different battle....)

--D
--


[ message continues ]
[view original message]
From: Darrick J. Wong
Subject: [PATCH] x86, Calgary: Increase max PHB number
Date: Wednesday, June 30, 2010 - 2:49 pm

Newer systems (x3950M2) can have 48 PHBs per chassis and 4 chassis, so bump the
limits up and provide an explanation of the requirements for each class.  Since
we can't have more than 256 PCI buses in these systems, we don't need the array
check.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
---

 arch/x86/kernel/pci-calgary_64.c |   14 ++++++++------
 1 files changed, 8 insertions(+), 6 deletions(-)


diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index fb99f7e..fd4e27b 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -103,10 +103,14 @@ int use_calgary __read_mostly = 0;
 #define PMR_SOFTSTOPFAULT	0x40000000
 #define PMR_HARDSTOP		0x20000000
 
-#define MAX_NUM_OF_PHBS		8 /* how many PHBs in total? */
-#define MAX_NUM_CHASSIS		8 /* max number of chassis */
-/* MAX_PHB_BUS_NUM is the maximal possible dev->bus->number */
-#define MAX_PHB_BUS_NUM		(MAX_NUM_OF_PHBS * MAX_NUM_CHASSIS * 2)
+/*
+ * The maximum PHB bus number.
+ * x3950M2: 4 chassis, 48 PHBs per chassis        = 192
+ * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
+ * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
+ */
+#define MAX_PHB_BUS_NUM		256
+
 #define PHBS_PER_CALGARY	4
 
 /* register offsets in Calgary's internal register space */
@@ -1051,8 +1055,6 @@ static int __init calgary_init_one(struct pci_dev *dev)
 	struct iommu_table *tbl;
 	int ret;
 
-	BUG_ON(dev->bus->number >= MAX_PHB_BUS_NUM);
-
 	bbar = busno_to_bbar(dev->bus->number);
 	ret = calgary_setup_tar(dev, bbar);
 	if (ret)
--


[ message continues ]
[view original message]
From: H. Peter Anvin
Subject: Re: [PATCH] x86, Calgary: Increase max PHB number
Date: Wednesday, June 30, 2010 - 4:05 pm

The 384-entry patch is already upstream.  Can you send a patch relative
to current -linus?

	-hpa

--


[ message continues ]
[view original message]
From: Darrick J. Wong
Subject: [PATCH] x86, Calgary: Increase max PHB number
Date: Wednesday, June 30, 2010 - 5:45 pm

The x3950 family can have as many as 256 PCI buses in a single system, so
change the limits to the maximum.  Since there can only be 256 PCI buses in one
domain, we no longer need the BUG_ON check.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
---

 arch/x86/kernel/pci-calgary_64.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)


diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 0b96b55..078d4ec 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -110,7 +110,7 @@ int use_calgary __read_mostly = 0;
  * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
  * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
  */
-#define MAX_PHB_BUS_NUM		384
+#define MAX_PHB_BUS_NUM		256
 
 #define PHBS_PER_CALGARY	  4
 
@@ -1056,8 +1056,6 @@ static int __init calgary_init_one(struct pci_dev *dev)
 	struct iommu_table *tbl;
 	int ret;
 
-	BUG_ON(dev->bus->number >= MAX_PHB_BUS_NUM);
-
 	bbar = busno_to_bbar(dev->bus->number);
 	ret = calgary_setup_tar(dev, bbar);
 	if (ret)
--


[ message continues ]
[view original message]
From: tip-bot for Darrick J. Wong
Subject: [tip:x86/urgent] x86, Calgary: Limit the max PHB number to 256
Date: Wednesday, June 30, 2010 - 10:45 pm

Commit-ID:  d596043d71ff0d7b3d0bead19b1d68c55f003093
Gitweb:     http://git.kernel.org/tip/d596043d71ff0d7b3d0bead19b1d68c55f003093
Author:     Darrick J. Wong <djwong@us.ibm.com>
AuthorDate: Wed, 30 Jun 2010 17:45:19 -0700
Committer:  H. Peter Anvin <hpa@zytor.com>
CommitDate: Wed, 30 Jun 2010 22:41:42 -0700

x86, Calgary: Limit the max PHB number to 256

The x3950 family can have as many as 256 PCI buses in a single system, so
change the limits to the maximum.  Since there can only be 256 PCI buses in one
domain, we no longer need the BUG_ON check.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
LKML-Reference: <20100701004519.GQ15515@tux1.beaverton.ibm.com>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
---
 arch/x86/kernel/pci-calgary_64.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 0b96b55..078d4ec 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -110,7 +110,7 @@ int use_calgary __read_mostly = 0;
  * x3950 (PCIE): 8 chassis, 32 PHBs per chassis   = 256
  * x3950 (PCIX): 8 chassis, 16 PHBs per chassis   = 128
  */
-#define MAX_PHB_BUS_NUM		384
+#define MAX_PHB_BUS_NUM		256
 
 #define PHBS_PER_CALGARY	  4
 
@@ -1056,8 +1056,6 @@ static int __init calgary_init_one(struct pci_dev *dev)
 	struct iommu_table *tbl;
 	int ret;
 
-	BUG_ON(dev->bus->number >= MAX_PHB_BUS_NUM);
-
 	bbar = busno_to_bbar(dev->bus->number);
 	ret = calgary_setup_tar(dev, bbar);
 	if (ret)
--


[ message continues ]
Previous thread: [PATCH] x86/mrst/pci: avoid enabling intx for msi capable devices by Jacob Pan on Monday, June 21, 2010 - 11:30 am. (3 messages)

Next thread: [PATCH 0/2] cfq: fixes to bring cfq in line with deadline performance for mid- to high-end storage by Jeff Moyer on Monday, June 21, 2010 - 12:49 pm. (19 messages)