Re: [PATCH NEXT 0/2] qlcnic: Add Qlogic 1/10Gb Ethernet driver for CNA devices

Previous thread: [PATCH] cs89x0: Always report failure to request interrupt by Mark Brown on Tuesday, January 5, 2010 - 4:16 am. (2 messages)

Next thread: [PATCH 5/6] hso: Fix for 5 sec timeouts with v2.x firmware by Jan Dumon on Tuesday, January 5, 2010 - 7:52 am. (2 messages)
From: Amit Kumar Salecha
Date: Tuesday, January 5, 2010 - 6:24 am

Hi
  Series of 2 patches to add Qlogic ethernet driver for QLE8240 and
  QLE8242 CNA devices. QLE8240 and QLE8242 devices support various 
  class of drivers.
  Please apply to net-next tree.

-Amit Salecha
--

From: Amit Kumar Salecha
Date: Tuesday, January 5, 2010 - 6:24 am

o Separate Ethernet driver for Qlogic CNA devices

Signed-off-by: Amit Kumar Salecha <amit.salecha@qlogic.com>
---
 MAINTAINERS          |    7 +++++++
 drivers/net/Kconfig  |    7 +++++++
 drivers/net/Makefile |    1 +
 3 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 745643b..ed460c5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4452,6 +4452,13 @@ S:	Supported
 F:	Documentation/networking/LICENSE.qla3xxx
 F:	drivers/net/qla3xxx.*
 
+QLOGIC QLCNIC (1/10)Gb ETHERNET DRIVER
+M:	Amit Kumar Salecha <amit.salecha@qlogic.com>
+M:	linux-driver@qlogic.com
+L:	netdev@vger.kernel.org
+S:	Supported
+F:	drivers/net/qlcnic/
+
 QLOGIC QLGE 10Gb ETHERNET DRIVER
 M:	Ron Mercer <ron.mercer@qlogic.com>
 M:	linux-driver@qlogic.com
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index dd9a09c..4fdb20b 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -2756,6 +2756,13 @@ config BNX2X
 	  To compile this driver as a module, choose M here: the module
 	  will be called bnx2x.  This is recommended.
 
+config QLCNIC
+	tristate "QLOGIC QLCNIC 1/10Gb Converged Ethernet NIC Support"
+	depends on PCI
+	help
+	  This driver supports QLogic QLE8240 and QLE8242 Converged Ethernet
+	  devices.
+
 config QLGE
 	tristate "QLogic QLGE 10Gb Ethernet Driver Support"
 	depends on PCI
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index ad1346d..a9c4b01 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -148,6 +148,7 @@ ll_temac-objs := ll_temac_main.o ll_temac_mdio.o
 obj-$(CONFIG_XILINX_LL_TEMAC) += ll_temac.o
 obj-$(CONFIG_XILINX_EMACLITE) += xilinx_emaclite.o
 obj-$(CONFIG_QLA3XXX) += qla3xxx.o
+obj-$(CONFIG_QLCNIC) += qlcnic/
 obj-$(CONFIG_QLGE) += qlge/
 
 obj-$(CONFIG_PPP) += ppp_generic.o
-- 
1.6.0.2

--

From: Amit Kumar Salecha
Date: Tuesday, January 5, 2010 - 6:24 am

o 1G/10G Ethernet Driver for Qlgic QLE8240 and QLE8242 CNA devices.

Signed-off-by: Amit Kumar Salecha <amit.salecha@qlogic.com>
---
 drivers/net/qlcnic/Makefile         |   28 +
 drivers/net/qlcnic/qlcnic.h         | 1354 +++++++++++++++++++
 drivers/net/qlcnic/qlcnic_ctx.c     |  582 ++++++++
 drivers/net/qlcnic/qlcnic_ethtool.c |  877 ++++++++++++
 drivers/net/qlcnic/qlcnic_hdr.h     | 1060 +++++++++++++++
 drivers/net/qlcnic/qlcnic_hw.c      | 1285 ++++++++++++++++++
 drivers/net/qlcnic/qlcnic_hw.h      |  284 ++++
 drivers/net/qlcnic/qlcnic_init.c    | 1592 ++++++++++++++++++++++
 drivers/net/qlcnic/qlcnic_main.c    | 2549 +++++++++++++++++++++++++++++++++++
 9 files changed, 9611 insertions(+), 0 deletions(-)
 create mode 100644 drivers/net/qlcnic/Makefile
 create mode 100644 drivers/net/qlcnic/qlcnic.h
 create mode 100644 drivers/net/qlcnic/qlcnic_ctx.c
 create mode 100644 drivers/net/qlcnic/qlcnic_ethtool.c
 create mode 100644 drivers/net/qlcnic/qlcnic_hdr.h
 create mode 100644 drivers/net/qlcnic/qlcnic_hw.c
 create mode 100644 drivers/net/qlcnic/qlcnic_hw.h
 create mode 100644 drivers/net/qlcnic/qlcnic_init.c
 create mode 100644 drivers/net/qlcnic/qlcnic_main.c

diff --git a/drivers/net/qlcnic/Makefile b/drivers/net/qlcnic/Makefile
new file mode 100644
index 0000000..5260f8a
--- /dev/null
+++ b/drivers/net/qlcnic/Makefile
@@ -0,0 +1,28 @@
+# Copyright (C) 2009 - QLogic Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General ...
From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 10:08 am

On Tue,  5 Jan 2010 05:24:36 -0800

Don't you mean:
obj-$(CONFIG_QLCNIC) := qlcnic.o
--

From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 10:12 am

One valid complaint from checkpatch:

ERROR: Macros with complex values should be enclosed in parenthesis
#2096: FILE: drivers/net/qlcnic/qlcnic_ethtool.c:41:
+#define QLCNIC_STAT(m) sizeof(((struct qlcnic_adapter *)0)->m), \
+			offsetof(struct qlcnic_adapter, m)

total: 1 errors, 0 warnings, 9611 lines checked
--

From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 10:07 am

On Tue,  5 Jan 2010 05:24:34 -0800

The patches are out of order. If patch 1 was applied (but not patch 2),
then it would be possible to configure device but the actual driver
code would not be there. This could happen when doing git bisect.

I assume Dave will just resolve this by combining the two before
merging.

-- 
--

From: David Miller
Date: Tuesday, January 5, 2010 - 1:39 pm

From: Stephen Hemminger <shemminger@vyatta.com>

Don't assume anything. :-)

Patches should be submitted such that they compile when applied
in-order and are fully bisectable.
--

From: Amit Salecha
Date: Tuesday, January 5, 2010 - 7:24 pm

I bisected patches logically, for ease of understanding.
If single patch is requirement, I can do that also.
________________________________________
From: Stephen Hemminger [shemminger@vyatta.com]
Sent: Tuesday, January 05, 2010 10:37 PM
To: Amit Salecha
Cc: davem@davemloft.net; netdev@vger.kernel.org; Dhananjay Phadke
Subject: Re: [PATCH NEXT 0/2] qlcnic: Add Qlogic 1/10Gb Ethernet driver for CNA devices

On Tue,  5 Jan 2010 05:24:34 -0800

The patches are out of order. If patch 1 was applied (but not patch 2),
then it would be possible to configure device but the actual driver
code would not be there. This could happen when doing git bisect.

I assume Dave will just resolve this by combining the two before
merging.

--
--

From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 7:36 pm

On Tue, 5 Jan 2010 20:24:18 -0600

Two or more patches are fine, but the order is important.
You can add each file separately, then add the config option
last.

-- 
--

From: David Miller
Date: Tuesday, January 5, 2010 - 8:53 pm

From: Amit Salecha <amit.salecha@qlogic.com>

You didn't bisect anything, we're talking about GIT bisection.

At each step of the way, the tree must build if all available
kernel config options are enabled.
--

From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 10:15 am

On Tue,  5 Jan 2010 05:24:34 -0800

It looks like this driver is tracking IP addresses? Is it doing
some offload? will it work with IPV6?
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 11:40 am

Driver listens for "local" IP addresses that helps
firmware to detect local TCP flows for LRO. We don't
want firmware to accumulate non-local (forwarded)
packets. This is inherited from netxen_nic.

Currently only IPv4 addresses are supported, IPv6 will
be enabled as soon as firmware advertises support.

-Dhananjay
--

From: Stephen Hemminger
Date: Tuesday, January 5, 2010 - 11:45 am

On Tue, 5 Jan 2010 10:40:35 -0800

You should be doing GRO, what about multiple addresses per interface,
bridging, routing, ...
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 12:02 pm

We do GRO too (that helps when old firmware didn't support LRO).
Since all addresses associated with a net device are tracked
we do take care of IP aliases. Note that LRO packets are not
fed into GRO, although there's nothing that prevents it.

On older machines, GRO tops out CPU to get close to line rate,
LRO doesn't. Also, LRO reduces number of PCIe transactions.

LRO across bridge / routing is topology violation right?
We don't get inet notification for remote IP addresses (correct
me if wrong), so these cases are excluded from LRO.

-Dhananjay
--

From: David Miller
Date: Tuesday, January 5, 2010 - 1:38 pm

From: Dhananjay Phadke <dhananjay.phadke@qlogic.com>

You should only do GRO.
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 3:01 pm

> You should only do GRO.

Is there problem with stateless packet accumulation by
hardware / firmware when it helps to cut 20% cpu cycles
over GRO and helping some older machines to get line rates
at 10Gbps?

HW LRO is there already in many drivers under different
names (RSC, etc). The difference with netxen_nic and
qlcnic is that it takes care of allowing only local
flows not breaking across bridging or routing topology.

-Dhananjay
--

From: David Miller
Date: Tuesday, January 5, 2010 - 4:49 pm

From: Dhananjay Phadke <dhananjay.phadke@qlogic.com>

We've been gradually removing LRO support from drivers and replacing
it with GRO support.

And meanwhile also encouraging device manufacturers to make their
hardware accumulation support more closely align with being able to
offload GRO natively.
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 4:55 pm

The LRO you are talking about was still kernel based
LRO. Even GRO doesn't save PCIe transactions as much
HW LRO does. That is the key. The difference between
HW LRO and GRO is significant to keep former alive
for a while.

-Dhananjay
--

From: David Miller
Date: Tuesday, January 5, 2010 - 5:33 pm

From: Dhananjay Phadke <dhananjay.phadke@qlogic.com>

But HW LRO is not bridge/forwarding/etc. agnostic because it loses
information.

As soon as I turn on forwarding to make a NAT box or whatever, all of
the hw LRO gets turned off and we're back to square one for local
connections on the machine.
--

From: Rick Jones
Date: Tuesday, January 5, 2010 - 6:33 pm

Is it really the PCIe transactions (*)?  ISTR that HW LRO also tended to 
significantly cut-down on the ACK rate, rather like the oft-derided ACK 
avoidance heuristics in Solaris, HP-UX and various MacOSX revs.

rick jones

* OK, I am recalling that long ago and far away, the Alteon AceNIC (PCI) had 
very high DMA setup times - so long that if an outbound (in this case) packet 
was more than 1.something buffers the card was unable to acheive link-rate with 
a 1500 byte MTU.  It meant that for straight copy outbound was fine, but for 
copy avoided (eg sendfile() - this was HP-UX 11) one was sub-par because it was 
at least two buffers per packet - aka two DMA setups - and sometimes three.  One 

--

From: David Miller
Date: Tuesday, January 5, 2010 - 8:57 pm

From: Rick Jones <rick.jones2@hp.com>

Yes, it causes stretch ACKs, but so does GRO.
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 6:56 pm

I won't defend other vendors' implementation. Even if LRO was
not turned off for bridge/forwarding, we still wouldn't accumulate
those remote flows because the dest IP address is not configured in
firmware in our case. This is the reason why we have a inet
notification listener (to tell firmware about local addresses).

-Dhananjay

--

From: David Miller
Date: Tuesday, January 5, 2010 - 8:51 pm

From: Dhananjay Phadke <dhananjay.phadke@qlogic.com>

Ok, but you do realize that because GRO maintains enough state
we can reconstitute packets across forwarding and bridging
thus decreasing the per-packet costs in those situations as
well right?

That's why hardware vendors should strive to align their hw
LRO implementations with what GRO does, since it is truly
generic and allows to optimize all cases not just local
TCP streams.
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 9:27 pm

Yes, GRO has better coverage but HW LRO allows better interrupt
mitigation since fewer buffers are cycled. While the size of
LRO buffer is debatable in terms of memory allocation (recall
discussion on subject "netxen: fix lro buffer allocation"),
it can cause fewer skb allocations.

May be some of this can be achieved with GRO + rx interrupt
coalesce thresholds, etc.

For now, we need to have option of HW LRO that can be turned
off based on use case (NAT etc) via ethtool or whatever.

Thanks,
Dhananjay
--

From: David Miller
Date: Tuesday, January 5, 2010 - 9:40 pm

From: Dhananjay Phadke <dhananjay.phadke@qlogic.com>

GRO achieves this with NAPI.

Of course, conservative HW interrupt settings help this
even more, and that is what we suggest NAPI drivers do
anyways.
--

From: Dhananjay Phadke
Date: Tuesday, January 5, 2010 - 7:19 pm

May be I should clarify further. Driver does check if the net device
is owned by a qlogic (or netxen) pci device then only will care
about telling firmware about the IP address.

On NAT box, the IP address belonging to tap/tun device won't land into
firmware preventing any packet accumulation in HW.

-Dhananjay

--

From: Ben Hutchings
Date: Tuesday, January 5, 2010 - 4:43 pm

LRO will automatically be disabled on interfaces that have forwarding or
bridging enabled.  You should not need to check for this yourself.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

--

Previous thread: [PATCH] cs89x0: Always report failure to request interrupt by Mark Brown on Tuesday, January 5, 2010 - 4:16 am. (2 messages)

Next thread: [PATCH 5/6] hso: Fix for 5 sec timeouts with v2.x firmware by Jan Dumon on Tuesday, January 5, 2010 - 7:52 am. (2 messages)