up to 2 rx/tx queues for the 82574.

Program the 82574 to enable 5 msix vectors, assign 1 to each rx queue,
1 to each tx queue and 1 to the link handler.

Inspired by DragonFlyBSD, enable some RSS logic for handling tx queue
handling/processing.

Move multiqueue handler functions so that they line up better in a diff
review to if_igb.c

Always enqueue tx work to be done in em_mq_start, if unable to acquire
the TX lock, then this will be processed in the background later by the
taskqueue.  Remove mbuf argument from em_start_mq_locked() as the work
is always enqueued.  (stolen from igb)

Setup TARC, TXDCTL and RXDCTL registers for better performance and stability
in multiqueue and singlequeue implementations. Handle Intel errata  3 and
generic multiqueue behavior with the initialization of TARC(0) and TARC(1)

Bind interrupt threads to cpus in order.  (stolen from igb)

Add 2 new DDB functions, one to display the queue(s) and their settings and
one to reset the adapter.  Primarily used for debugging.

In the multiqueue configuration, bump RXD and TXD ring size to max for the
adapter (4096).  Setup an RDTR of 64 and an RADV of 128 in multiqueue configuration
to cut down on the number of interrupts.  RADV was arbitrarily set to 2x RDTR
and can be adjusted as needed.

Cleanup the display in top a bit to make it clearer where the taskqueue threads
are running and what they should be doing.

Ensure that both queues are processed by em_local_timer() by writing them both
to the IMS register to generate soft interrupts.

Ensure that an soft interrupt is generated when em_msix_link() is run so that
any races between assertion of the link/status interrupt and a rx/tx interrupt
are handled.

Document existing tuneables: hw.em.eee_setting, hw.em.msix, hw.em.smart_pwr_down, hw.em.sbp

Document use of hw.em.num_queues and the new kernel option EM_MULTIQUEUE

Thanks to Intel for their continued support of FreeBSD.

Reviewed by:	erj jfv hiren gnn wblock
Obtained from:	Intel Corporation
MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D1994

applying them to em(4).

Rely on iterations through the local timer, and the tx queue state to
determine if an actual hang has occurred. Any time a descriptor is used
(packet sent), the tx queue is flagged as busy. Then when txeof runs, it
either clears the flag when all is clean, or resets it to 1 if ANY are
cleaned, if nothing is cleaned it increments the flag.

Local timer simply checks to see if busy ever reaches MAX (10, which
is compile time configurable), and then sets it as HUNG, at that point
there is one more timer cycle in which to have any cleans, if not a
watchdog reset will occur.

Differential Revision:	https://reviews.freebsd.org/D2019
Submitted by:	jfv
Reviewed by:	hiren
Obtained from:	Intel Corporation
MFC after:	2 weeks
Relnotes:	Yes
Sponsored by:	Limelight Networks

Submitted by:   Anuranjan Shukla <anshukla@juniper.net>
Obtained from:  Juniper Networks, Inc.

	irrespective of the setting of lem_rx_process_limit, while
	giving a chance to the taskqueue scheduler to act after
	each chunk.
	This makes lem_rxeof similar to the one in if_em.c and if_igb.c .

if_lem.c and if_em.c: add a sysctl to manually configure the
	'itr' moderation register.

Approved by:	Jack Vogel

There have still been intermittent problems with apparent TX
hangs for some customers. These have been problematic to reproduce
but I believe these changes will address them. Testing on a number
of fronts have been positive.

EM: there is an important 'chicken bit' fix for 82574 in the shared
code this is supported in the core here.
    - The TX path has been tightened up to improve performance. In
      particular UDP with jumbo frames was having problems, and the
      changes here have improved that.
    - OACTIVE has been used more carefully on the theory that some
      hangs may be due to a problem in this interaction
    - Problems with the RX init code, the "lazy" allocation and
      ring initialization has been found to cause problems in some
      newer client systems, and as it really is not that big a win
      (its not in a hot path) it seems best to remove it.
    - HWTSO was broken when VLAN HWTAGGING or HWFILTER is used, I
      found this was due to an error in setting up the descriptors
      in em_xmit.

IGB:
    - TX is also improved here. With multiqueue I realized its very
      important to handle OACTIVE only under the CORE lock so there
      are no races between the queues.
    - Flow Control handling was broken in a couple ways, I have changed
      and I hope improved that in this delta.
    - UDP also had a problem in the TX path here, it was change to
      improve that.
    - On some hardware, with the driver static, a weird stray interrupt
      seems to sometimes fire and cause a panic in the RX mbuf refresh
      code. This is addressed by setting interrupts late in the init
      path, and also to set all interrupts bits off at the start of that.

to calculate the outstanding descriptors that need to be
refreshed at any time, and use THAT in rxeof to determine
if refreshing needs to be done. Also change the local_timer
to simply fire off the appropriate interrupt rather than
schedule a tasklet, its simpler.

MFC in two weeks

been undergoing test for some weeks. This improves the RX
mbuf handling to avoid system hang due to depletion. Thanks
to all those who have been testing the code, and to Beezar
Liu for the design changes.

Next the igb driver is updated for similar RX changes, but
also to add new features support for our upcoming i350 family
of adapters.

MFC after a week

	- Chasin down bogus watchdogs has led to an improved
	  design to this handling, the hang decision takes
	  place in the tx cleanup, with only a simple report
	  check in local_timer. Our tests have shown no false
	  watchdogs with this code.
	- VLAN fixes from jhb, the shadow vfta should be per
	  interface, but as global it was not. Thanks John.
	- Bug fixes in the support for new PCH2 hardware.
	- Thanks for all the help and feedback on the driver,
	  changes to lem with be coming shortly as well.

by /etc/rc.d/jail.

	- Sync shared code with Intel internal
	- New client chipset support added
	- em driver - fixes to 82574, limit queues to 1 but use MSIX
	- em driver - large changes in TX checksum offload and tso
	  code, thanks to yongari.
	- some small changes for watchdog issues.
	- igb driver - local timer watchdog code was missing locking
	  this and a couple other watchdog related fixes.
	- bug in rx discard found by Andrew Boyer, check for null pointer

MFC: a week

discrepencies from the igb version which was the target.

Change the message when neither MSI or MSIX are enabled
and a fallback to Legacy interrupts happen, the existing
message was confusing.

configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by:	jfv

	- code flow in handler could let interrupt be
	  reenabled when not wanted.
	- change where the RX lock is taken to improve
	  performance.
	- adapter->msix is true for MSI systems also,
	  it needs to explicitly test for 82574, good one :)

Also, from feedback, make the multiqueue code an
option (EM_MULTIQUEUE) that is off by default.
Problems have been seen with UDP when its on.

led(4) so they can be lit or f.e. made blink via `echo f2 > /dev/led/em0`
for localization purposes.

Approved by:	jfv
MFC afer:	1 week (after r205869)

em revision 7.0.0:
	- Using driver devclass, seperate legacy (pre-pcie) code
	  into a seperate source file. This will at least help
	  protect against regression issues. It compiles along
	  with em, and is transparent to end use, devices in each
	  appear to be 'emX'. When using em in a modular form this
	  also allows the legacy stuff to be defined out.
	- Add tx and rx rings as in igb, in the 82574 this becomes
	  actual multiqueue for the first time (2 queues) while in
	  other PCIE adapters its just make code cleaner.
	- Add RX mbuf handling logic that matches igb, this will
	  eliminate packet drops due to temporary mbuf shortage.

igb revision 1.9.3:
	- Following the ixgbe code, use a new approach in what
	  was called 'get_buf', the routine now has been made
	  independent of rxeof, it now does the update to the
	  engine TDT register, this design allows temporary
	  mbuf resources to become non-critical, not requiring
	  a packet to be discarded, instead it just returns and
	  does not increment the tail pointer.
	- With the above change it was also unnecessary to keep
	  'spare' maps around, since we do not have the discard
	  issue.
	- Performance tweaks and improvements to the code also.

MFC in a week

the checkin, sorry all :(

igb now has a queue notion that has a single interrupt
with an RX/TX pair, this will reduce the total interrupts
seen on a system. Both em and igb have a new watchdog
method. igb has fixes from Pyun Yong-Hyeon that have
improved stability, thank you :)

I wish to MFC this for 7.3 asap, please test if able.

drivers. These add new hardware support, most importantly
the pch (i5 chipset) in the em driver. Also, both drivers
now have the simplified (and I hope improved) watchdog
code. The igb driver uses the new RX cleanup that I
first implemented in ixgbe.

em  - version 6.9.24
igb - version 1.8.4

for multiqueue tx, shared code updates, new device
support, and some bug fixes.

- make buF_ring usage conditional but enabled by default

Reviewed by:	jfv

will sometimes fail to initialize problem due to a lock
contention with management hardware. However, in order to
deliver that fix it was necessary to take a shared code
update as a whole, and this required scattered changes in
the core code to be compatible.

The em driver now has VLAN HW support added as the igb
driver had previously.

MFC after:  ASAP - in time for 7.1 RELEASE

proved to be necessary to make the static drivers work
in EITHER/OR or BOTH configurations. Modules will still
build in sys/modules/igb or em as before.

This also updates the igb driver for support for the 82576
adapter, adds shared code fixes, and etc....

MFC after:	ASAP

PR 122839 is fixed in both em and in igb

Second, the issue on building modules since the static kernel
build changes is now resolved. I was not able to get the fancier
directory hierarchy working, but this works, both em and igb
build as modules now.

Third, there is now support in em for two new NICs, Hartwell
(or 82574) is a low cost PCIE dual port adapter that has MSIX,
for this release it uses 3 vectors only, RX, TX, and LINK. In
the next release I will add a second TX and RX queue. Also, there
is support here for ICH10, the followon to ICH9. Both of these are
early releases, general availability will follow soon.

Fourth: On Hartwell and ICH10 we now have IEEE 1588 PTP support,
I have implemented this in a provisional way so that early adopters
may try and comment on the functionality. The IOCTL structure may
change. This feature is off by default, you need to edit the Makefile
and add the EM_TIMESYNC define to get the code.

Enjoy all!!

and the igb driver static in the kernel. But it also reflects
some other bug fixes in my development stream at Intel.
PR 122373 is also fixed in this code.

just em, there is an igb driver (this follows behavior with our Linux drivers).
All adapters up to the 82575 are supported in em, and new client/desktop support
will continue to be in that adapter.

The igb driver is for new server NICs like the 82575 and its followons.
Advanced features for virtualization and performance will be in this driver.

Also, both drivers now have shared code that is up to the latest we have
released. Some stylistic changes as well.

Enjoy :)

when the two locks are the same...

between RELEASES, but we want it on by default in 7 and later,
add that define, and take out a fragment left from a workaround
being removed.

 - Bring HEAD up to the latest shared code
 - Fix TSO problem using limited MSS and forwarding
 - Dual lock implementation
 - New device support
 - For my ease, this code can compile in either 6.x or later
 - brings this driver in sync with the 6.3

	- duplicate #define in header, thanks to Kevin Lo for pointing out.
	- incorrect BUSMASTER enable logic, thanks Patrick Oeschger
	- 82543 fails due to bogus IO BAR logic
	- Allow 82571 to use MSI interrupts
	- Checksum Offload for UDP not working on 82575

Approved by:re

adapter list still capable, but only PCI-E adapters are now enabled.
The user can enable older PCI-X or PCI adapters using ifconfig.
Secondly, Arthur Hartwig pointed out my MSI change was not working
correctly, changed to something that now does. Thanks Arthur.
There was also a fundamental bug in the 82575 MSIX code, the MSIX
registers had to be mapped, opps :)

Rubber-stamped by: Pdeuskar

legacy codepath match the 82575, without this we were seeing bridging
fail on 82546 adapters. Secondly, I have limited TSO to PCI Express
adapters, I meant to do this and it got dropped in the earlier delta.
Next, I am dropping in the latest shared code from our development
team, consensus was that this should be done frequently, so I am :)

Approved by: pdeuskar