553 lines · plain
1.. SPDX-License-Identifier: GPL-2.0+2 3===========================================================================4Linux Base Driver for the Intel(R) Ethernet 10 Gigabit PCI Express Adapters5===========================================================================6 7Intel 10 Gigabit Linux driver.8Copyright(c) 1999-2018 Intel Corporation.9 10Contents11========12 13- Identifying Your Adapter14- Command Line Parameters15- Additional Configurations16- Known Issues17- Support18 19Identifying Your Adapter20========================21The driver is compatible with devices based on the following:22 23 * Intel(R) Ethernet Controller 8259824 * Intel(R) Ethernet Controller 8259925 * Intel(R) Ethernet Controller X52026 * Intel(R) Ethernet Controller X54027 * Intel(R) Ethernet Controller x55028 * Intel(R) Ethernet Controller X55229 * Intel(R) Ethernet Controller X55330 31For information on how to identify your adapter, and for the latest Intel32network drivers, refer to the Intel Support website:33https://www.intel.com/support34 35SFP+ Devices with Pluggable Optics36----------------------------------37 3882599-BASED ADAPTERS39~~~~~~~~~~~~~~~~~~~~40NOTES:41- If your 82599-based Intel(R) Network Adapter came with Intel optics or is an42Intel(R) Ethernet Server Adapter X520-2, then it only supports Intel optics43and/or the direct attach cables listed below.44- When 82599-based SFP+ devices are connected back to back, they should be set45to the same Speed setting via ethtool. Results may vary if you mix speed46settings.47 48+---------------+---------------------------------------+------------------+49| Supplier | Type | Part Numbers |50+===============+=======================================+==================+51| SR Modules |52+---------------+---------------------------------------+------------------+53| Intel | DUAL RATE 1G/10G SFP+ SR (bailed) | FTLX8571D3BCV-IT |54+---------------+---------------------------------------+------------------+55| Intel | DUAL RATE 1G/10G SFP+ SR (bailed) | AFBR-703SDZ-IN2 |56+---------------+---------------------------------------+------------------+57| Intel | DUAL RATE 1G/10G SFP+ SR (bailed) | AFBR-703SDDZ-IN1 |58+---------------+---------------------------------------+------------------+59| LR Modules |60+---------------+---------------------------------------+------------------+61| Intel | DUAL RATE 1G/10G SFP+ LR (bailed) | FTLX1471D3BCV-IT |62+---------------+---------------------------------------+------------------+63| Intel | DUAL RATE 1G/10G SFP+ LR (bailed) | AFCT-701SDZ-IN2 |64+---------------+---------------------------------------+------------------+65| Intel | DUAL RATE 1G/10G SFP+ LR (bailed) | AFCT-701SDDZ-IN1 |66+---------------+---------------------------------------+------------------+67 68The following is a list of 3rd party SFP+ modules that have received some69testing. Not all modules are applicable to all devices.70 71+---------------+---------------------------------------+------------------+72| Supplier | Type | Part Numbers |73+===============+=======================================+==================+74| Finisar | SFP+ SR bailed, 10g single rate | FTLX8571D3BCL |75+---------------+---------------------------------------+------------------+76| Avago | SFP+ SR bailed, 10g single rate | AFBR-700SDZ |77+---------------+---------------------------------------+------------------+78| Finisar | SFP+ LR bailed, 10g single rate | FTLX1471D3BCL |79+---------------+---------------------------------------+------------------+80| Finisar | DUAL RATE 1G/10G SFP+ SR (No Bail) | FTLX8571D3QCV-IT |81+---------------+---------------------------------------+------------------+82| Avago | DUAL RATE 1G/10G SFP+ SR (No Bail) | AFBR-703SDZ-IN1 |83+---------------+---------------------------------------+------------------+84| Finisar | DUAL RATE 1G/10G SFP+ LR (No Bail) | FTLX1471D3QCV-IT |85+---------------+---------------------------------------+------------------+86| Avago | DUAL RATE 1G/10G SFP+ LR (No Bail) | AFCT-701SDZ-IN1 |87+---------------+---------------------------------------+------------------+88| Finisar | 1000BASE-T SFP | FCLF8522P2BTL |89+---------------+---------------------------------------+------------------+90| Avago | 1000BASE-T | ABCU-5710RZ |91+---------------+---------------------------------------+------------------+92| HP | 1000BASE-SX SFP | 453153-001 |93+---------------+---------------------------------------+------------------+94 9582599-based adapters support all passive and active limiting direct attach96cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications.97 98Laser turns off for SFP+ when ifconfig ethX down99~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~100"ifconfig ethX down" turns off the laser for 82599-based SFP+ fiber adapters.101"ifconfig ethX up" turns on the laser.102Alternatively, you can use "ip link set [down/up] dev ethX" to turn the103laser off and on.104 105 10682599-based QSFP+ Adapters107~~~~~~~~~~~~~~~~~~~~~~~~~~108NOTES:109- If your 82599-based Intel(R) Network Adapter came with Intel optics, it only110supports Intel optics.111- 82599-based QSFP+ adapters only support 4x10 Gbps connections. 1x40 Gbps112connections are not supported. QSFP+ link partners must be configured for1134x10 Gbps.114- 82599-based QSFP+ adapters do not support automatic link speed detection.115The link speed must be configured to either 10 Gbps or 1 Gbps to match the link116partners speed capabilities. Incorrect speed configurations will result in117failure to link.118- Intel(R) Ethernet Converged Network Adapter X520-Q1 only supports the optics119and direct attach cables listed below.120 121+---------------+---------------------------------------+------------------+122| Supplier | Type | Part Numbers |123+===============+=======================================+==================+124| Intel | DUAL RATE 1G/10G QSFP+ SRL (bailed) | E10GQSFPSR |125+---------------+---------------------------------------+------------------+126 12782599-based QSFP+ adapters support all passive and active limiting QSFP+128direct attach cables that comply with SFF-8436 v4.1 specifications.129 13082598-BASED ADAPTERS131~~~~~~~~~~~~~~~~~~~~132NOTES:133- Intel(r) Ethernet Network Adapters that support removable optical modules134only support their original module type (for example, the Intel(R) 10 Gigabit135SR Dual Port Express Module only supports SR optical modules). If you plug in136a different type of module, the driver will not load.137- Hot Swapping/hot plugging optical modules is not supported.138- Only single speed, 10 gigabit modules are supported.139- LAN on Motherboard (LOMs) may support DA, SR, or LR modules. Other module140types are not supported. Please see your system documentation for details.141 142The following is a list of SFP+ modules and direct attach cables that have143received some testing. Not all modules are applicable to all devices.144 145+---------------+---------------------------------------+------------------+146| Supplier | Type | Part Numbers |147+===============+=======================================+==================+148| Finisar | SFP+ SR bailed, 10g single rate | FTLX8571D3BCL |149+---------------+---------------------------------------+------------------+150| Avago | SFP+ SR bailed, 10g single rate | AFBR-700SDZ |151+---------------+---------------------------------------+------------------+152| Finisar | SFP+ LR bailed, 10g single rate | FTLX1471D3BCL |153+---------------+---------------------------------------+------------------+154 15582598-based adapters support all passive direct attach cables that comply with156SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach cables157are not supported.158 159Third party optic modules and cables referred to above are listed only for the160purpose of highlighting third party specifications and potential161compatibility, and are not recommendations or endorsements or sponsorship of162any third party's product by Intel. Intel is not endorsing or promoting163products made by any third party and the third party reference is provided164only to share information regarding certain optic modules and cables with the165above specifications. There may be other manufacturers or suppliers, producing166or supplying optic modules and cables with similar or matching descriptions.167Customers must use their own discretion and diligence to purchase optic168modules and cables from any third party of their choice. Customers are solely169responsible for assessing the suitability of the product and/or devices and170for the selection of the vendor for purchasing any product. THE OPTIC MODULES171AND CABLES REFERRED TO ABOVE ARE NOT WARRANTED OR SUPPORTED BY INTEL. INTEL172ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED173WARRANTY, RELATING TO SALE AND/OR USE OF SUCH THIRD PARTY PRODUCTS OR174SELECTION OF VENDOR BY CUSTOMERS.175 176Command Line Parameters177=======================178 179max_vfs180-------181:Valid Range: 1-63182 183This parameter adds support for SR-IOV. It causes the driver to spawn up to184max_vfs worth of virtual functions.185If the value is greater than 0 it will also force the VMDq parameter to be 1 or186more.187 188NOTE: This parameter is only used on kernel 3.7.x and below. On kernel 3.8.x189and above, use sysfs to enable VFs. Also, for Red Hat distributions, this190parameter is only used on version 6.6 and older. For version 6.7 and newer, use191sysfs. For example::192 193 #echo $num_vf_enabled > /sys/class/net/$dev/device/sriov_numvfs // enable VFs194 #echo 0 > /sys/class/net/$dev/device/sriov_numvfs //disable VFs195 196The parameters for the driver are referenced by position. Thus, if you have a197dual port adapter, or more than one adapter in your system, and want N virtual198functions per port, you must specify a number for each port with each parameter199separated by a comma. For example::200 201 modprobe ixgbe max_vfs=4202 203This will spawn 4 VFs on the first port.204 205::206 207 modprobe ixgbe max_vfs=2,4208 209This will spawn 2 VFs on the first port and 4 VFs on the second port.210 211NOTE: Caution must be used in loading the driver with these parameters.212Depending on your system configuration, number of slots, etc., it is impossible213to predict in all cases where the positions would be on the command line.214 215NOTE: Neither the device nor the driver control how VFs are mapped into config216space. Bus layout will vary by operating system. On operating systems that217support it, you can check sysfs to find the mapping.218 219NOTE: When either SR-IOV mode or VMDq mode is enabled, hardware VLAN filtering220and VLAN tag stripping/insertion will remain enabled. Please remove the old221VLAN filter before the new VLAN filter is added. For example,222 223::224 225 ip link set eth0 vf 0 vlan 100 // set VLAN 100 for VF 0226 ip link set eth0 vf 0 vlan 0 // Delete VLAN 100227 ip link set eth0 vf 0 vlan 200 // set a new VLAN 200 for VF 0228 229With kernel 3.6, the driver supports the simultaneous usage of max_vfs and DCB230features, subject to the constraints described below. Prior to kernel 3.6, the231driver did not support the simultaneous operation of max_vfs greater than 0 and232the DCB features (multiple traffic classes utilizing Priority Flow Control and233Extended Transmission Selection).234 235When DCB is enabled, network traffic is transmitted and received through236multiple traffic classes (packet buffers in the NIC). The traffic is associated237with a specific class based on priority, which has a value of 0 through 7 used238in the VLAN tag. When SR-IOV is not enabled, each traffic class is associated239with a set of receive/transmit descriptor queue pairs. The number of queue240pairs for a given traffic class depends on the hardware configuration. When241SR-IOV is enabled, the descriptor queue pairs are grouped into pools. The242Physical Function (PF) and each Virtual Function (VF) is allocated a pool of243receive/transmit descriptor queue pairs. When multiple traffic classes are244configured (for example, DCB is enabled), each pool contains a queue pair from245each traffic class. When a single traffic class is configured in the hardware,246the pools contain multiple queue pairs from the single traffic class.247 248The number of VFs that can be allocated depends on the number of traffic249classes that can be enabled. The configurable number of traffic classes for250each enabled VF is as follows:2510 - 15 VFs = Up to 8 traffic classes, depending on device support25216 - 31 VFs = Up to 4 traffic classes25332 - 63 VFs = 1 traffic class254 255When VFs are configured, the PF is allocated one pool as well. The PF supports256the DCB features with the constraint that each traffic class will only use a257single queue pair. When zero VFs are configured, the PF can support multiple258queue pairs per traffic class.259 260allow_unsupported_sfp261---------------------262:Valid Range: 0,1263:Default Value: 0 (disabled)264 265This parameter allows unsupported and untested SFP+ modules on 82599-based266adapters, as long as the type of module is known to the driver.267 268debug269-----270:Valid Range: 0-16 (0=none,...,16=all)271:Default Value: 0272 273This parameter adjusts the level of debug messages displayed in the system274logs.275 276 277Additional Features and Configurations278======================================279 280Flow Control281------------282Ethernet Flow Control (IEEE 802.3x) can be configured with ethtool to enable283receiving and transmitting pause frames for ixgbe. When transmit is enabled,284pause frames are generated when the receive packet buffer crosses a predefined285threshold. When receive is enabled, the transmit unit will halt for the time286delay specified when a pause frame is received.287 288NOTE: You must have a flow control capable link partner.289 290Flow Control is enabled by default.291 292Use ethtool to change the flow control settings. To enable or disable Rx or293Tx Flow Control::294 295 ethtool -A eth? rx <on|off> tx <on|off>296 297Note: This command only enables or disables Flow Control if auto-negotiation is298disabled. If auto-negotiation is enabled, this command changes the parameters299used for auto-negotiation with the link partner.300 301To enable or disable auto-negotiation::302 303 ethtool -s eth? autoneg <on|off>304 305Note: Flow Control auto-negotiation is part of link auto-negotiation. Depending306on your device, you may not be able to change the auto-negotiation setting.307 308NOTE: For 82598 backplane cards entering 1 gigabit mode, flow control default309behavior is changed to off. Flow control in 1 gigabit mode on these devices can310lead to transmit hangs.311 312Intel(R) Ethernet Flow Director313-------------------------------314The Intel Ethernet Flow Director performs the following tasks:315 316- Directs receive packets according to their flows to different queues.317- Enables tight control on routing a flow in the platform.318- Matches flows and CPU cores for flow affinity.319- Supports multiple parameters for flexible flow classification and load320 balancing (in SFP mode only).321 322NOTE: Intel Ethernet Flow Director masking works in the opposite manner from323subnet masking. In the following command::324 325 #ethtool -N eth11 flow-type ip4 src-ip 172.4.1.2 m 255.0.0.0 dst-ip \326 172.21.1.1 m 255.128.0.0 action 31327 328The src-ip value that is written to the filter will be 0.4.1.2, not 172.0.0.0329as might be expected. Similarly, the dst-ip value written to the filter will be3300.21.1.1, not 172.0.0.0.331 332To enable or disable the Intel Ethernet Flow Director::333 334 # ethtool -K ethX ntuple <on|off>335 336When disabling ntuple filters, all the user programmed filters are flushed from337the driver cache and hardware. All needed filters must be re-added when ntuple338is re-enabled.339 340To add a filter that directs packet to queue 2, use -U or -N switch::341 342 # ethtool -N ethX flow-type tcp4 src-ip 192.168.10.1 dst-ip \343 192.168.10.2 src-port 2000 dst-port 2001 action 2 [loc 1]344 345To see the list of filters currently present::346 347 # ethtool <-u|-n> ethX348 349Sideband Perfect Filters350------------------------351Sideband Perfect Filters are used to direct traffic that matches specified352characteristics. They are enabled through ethtool's ntuple interface. To add a353new filter use the following command::354 355 ethtool -U <device> flow-type <type> src-ip <ip> dst-ip <ip> src-port <port> \356 dst-port <port> action <queue>357 358Where:359 <device> - the ethernet device to program360 <type> - can be ip4, tcp4, udp4, or sctp4361 <ip> - the IP address to match on362 <port> - the port number to match on363 <queue> - the queue to direct traffic towards (-1 discards the matched traffic)364 365Use the following command to delete a filter::366 367 ethtool -U <device> delete <N>368 369Where <N> is the filter id displayed when printing all the active filters, and370may also have been specified using "loc <N>" when adding the filter.371 372The following example matches TCP traffic sent from 192.168.0.1, port 5300,373directed to 192.168.0.5, port 80, and sends it to queue 7::374 375 ethtool -U enp130s0 flow-type tcp4 src-ip 192.168.0.1 dst-ip 192.168.0.5 \376 src-port 5300 dst-port 80 action 7377 378For each flow-type, the programmed filters must all have the same matching379input set. For example, issuing the following two commands is acceptable::380 381 ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.1 src-port 5300 action 7382 ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.5 src-port 55 action 10383 384Issuing the next two commands, however, is not acceptable, since the first385specifies src-ip and the second specifies dst-ip::386 387 ethtool -U enp130s0 flow-type ip4 src-ip 192.168.0.1 src-port 5300 action 7388 ethtool -U enp130s0 flow-type ip4 dst-ip 192.168.0.5 src-port 55 action 10389 390The second command will fail with an error. You may program multiple filters391with the same fields, using different values, but, on one device, you may not392program two TCP4 filters with different matching fields.393 394Matching on a sub-portion of a field is not supported by the ixgbe driver, thus395partial mask fields are not supported.396 397To create filters that direct traffic to a specific Virtual Function, use the398"user-def" parameter. Specify the user-def as a 64 bit value, where the lower 32399bits represents the queue number, while the next 8 bits represent which VF.400Note that 0 is the PF, so the VF identifier is offset by 1. For example::401 402 ... user-def 0x800000002 ...403 404specifies to direct traffic to Virtual Function 7 (8 minus 1) into queue 2 of405that VF.406 407Note that these filters will not break internal routing rules, and will not408route traffic that otherwise would not have been sent to the specified Virtual409Function.410 411Jumbo Frames412------------413Jumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU)414to a value larger than the default value of 1500.415 416Use the ifconfig command to increase the MTU size. For example, enter the417following where <x> is the interface number::418 419 ifconfig eth<x> mtu 9000 up420 421Alternatively, you can use the ip command as follows::422 423 ip link set mtu 9000 dev eth<x>424 ip link set up dev eth<x>425 426This setting is not saved across reboots. The setting change can be made427permanent by adding 'MTU=9000' to the file::428 429 /etc/sysconfig/network-scripts/ifcfg-eth<x> // for RHEL430 /etc/sysconfig/network/<config_file> // for SLES431 432NOTE: The maximum MTU setting for Jumbo Frames is 9710. This value coincides433with the maximum Jumbo Frames size of 9728 bytes.434 435NOTE: This driver will attempt to use multiple page sized buffers to receive436each jumbo packet. This should help to avoid buffer starvation issues when437allocating receive packets.438 439NOTE: For 82599-based network connections, if you are enabling jumbo frames in440a virtual function (VF), jumbo frames must first be enabled in the physical441function (PF). The VF MTU setting cannot be larger than the PF MTU.442 443NBASE-T Support444---------------445The ixgbe driver supports NBASE-T on some devices. However, the advertisement446of NBASE-T speeds is suppressed by default, to accommodate broken network447switches which cannot cope with advertised NBASE-T speeds. Use the ethtool448command to enable advertising NBASE-T speeds on devices which support it::449 450 ethtool -s eth? advertise 0x1800000001028451 452On Linux systems with INTERFACES(5), this can be specified as a pre-up command453in /etc/network/interfaces so that the interface is always brought up with454NBASE-T support, e.g.::455 456 iface eth? inet dhcp457 pre-up ethtool -s eth? advertise 0x1800000001028 || true458 459Generic Receive Offload, aka GRO460--------------------------------461The driver supports the in-kernel software implementation of GRO. GRO has462shown that by coalescing Rx traffic into larger chunks of data, CPU463utilization can be significantly reduced when under large Rx load. GRO is an464evolution of the previously-used LRO interface. GRO is able to coalesce465other protocols besides TCP. It's also safe to use with configurations that466are problematic for LRO, namely bridging and iSCSI.467 468Data Center Bridging (DCB)469--------------------------470NOTE:471The kernel assumes that TC0 is available, and will disable Priority Flow472Control (PFC) on the device if TC0 is not available. To fix this, ensure TC0 is473enabled when setting up DCB on your switch.474 475DCB is a configuration Quality of Service implementation in hardware. It uses476the VLAN priority tag (802.1p) to filter traffic. That means that there are 8477different priorities that traffic can be filtered into. It also enables478priority flow control (802.1Qbb) which can limit or eliminate the number of479dropped packets during network stress. Bandwidth can be allocated to each of480these priorities, which is enforced at the hardware level (802.1Qaz).481 482Adapter firmware implements LLDP and DCBX protocol agents as per 802.1AB and483802.1Qaz respectively. The firmware based DCBX agent runs in willing mode only484and can accept settings from a DCBX capable peer. Software configuration of485DCBX parameters via dcbtool/lldptool are not supported.486 487The ixgbe driver implements the DCB netlink interface layer to allow user-space488to communicate with the driver and query DCB configuration for the port.489 490ethtool491-------492The driver utilizes the ethtool interface for driver configuration and493diagnostics, as well as displaying statistical information. The latest ethtool494version is required for this functionality. Download it at:495https://www.kernel.org/pub/software/network/ethtool/496 497FCoE498----499The ixgbe driver supports Fiber Channel over Ethernet (FCoE) and Data Center500Bridging (DCB). This code has no default effect on the regular driver501operation. Configuring DCB and FCoE is outside the scope of this README. Refer502to http://www.open-fcoe.org/ for FCoE project information and contact503ixgbe-eedc@lists.sourceforge.net for DCB information.504 505MAC and VLAN anti-spoofing feature506----------------------------------507When a malicious driver attempts to send a spoofed packet, it is dropped by the508hardware and not transmitted.509 510An interrupt is sent to the PF driver notifying it of the spoof attempt. When a511spoofed packet is detected, the PF driver will send the following message to512the system log (displayed by the "dmesg" command)::513 514 ixgbe ethX: ixgbe_spoof_check: n spoofed packets detected515 516where "x" is the PF interface number; and "n" is number of spoofed packets.517NOTE: This feature can be disabled for a specific Virtual Function (VF)::518 519 ip link set <pf dev> vf <vf id> spoofchk {off|on}520 521IPsec Offload522-------------523The ixgbe driver supports IPsec Hardware Offload. When creating Security524Associations with "ip xfrm ..." the 'offload' tag option can be used to525register the IPsec SA with the driver in order to get higher throughput in526the secure communications.527 528The offload is also supported for ixgbe's VFs, but the VF must be set as529'trusted' and the support must be enabled with::530 531 ethtool --set-priv-flags eth<x> vf-ipsec on532 ip link set eth<x> vf <y> trust on533 534 535Known Issues/Troubleshooting536============================537 538Enabling SR-IOV in a 64-bit Microsoft Windows Server 2012/R2 guest OS539---------------------------------------------------------------------540Linux KVM Hypervisor/VMM supports direct assignment of a PCIe device to a VM.541This includes traditional PCIe devices, as well as SR-IOV-capable devices based542on the Intel Ethernet Controller XL710.543 544 545Support546=======547For general information, go to the Intel support website at:548https://www.intel.com/support/549 550If an issue is identified with the released source code on a supported kernel551with a supported adapter, email the specific information related to the issue552to intel-wired-lan@lists.osuosl.org.553