300 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=====================================4Network Devices, the Kernel, and You!5=====================================6 7 8Introduction9============10The following is a random collection of documentation regarding11network devices.12 13struct net_device lifetime rules14================================15Network device structures need to persist even after module is unloaded and16must be allocated with alloc_netdev_mqs() and friends.17If device has registered successfully, it will be freed on last use18by free_netdev(). This is required to handle the pathological case cleanly19(example: ``rmmod mydriver </sys/class/net/myeth/mtu``)20 21alloc_netdev_mqs() / alloc_netdev() reserve extra space for driver22private data which gets freed when the network device is freed. If23separately allocated data is attached to the network device24(netdev_priv()) then it is up to the module exit handler to free that.25 26There are two groups of APIs for registering struct net_device.27First group can be used in normal contexts where ``rtnl_lock`` is not already28held: register_netdev(), unregister_netdev().29Second group can be used when ``rtnl_lock`` is already held:30register_netdevice(), unregister_netdevice(), free_netdevice().31 32Simple drivers33--------------34 35Most drivers (especially device drivers) handle lifetime of struct net_device36in context where ``rtnl_lock`` is not held (e.g. driver probe and remove paths).37 38In that case the struct net_device registration is done using39the register_netdev(), and unregister_netdev() functions:40 41.. code-block:: c42 43 int probe()44 {45 struct my_device_priv *priv;46 int err;47 48 dev = alloc_netdev_mqs(...);49 if (!dev)50 return -ENOMEM;51 priv = netdev_priv(dev);52 53 /* ... do all device setup before calling register_netdev() ...54 */55 56 err = register_netdev(dev);57 if (err)58 goto err_undo;59 60 /* net_device is visible to the user! */61 62 err_undo:63 /* ... undo the device setup ... */64 free_netdev(dev);65 return err;66 }67 68 void remove()69 {70 unregister_netdev(dev);71 free_netdev(dev);72 }73 74Note that after calling register_netdev() the device is visible in the system.75Users can open it and start sending / receiving traffic immediately,76or run any other callback, so all initialization must be done prior to77registration.78 79unregister_netdev() closes the device and waits for all users to be done80with it. The memory of struct net_device itself may still be referenced81by sysfs but all operations on that device will fail.82 83free_netdev() can be called after unregister_netdev() returns on when84register_netdev() failed.85 86Device management under RTNL87----------------------------88 89Registering struct net_device while in context which already holds90the ``rtnl_lock`` requires extra care. In those scenarios most drivers91will want to make use of struct net_device's ``needs_free_netdev``92and ``priv_destructor`` members for freeing of state.93 94Example flow of netdev handling under ``rtnl_lock``:95 96.. code-block:: c97 98 static void my_setup(struct net_device *dev)99 {100 dev->needs_free_netdev = true;101 }102 103 static void my_destructor(struct net_device *dev)104 {105 some_obj_destroy(priv->obj);106 some_uninit(priv);107 }108 109 int create_link()110 {111 struct my_device_priv *priv;112 int err;113 114 ASSERT_RTNL();115 116 dev = alloc_netdev(sizeof(*priv), "net%d", NET_NAME_UNKNOWN, my_setup);117 if (!dev)118 return -ENOMEM;119 priv = netdev_priv(dev);120 121 /* Implicit constructor */122 err = some_init(priv);123 if (err)124 goto err_free_dev;125 126 priv->obj = some_obj_create();127 if (!priv->obj) {128 err = -ENOMEM;129 goto err_some_uninit;130 }131 /* End of constructor, set the destructor: */132 dev->priv_destructor = my_destructor;133 134 err = register_netdevice(dev);135 if (err)136 /* register_netdevice() calls destructor on failure */137 goto err_free_dev;138 139 /* If anything fails now unregister_netdevice() (or unregister_netdev())140 * will take care of calling my_destructor and free_netdev().141 */142 143 return 0;144 145 err_some_uninit:146 some_uninit(priv);147 err_free_dev:148 free_netdev(dev);149 return err;150 }151 152If struct net_device.priv_destructor is set it will be called by the core153some time after unregister_netdevice(), it will also be called if154register_netdevice() fails. The callback may be invoked with or without155``rtnl_lock`` held.156 157There is no explicit constructor callback, driver "constructs" the private158netdev state after allocating it and before registration.159 160Setting struct net_device.needs_free_netdev makes core call free_netdevice()161automatically after unregister_netdevice() when all references to the device162are gone. It only takes effect after a successful call to register_netdevice()163so if register_netdevice() fails driver is responsible for calling164free_netdev().165 166free_netdev() is safe to call on error paths right after unregister_netdevice()167or when register_netdevice() fails. Parts of netdev (de)registration process168happen after ``rtnl_lock`` is released, therefore in those cases free_netdev()169will defer some of the processing until ``rtnl_lock`` is released.170 171Devices spawned from struct rtnl_link_ops should never free the172struct net_device directly.173 174.ndo_init and .ndo_uninit175~~~~~~~~~~~~~~~~~~~~~~~~~176 177``.ndo_init`` and ``.ndo_uninit`` callbacks are called during net_device178registration and de-registration, under ``rtnl_lock``. Drivers can use179those e.g. when parts of their init process need to run under ``rtnl_lock``.180 181``.ndo_init`` runs before device is visible in the system, ``.ndo_uninit``182runs during de-registering after device is closed but other subsystems183may still have outstanding references to the netdevice.184 185MTU186===187Each network device has a Maximum Transfer Unit. The MTU does not188include any link layer protocol overhead. Upper layer protocols must189not pass a socket buffer (skb) to a device to transmit with more data190than the mtu. The MTU does not include link layer header overhead, so191for example on Ethernet if the standard MTU is 1500 bytes used, the192actual skb will contain up to 1514 bytes because of the Ethernet193header. Devices should allow for the 4 byte VLAN header as well.194 195Segmentation Offload (GSO, TSO) is an exception to this rule. The196upper layer protocol may pass a large socket buffer to the device197transmit routine, and the device will break that up into separate198packets based on the current MTU.199 200MTU is symmetrical and applies both to receive and transmit. A device201must be able to receive at least the maximum size packet allowed by202the MTU. A network device may use the MTU as mechanism to size receive203buffers, but the device should allow packets with VLAN header. With204standard Ethernet mtu of 1500 bytes, the device should allow up to2051518 byte packets (1500 + 14 header + 4 tag). The device may either:206drop, truncate, or pass up oversize packets, but dropping oversize207packets is preferred.208 209 210struct net_device synchronization rules211=======================================212ndo_open:213 Synchronization: rtnl_lock() semaphore.214 Context: process215 216ndo_stop:217 Synchronization: rtnl_lock() semaphore.218 Context: process219 Note: netif_running() is guaranteed false220 221ndo_do_ioctl:222 Synchronization: rtnl_lock() semaphore.223 Context: process224 225 This is only called by network subsystems internally,226 not by user space calling ioctl as it was in before227 linux-5.14.228 229ndo_siocbond:230 Synchronization: rtnl_lock() semaphore.231 Context: process232 233 Used by the bonding driver for the SIOCBOND family of234 ioctl commands.235 236ndo_siocwandev:237 Synchronization: rtnl_lock() semaphore.238 Context: process239 240 Used by the drivers/net/wan framework to handle241 the SIOCWANDEV ioctl with the if_settings structure.242 243ndo_siocdevprivate:244 Synchronization: rtnl_lock() semaphore.245 Context: process246 247 This is used to implement SIOCDEVPRIVATE ioctl helpers.248 These should not be added to new drivers, so don't use.249 250ndo_eth_ioctl:251 Synchronization: rtnl_lock() semaphore.252 Context: process253 254ndo_get_stats:255 Synchronization: rtnl_lock() semaphore, or RCU.256 Context: atomic (can't sleep under RCU)257 258ndo_start_xmit:259 Synchronization: __netif_tx_lock spinlock.260 261 When the driver sets dev->lltx this will be262 called without holding netif_tx_lock. In this case the driver263 has to lock by itself when needed.264 The locking there should also properly protect against265 set_rx_mode. WARNING: use of dev->lltx is deprecated.266 Don't use it for new drivers.267 268 Context: Process with BHs disabled or BH (timer),269 will be called with interrupts disabled by netconsole.270 271 Return codes:272 273 * NETDEV_TX_OK everything ok.274 * NETDEV_TX_BUSY Cannot transmit packet, try later275 Usually a bug, means queue start/stop flow control is broken in276 the driver. Note: the driver must NOT put the skb in its DMA ring.277 278ndo_tx_timeout:279 Synchronization: netif_tx_lock spinlock; all TX queues frozen.280 Context: BHs disabled281 Notes: netif_queue_stopped() is guaranteed true282 283ndo_set_rx_mode:284 Synchronization: netif_addr_lock spinlock.285 Context: BHs disabled286 287struct napi_struct synchronization rules288========================================289napi->poll:290 Synchronization:291 NAPI_STATE_SCHED bit in napi->state. Device292 driver's ndo_stop method will invoke napi_disable() on293 all NAPI instances which will do a sleeping poll on the294 NAPI_STATE_SCHED napi->state bit, waiting for all pending295 NAPI activity to cease.296 297 Context:298 softirq299 will be called with interrupts disabled by netconsole.300