1085 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3===========4Packet MMAP5===========6 7Abstract8========9 10This file documents the mmap() facility available with the PACKET11socket interface. This type of sockets is used for12 13i) capture network traffic with utilities like tcpdump,14ii) transmit network traffic, or any other that needs raw15 access to network interface.16 17Howto can be found at:18 19 https://web.archive.org/web/20220404160947/https://sites.google.com/site/packetmmap/20 21Please send your comments to22 - Ulisses Alonso Camaró <uaca@i.hate.spam.alumni.uv.es>23 - Johann Baudy24 25Why use PACKET_MMAP26===================27 28Non PACKET_MMAP capture process (plain AF_PACKET) is very29inefficient. It uses very limited buffers and requires one system call to30capture each packet, it requires two if you want to get packet's timestamp31(like libpcap always does).32 33On the other hand PACKET_MMAP is very efficient. PACKET_MMAP provides a size34configurable circular buffer mapped in user space that can be used to either35send or receive packets. This way reading packets just needs to wait for them,36most of the time there is no need to issue a single system call. Concerning37transmission, multiple packets can be sent through one system call to get the38highest bandwidth. By using a shared buffer between the kernel and the user39also has the benefit of minimizing packet copies.40 41It's fine to use PACKET_MMAP to improve the performance of the capture and42transmission process, but it isn't everything. At least, if you are capturing43at high speeds (this is relative to the cpu speed), you should check if the44device driver of your network interface card supports some sort of interrupt45load mitigation or (even better) if it supports NAPI, also make sure it is46enabled. For transmission, check the MTU (Maximum Transmission Unit) used and47supported by devices of your network. CPU IRQ pinning of your network interface48card can also be an advantage.49 50How to use mmap() to improve capture process51============================================52 53From the user standpoint, you should use the higher level libpcap library, which54is a de facto standard, portable across nearly all operating systems55including Win32.56 57Packet MMAP support was integrated into libpcap around the time of version 1.3.0;58TPACKET_V3 support was added in version 1.5.059 60How to use mmap() directly to improve capture process61=====================================================62 63From the system calls stand point, the use of PACKET_MMAP involves64the following process::65 66 67 [setup] socket() -------> creation of the capture socket68 setsockopt() ---> allocation of the circular buffer (ring)69 option: PACKET_RX_RING70 mmap() ---------> mapping of the allocated buffer to the71 user process72 73 [capture] poll() ---------> to wait for incoming packets74 75 [shutdown] close() --------> destruction of the capture socket and76 deallocation of all associated77 resources.78 79 80socket creation and destruction is straight forward, and is done81the same way with or without PACKET_MMAP::82 83 int fd = socket(PF_PACKET, mode, htons(ETH_P_ALL));84 85where mode is SOCK_RAW for the raw interface were link level86information can be captured or SOCK_DGRAM for the cooked87interface where link level information capture is not88supported and a link level pseudo-header is provided89by the kernel.90 91The destruction of the socket and all associated resources92is done by a simple call to close(fd).93 94Similarly as without PACKET_MMAP, it is possible to use one socket95for capture and transmission. This can be done by mapping the96allocated RX and TX buffer ring with a single mmap() call.97See "Mapping and use of the circular buffer (ring)".98 99Next I will describe PACKET_MMAP settings and its constraints,100also the mapping of the circular buffer in the user process and101the use of this buffer.102 103How to use mmap() directly to improve transmission process104==========================================================105Transmission process is similar to capture as shown below::106 107 [setup] socket() -------> creation of the transmission socket108 setsockopt() ---> allocation of the circular buffer (ring)109 option: PACKET_TX_RING110 bind() ---------> bind transmission socket with a network interface111 mmap() ---------> mapping of the allocated buffer to the112 user process113 114 [transmission] poll() ---------> wait for free packets (optional)115 send() ---------> send all packets that are set as ready in116 the ring117 The flag MSG_DONTWAIT can be used to return118 before end of transfer.119 120 [shutdown] close() --------> destruction of the transmission socket and121 deallocation of all associated resources.122 123Socket creation and destruction is also straight forward, and is done124the same way as in capturing described in the previous paragraph::125 126 int fd = socket(PF_PACKET, mode, 0);127 128The protocol can optionally be 0 in case we only want to transmit129via this socket, which avoids an expensive call to packet_rcv().130In this case, you also need to bind(2) the TX_RING with sll_protocol = 0131set. Otherwise, htons(ETH_P_ALL) or any other protocol, for example.132 133Binding the socket to your network interface is mandatory (with zero copy) to134know the header size of frames used in the circular buffer.135 136As capture, each frame contains two parts::137 138 --------------------139 | struct tpacket_hdr | Header. It contains the status of140 | | of this frame141 |--------------------|142 | data buffer |143 . . Data that will be sent over the network interface.144 . .145 --------------------146 147 bind() associates the socket to your network interface thanks to148 sll_ifindex parameter of struct sockaddr_ll.149 150 Initialization example::151 152 struct sockaddr_ll my_addr;153 struct ifreq s_ifr;154 ...155 156 strscpy_pad (s_ifr.ifr_name, "eth0", sizeof(s_ifr.ifr_name));157 158 /* get interface index of eth0 */159 ioctl(this->socket, SIOCGIFINDEX, &s_ifr);160 161 /* fill sockaddr_ll struct to prepare binding */162 my_addr.sll_family = AF_PACKET;163 my_addr.sll_protocol = htons(ETH_P_ALL);164 my_addr.sll_ifindex = s_ifr.ifr_ifindex;165 166 /* bind socket to eth0 */167 bind(this->socket, (struct sockaddr *)&my_addr, sizeof(struct sockaddr_ll));168 169 A complete tutorial is available at:170 https://web.archive.org/web/20220404160947/https://sites.google.com/site/packetmmap/171 172By default, the user should put data at::173 174 frame base + TPACKET_HDRLEN - sizeof(struct sockaddr_ll)175 176So, whatever you choose for the socket mode (SOCK_DGRAM or SOCK_RAW),177the beginning of the user data will be at::178 179 frame base + TPACKET_ALIGN(sizeof(struct tpacket_hdr))180 181If you wish to put user data at a custom offset from the beginning of182the frame (for payload alignment with SOCK_RAW mode for instance) you183can set tp_net (with SOCK_DGRAM) or tp_mac (with SOCK_RAW). In order184to make this work it must be enabled previously with setsockopt()185and the PACKET_TX_HAS_OFF option.186 187PACKET_MMAP settings188====================189 190To setup PACKET_MMAP from user level code is done with a call like191 192 - Capture process::193 194 setsockopt(fd, SOL_PACKET, PACKET_RX_RING, (void *) &req, sizeof(req))195 196 - Transmission process::197 198 setsockopt(fd, SOL_PACKET, PACKET_TX_RING, (void *) &req, sizeof(req))199 200The most significant argument in the previous call is the req parameter,201this parameter must to have the following structure::202 203 struct tpacket_req204 {205 unsigned int tp_block_size; /* Minimal size of contiguous block */206 unsigned int tp_block_nr; /* Number of blocks */207 unsigned int tp_frame_size; /* Size of frame */208 unsigned int tp_frame_nr; /* Total number of frames */209 };210 211This structure is defined in /usr/include/linux/if_packet.h and establishes a212circular buffer (ring) of unswappable memory.213Being mapped in the capture process allows reading the captured frames and214related meta-information like timestamps without requiring a system call.215 216Frames are grouped in blocks. Each block is a physically contiguous217region of memory and holds tp_block_size/tp_frame_size frames. The total number218of blocks is tp_block_nr. Note that tp_frame_nr is a redundant parameter because::219 220 frames_per_block = tp_block_size/tp_frame_size221 222indeed, packet_set_ring checks that the following condition is true::223 224 frames_per_block * tp_block_nr == tp_frame_nr225 226Lets see an example, with the following values::227 228 tp_block_size= 4096229 tp_frame_size= 2048230 tp_block_nr = 4231 tp_frame_nr = 8232 233we will get the following buffer structure::234 235 block #1 block #2236 +---------+---------+ +---------+---------+237 | frame 1 | frame 2 | | frame 3 | frame 4 |238 +---------+---------+ +---------+---------+239 240 block #3 block #4241 +---------+---------+ +---------+---------+242 | frame 5 | frame 6 | | frame 7 | frame 8 |243 +---------+---------+ +---------+---------+244 245A frame can be of any size with the only condition it can fit in a block. A block246can only hold an integer number of frames, or in other words, a frame cannot247be spawned across two blocks, so there are some details you have to take into248account when choosing the frame_size. See "Mapping and use of the circular249buffer (ring)".250 251PACKET_MMAP setting constraints252===============================253 254In kernel versions prior to 2.4.26 (for the 2.4 branch) and 2.6.5 (2.6 branch),255the PACKET_MMAP buffer could hold only 32768 frames in a 32 bit architecture or25616384 in a 64 bit architecture.257 258Block size limit259----------------260 261As stated earlier, each block is a contiguous physical region of memory. These262memory regions are allocated with calls to the __get_free_pages() function. As263the name indicates, this function allocates pages of memory, and the second264argument is "order" or a power of two number of pages, that is265(for PAGE_SIZE == 4096) order=0 ==> 4096 bytes, order=1 ==> 8192 bytes,266order=2 ==> 16384 bytes, etc. The maximum size of a267region allocated by __get_free_pages is determined by the MAX_PAGE_ORDER macro.268More precisely the limit can be calculated as::269 270 PAGE_SIZE << MAX_PAGE_ORDER271 272 In a i386 architecture PAGE_SIZE is 4096 bytes273 In a 2.4/i386 kernel MAX_PAGE_ORDER is 10274 In a 2.6/i386 kernel MAX_PAGE_ORDER is 11275 276So get_free_pages can allocate as much as 4MB or 8MB in a 2.4/2.6 kernel277respectively, with an i386 architecture.278 279User space programs can include /usr/include/sys/user.h and280/usr/include/linux/mmzone.h to get PAGE_SIZE MAX_PAGE_ORDER declarations.281 282The pagesize can also be determined dynamically with the getpagesize (2)283system call.284 285Block number limit286------------------287 288To understand the constraints of PACKET_MMAP, we have to see the structure289used to hold the pointers to each block.290 291Currently, this structure is a dynamically allocated vector with kmalloc292called pg_vec, its size limits the number of blocks that can be allocated::293 294 +---+---+---+---+295 | x | x | x | x |296 +---+---+---+---+297 | | | |298 | | | v299 | | v block #4300 | v block #3301 v block #2302 block #1303 304kmalloc allocates any number of bytes of physically contiguous memory from305a pool of pre-determined sizes. This pool of memory is maintained by the slab306allocator which is at the end the responsible for doing the allocation and307hence which imposes the maximum memory that kmalloc can allocate.308 309In a 2.4/2.6 kernel and the i386 architecture, the limit is 131072 bytes. The310predetermined sizes that kmalloc uses can be checked in the "size-<bytes>"311entries of /proc/slabinfo312 313In a 32 bit architecture, pointers are 4 bytes long, so the total number of314pointers to blocks is::315 316 131072/4 = 32768 blocks317 318PACKET_MMAP buffer size calculator319==================================320 321Definitions:322 323============== ================================================================324<size-max> is the maximum size of allocable with kmalloc325 (see /proc/slabinfo)326<pointer size> depends on the architecture -- ``sizeof(void *)``327<page size> depends on the architecture -- PAGE_SIZE or getpagesize (2)328<max-order> is the value defined with MAX_PAGE_ORDER329<frame size> it's an upper bound of frame's capture size (more on this later)330============== ================================================================331 332from these definitions we will derive::333 334 <block number> = <size-max>/<pointer size>335 <block size> = <pagesize> << <max-order>336 337so, the max buffer size is::338 339 <block number> * <block size>340 341and, the number of frames be::342 343 <block number> * <block size> / <frame size>344 345Suppose the following parameters, which apply for 2.6 kernel and an346i386 architecture::347 348 <size-max> = 131072 bytes349 <pointer size> = 4 bytes350 <pagesize> = 4096 bytes351 <max-order> = 11352 353and a value for <frame size> of 2048 bytes. These parameters will yield::354 355 <block number> = 131072/4 = 32768 blocks356 <block size> = 4096 << 11 = 8 MiB.357 358and hence the buffer will have a 262144 MiB size. So it can hold359262144 MiB / 2048 bytes = 134217728 frames360 361Actually, this buffer size is not possible with an i386 architecture.362Remember that the memory is allocated in kernel space, in the case of363an i386 kernel's memory size is limited to 1GiB.364 365All memory allocations are not freed until the socket is closed. The memory366allocations are done with GFP_KERNEL priority, this basically means that367the allocation can wait and swap other process' memory in order to allocate368the necessary memory, so normally limits can be reached.369 370Other constraints371-----------------372 373If you check the source code you will see that what I draw here as a frame374is not only the link level frame. At the beginning of each frame there is a375header called struct tpacket_hdr used in PACKET_MMAP to hold link level's frame376meta information like timestamp. So what we draw here a frame it's really377the following (from include/linux/if_packet.h)::378 379 /*380 Frame structure:381 382 - Start. Frame must be aligned to TPACKET_ALIGNMENT=16383 - struct tpacket_hdr384 - pad to TPACKET_ALIGNMENT=16385 - struct sockaddr_ll386 - Gap, chosen so that packet data (Start+tp_net) aligns to387 TPACKET_ALIGNMENT=16388 - Start+tp_mac: [ Optional MAC header ]389 - Start+tp_net: Packet data, aligned to TPACKET_ALIGNMENT=16.390 - Pad to align to TPACKET_ALIGNMENT=16391 */392 393The following are conditions that are checked in packet_set_ring394 395 - tp_block_size must be a multiple of PAGE_SIZE (1)396 - tp_frame_size must be greater than TPACKET_HDRLEN (obvious)397 - tp_frame_size must be a multiple of TPACKET_ALIGNMENT398 - tp_frame_nr must be exactly frames_per_block*tp_block_nr399 400Note that tp_block_size should be chosen to be a power of two or there will401be a waste of memory.402 403Mapping and use of the circular buffer (ring)404---------------------------------------------405 406The mapping of the buffer in the user process is done with the conventional407mmap function. Even the circular buffer is compound of several physically408discontiguous blocks of memory, they are contiguous to the user space, hence409just one call to mmap is needed::410 411 mmap(0, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);412 413If tp_frame_size is a divisor of tp_block_size frames will be414contiguously spaced by tp_frame_size bytes. If not, each415tp_block_size/tp_frame_size frames there will be a gap between416the frames. This is because a frame cannot be spawn across two417blocks.418 419To use one socket for capture and transmission, the mapping of both the420RX and TX buffer ring has to be done with one call to mmap::421 422 ...423 setsockopt(fd, SOL_PACKET, PACKET_RX_RING, &foo, sizeof(foo));424 setsockopt(fd, SOL_PACKET, PACKET_TX_RING, &bar, sizeof(bar));425 ...426 rx_ring = mmap(0, size * 2, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);427 tx_ring = rx_ring + size;428 429RX must be the first as the kernel maps the TX ring memory right430after the RX one.431 432At the beginning of each frame there is an status field (see433struct tpacket_hdr). If this field is 0 means that the frame is ready434to be used for the kernel, If not, there is a frame the user can read435and the following flags apply:436 437Capture process438^^^^^^^^^^^^^^^439 440From include/linux/if_packet.h::441 442 #define TP_STATUS_COPY (1 << 1)443 #define TP_STATUS_LOSING (1 << 2)444 #define TP_STATUS_CSUMNOTREADY (1 << 3)445 #define TP_STATUS_CSUM_VALID (1 << 7)446 447====================== =======================================================448TP_STATUS_COPY This flag indicates that the frame (and associated449 meta information) has been truncated because it's450 larger than tp_frame_size. This packet can be451 read entirely with recvfrom().452 453 In order to make this work it must to be454 enabled previously with setsockopt() and455 the PACKET_COPY_THRESH option.456 457 The number of frames that can be buffered to458 be read with recvfrom is limited like a normal socket.459 See the SO_RCVBUF option in the socket (7) man page.460 461TP_STATUS_LOSING indicates there were packet drops from last time462 statistics where checked with getsockopt() and463 the PACKET_STATISTICS option.464 465TP_STATUS_CSUMNOTREADY currently it's used for outgoing IP packets which466 its checksum will be done in hardware. So while467 reading the packet we should not try to check the468 checksum.469 470TP_STATUS_CSUM_VALID This flag indicates that at least the transport471 header checksum of the packet has been already472 validated on the kernel side. If the flag is not set473 then we are free to check the checksum by ourselves474 provided that TP_STATUS_CSUMNOTREADY is also not set.475====================== =======================================================476 477for convenience there are also the following defines::478 479 #define TP_STATUS_KERNEL 0480 #define TP_STATUS_USER 1481 482The kernel initializes all frames to TP_STATUS_KERNEL, when the kernel483receives a packet it puts in the buffer and updates the status with484at least the TP_STATUS_USER flag. Then the user can read the packet,485once the packet is read the user must zero the status field, so the kernel486can use again that frame buffer.487 488The user can use poll (any other variant should apply too) to check if new489packets are in the ring::490 491 struct pollfd pfd;492 493 pfd.fd = fd;494 pfd.revents = 0;495 pfd.events = POLLIN|POLLRDNORM|POLLERR;496 497 if (status == TP_STATUS_KERNEL)498 retval = poll(&pfd, 1, timeout);499 500It doesn't incur in a race condition to first check the status value and501then poll for frames.502 503Transmission process504^^^^^^^^^^^^^^^^^^^^505 506Those defines are also used for transmission::507 508 #define TP_STATUS_AVAILABLE 0 // Frame is available509 #define TP_STATUS_SEND_REQUEST 1 // Frame will be sent on next send()510 #define TP_STATUS_SENDING 2 // Frame is currently in transmission511 #define TP_STATUS_WRONG_FORMAT 4 // Frame format is not correct512 513First, the kernel initializes all frames to TP_STATUS_AVAILABLE. To send a514packet, the user fills a data buffer of an available frame, sets tp_len to515current data buffer size and sets its status field to TP_STATUS_SEND_REQUEST.516This can be done on multiple frames. Once the user is ready to transmit, it517calls send(). Then all buffers with status equal to TP_STATUS_SEND_REQUEST are518forwarded to the network device. The kernel updates each status of sent519frames with TP_STATUS_SENDING until the end of transfer.520 521At the end of each transfer, buffer status returns to TP_STATUS_AVAILABLE.522 523::524 525 header->tp_len = in_i_size;526 header->tp_status = TP_STATUS_SEND_REQUEST;527 retval = send(this->socket, NULL, 0, 0);528 529The user can also use poll() to check if a buffer is available:530 531(status == TP_STATUS_SENDING)532 533::534 535 struct pollfd pfd;536 pfd.fd = fd;537 pfd.revents = 0;538 pfd.events = POLLOUT;539 retval = poll(&pfd, 1, timeout);540 541What TPACKET versions are available and when to use them?542=========================================================543 544::545 546 int val = tpacket_version;547 setsockopt(fd, SOL_PACKET, PACKET_VERSION, &val, sizeof(val));548 getsockopt(fd, SOL_PACKET, PACKET_VERSION, &val, sizeof(val));549 550where 'tpacket_version' can be TPACKET_V1 (default), TPACKET_V2, TPACKET_V3.551 552TPACKET_V1:553 - Default if not otherwise specified by setsockopt(2)554 - RX_RING, TX_RING available555 556TPACKET_V1 --> TPACKET_V2:557 - Made 64 bit clean due to unsigned long usage in TPACKET_V1558 structures, thus this also works on 64 bit kernel with 32 bit559 userspace and the like560 - Timestamp resolution in nanoseconds instead of microseconds561 - RX_RING, TX_RING available562 - VLAN metadata information available for packets563 (TP_STATUS_VLAN_VALID, TP_STATUS_VLAN_TPID_VALID),564 in the tpacket2_hdr structure:565 566 - TP_STATUS_VLAN_VALID bit being set into the tp_status field indicates567 that the tp_vlan_tci field has valid VLAN TCI value568 - TP_STATUS_VLAN_TPID_VALID bit being set into the tp_status field569 indicates that the tp_vlan_tpid field has valid VLAN TPID value570 571 - How to switch to TPACKET_V2:572 573 1. Replace struct tpacket_hdr by struct tpacket2_hdr574 2. Query header len and save575 3. Set protocol version to 2, set up ring as usual576 4. For getting the sockaddr_ll,577 use ``(void *)hdr + TPACKET_ALIGN(hdrlen)`` instead of578 ``(void *)hdr + TPACKET_ALIGN(sizeof(struct tpacket_hdr))``579 580TPACKET_V2 --> TPACKET_V3:581 - Flexible buffer implementation for RX_RING:582 1. Blocks can be configured with non-static frame-size583 2. Read/poll is at a block-level (as opposed to packet-level)584 3. Added poll timeout to avoid indefinite user-space wait585 on idle links586 4. Added user-configurable knobs:587 588 4.1 block::timeout589 4.2 tpkt_hdr::sk_rxhash590 591 - RX Hash data available in user space592 - TX_RING semantics are conceptually similar to TPACKET_V2;593 use tpacket3_hdr instead of tpacket2_hdr, and TPACKET3_HDRLEN594 instead of TPACKET2_HDRLEN. In the current implementation,595 the tp_next_offset field in the tpacket3_hdr MUST be set to596 zero, indicating that the ring does not hold variable sized frames.597 Packets with non-zero values of tp_next_offset will be dropped.598 599AF_PACKET fanout mode600=====================601 602In the AF_PACKET fanout mode, packet reception can be load balanced among603processes. This also works in combination with mmap(2) on packet sockets.604 605Currently implemented fanout policies are:606 607 - PACKET_FANOUT_HASH: schedule to socket by skb's packet hash608 - PACKET_FANOUT_LB: schedule to socket by round-robin609 - PACKET_FANOUT_CPU: schedule to socket by CPU packet arrives on610 - PACKET_FANOUT_RND: schedule to socket by random selection611 - PACKET_FANOUT_ROLLOVER: if one socket is full, rollover to another612 - PACKET_FANOUT_QM: schedule to socket by skbs recorded queue_mapping613 614Minimal example code by David S. Miller (try things like "./test eth0 hash",615"./test eth0 lb", etc.)::616 617 #include <stddef.h>618 #include <stdlib.h>619 #include <stdio.h>620 #include <string.h>621 622 #include <sys/types.h>623 #include <sys/wait.h>624 #include <sys/socket.h>625 #include <sys/ioctl.h>626 627 #include <unistd.h>628 629 #include <linux/if_ether.h>630 #include <linux/if_packet.h>631 632 #include <net/if.h>633 634 static const char *device_name;635 static int fanout_type;636 static int fanout_id;637 638 #ifndef PACKET_FANOUT639 # define PACKET_FANOUT 18640 # define PACKET_FANOUT_HASH 0641 # define PACKET_FANOUT_LB 1642 #endif643 644 static int setup_socket(void)645 {646 int err, fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_IP));647 struct sockaddr_ll ll;648 struct ifreq ifr;649 int fanout_arg;650 651 if (fd < 0) {652 perror("socket");653 return EXIT_FAILURE;654 }655 656 memset(&ifr, 0, sizeof(ifr));657 strcpy(ifr.ifr_name, device_name);658 err = ioctl(fd, SIOCGIFINDEX, &ifr);659 if (err < 0) {660 perror("SIOCGIFINDEX");661 return EXIT_FAILURE;662 }663 664 memset(&ll, 0, sizeof(ll));665 ll.sll_family = AF_PACKET;666 ll.sll_ifindex = ifr.ifr_ifindex;667 err = bind(fd, (struct sockaddr *) &ll, sizeof(ll));668 if (err < 0) {669 perror("bind");670 return EXIT_FAILURE;671 }672 673 fanout_arg = (fanout_id | (fanout_type << 16));674 err = setsockopt(fd, SOL_PACKET, PACKET_FANOUT,675 &fanout_arg, sizeof(fanout_arg));676 if (err) {677 perror("setsockopt");678 return EXIT_FAILURE;679 }680 681 return fd;682 }683 684 static void fanout_thread(void)685 {686 int fd = setup_socket();687 int limit = 10000;688 689 if (fd < 0)690 exit(fd);691 692 while (limit-- > 0) {693 char buf[1600];694 int err;695 696 err = read(fd, buf, sizeof(buf));697 if (err < 0) {698 perror("read");699 exit(EXIT_FAILURE);700 }701 if ((limit % 10) == 0)702 fprintf(stdout, "(%d) \n", getpid());703 }704 705 fprintf(stdout, "%d: Received 10000 packets\n", getpid());706 707 close(fd);708 exit(0);709 }710 711 int main(int argc, char **argp)712 {713 int fd, err;714 int i;715 716 if (argc != 3) {717 fprintf(stderr, "Usage: %s INTERFACE {hash|lb}\n", argp[0]);718 return EXIT_FAILURE;719 }720 721 if (!strcmp(argp[2], "hash"))722 fanout_type = PACKET_FANOUT_HASH;723 else if (!strcmp(argp[2], "lb"))724 fanout_type = PACKET_FANOUT_LB;725 else {726 fprintf(stderr, "Unknown fanout type [%s]\n", argp[2]);727 exit(EXIT_FAILURE);728 }729 730 device_name = argp[1];731 fanout_id = getpid() & 0xffff;732 733 for (i = 0; i < 4; i++) {734 pid_t pid = fork();735 736 switch (pid) {737 case 0:738 fanout_thread();739 740 case -1:741 perror("fork");742 exit(EXIT_FAILURE);743 }744 }745 746 for (i = 0; i < 4; i++) {747 int status;748 749 wait(&status);750 }751 752 return 0;753 }754 755AF_PACKET TPACKET_V3 example756============================757 758AF_PACKET's TPACKET_V3 ring buffer can be configured to use non-static frame759sizes by doing its own memory management. It is based on blocks where polling760works on a per block basis instead of per ring as in TPACKET_V2 and predecessor.761 762It is said that TPACKET_V3 brings the following benefits:763 764 * ~15% - 20% reduction in CPU-usage765 * ~20% increase in packet capture rate766 * ~2x increase in packet density767 * Port aggregation analysis768 * Non static frame size to capture entire packet payload769 770So it seems to be a good candidate to be used with packet fanout.771 772Minimal example code by Daniel Borkmann based on Chetan Loke's lolpcap (compile773it with gcc -Wall -O2 blob.c, and try things like "./a.out eth0", etc.)::774 775 /* Written from scratch, but kernel-to-user space API usage776 * dissected from lolpcap:777 * Copyright 2011, Chetan Loke <loke.chetan@gmail.com>778 * License: GPL, version 2.0779 */780 781 #include <stdio.h>782 #include <stdlib.h>783 #include <stdint.h>784 #include <string.h>785 #include <assert.h>786 #include <net/if.h>787 #include <arpa/inet.h>788 #include <netdb.h>789 #include <poll.h>790 #include <unistd.h>791 #include <signal.h>792 #include <inttypes.h>793 #include <sys/socket.h>794 #include <sys/mman.h>795 #include <linux/if_packet.h>796 #include <linux/if_ether.h>797 #include <linux/ip.h>798 799 #ifndef likely800 # define likely(x) __builtin_expect(!!(x), 1)801 #endif802 #ifndef unlikely803 # define unlikely(x) __builtin_expect(!!(x), 0)804 #endif805 806 struct block_desc {807 uint32_t version;808 uint32_t offset_to_priv;809 struct tpacket_hdr_v1 h1;810 };811 812 struct ring {813 struct iovec *rd;814 uint8_t *map;815 struct tpacket_req3 req;816 };817 818 static unsigned long packets_total = 0, bytes_total = 0;819 static sig_atomic_t sigint = 0;820 821 static void sighandler(int num)822 {823 sigint = 1;824 }825 826 static int setup_socket(struct ring *ring, char *netdev)827 {828 int err, i, fd, v = TPACKET_V3;829 struct sockaddr_ll ll;830 unsigned int blocksiz = 1 << 22, framesiz = 1 << 11;831 unsigned int blocknum = 64;832 833 fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL));834 if (fd < 0) {835 perror("socket");836 exit(1);837 }838 839 err = setsockopt(fd, SOL_PACKET, PACKET_VERSION, &v, sizeof(v));840 if (err < 0) {841 perror("setsockopt");842 exit(1);843 }844 845 memset(&ring->req, 0, sizeof(ring->req));846 ring->req.tp_block_size = blocksiz;847 ring->req.tp_frame_size = framesiz;848 ring->req.tp_block_nr = blocknum;849 ring->req.tp_frame_nr = (blocksiz * blocknum) / framesiz;850 ring->req.tp_retire_blk_tov = 60;851 ring->req.tp_feature_req_word = TP_FT_REQ_FILL_RXHASH;852 853 err = setsockopt(fd, SOL_PACKET, PACKET_RX_RING, &ring->req,854 sizeof(ring->req));855 if (err < 0) {856 perror("setsockopt");857 exit(1);858 }859 860 ring->map = mmap(NULL, ring->req.tp_block_size * ring->req.tp_block_nr,861 PROT_READ | PROT_WRITE, MAP_SHARED | MAP_LOCKED, fd, 0);862 if (ring->map == MAP_FAILED) {863 perror("mmap");864 exit(1);865 }866 867 ring->rd = malloc(ring->req.tp_block_nr * sizeof(*ring->rd));868 assert(ring->rd);869 for (i = 0; i < ring->req.tp_block_nr; ++i) {870 ring->rd[i].iov_base = ring->map + (i * ring->req.tp_block_size);871 ring->rd[i].iov_len = ring->req.tp_block_size;872 }873 874 memset(&ll, 0, sizeof(ll));875 ll.sll_family = PF_PACKET;876 ll.sll_protocol = htons(ETH_P_ALL);877 ll.sll_ifindex = if_nametoindex(netdev);878 ll.sll_hatype = 0;879 ll.sll_pkttype = 0;880 ll.sll_halen = 0;881 882 err = bind(fd, (struct sockaddr *) &ll, sizeof(ll));883 if (err < 0) {884 perror("bind");885 exit(1);886 }887 888 return fd;889 }890 891 static void display(struct tpacket3_hdr *ppd)892 {893 struct ethhdr *eth = (struct ethhdr *) ((uint8_t *) ppd + ppd->tp_mac);894 struct iphdr *ip = (struct iphdr *) ((uint8_t *) eth + ETH_HLEN);895 896 if (eth->h_proto == htons(ETH_P_IP)) {897 struct sockaddr_in ss, sd;898 char sbuff[NI_MAXHOST], dbuff[NI_MAXHOST];899 900 memset(&ss, 0, sizeof(ss));901 ss.sin_family = PF_INET;902 ss.sin_addr.s_addr = ip->saddr;903 getnameinfo((struct sockaddr *) &ss, sizeof(ss),904 sbuff, sizeof(sbuff), NULL, 0, NI_NUMERICHOST);905 906 memset(&sd, 0, sizeof(sd));907 sd.sin_family = PF_INET;908 sd.sin_addr.s_addr = ip->daddr;909 getnameinfo((struct sockaddr *) &sd, sizeof(sd),910 dbuff, sizeof(dbuff), NULL, 0, NI_NUMERICHOST);911 912 printf("%s -> %s, ", sbuff, dbuff);913 }914 915 printf("rxhash: 0x%x\n", ppd->hv1.tp_rxhash);916 }917 918 static void walk_block(struct block_desc *pbd, const int block_num)919 {920 int num_pkts = pbd->h1.num_pkts, i;921 unsigned long bytes = 0;922 struct tpacket3_hdr *ppd;923 924 ppd = (struct tpacket3_hdr *) ((uint8_t *) pbd +925 pbd->h1.offset_to_first_pkt);926 for (i = 0; i < num_pkts; ++i) {927 bytes += ppd->tp_snaplen;928 display(ppd);929 930 ppd = (struct tpacket3_hdr *) ((uint8_t *) ppd +931 ppd->tp_next_offset);932 }933 934 packets_total += num_pkts;935 bytes_total += bytes;936 }937 938 static void flush_block(struct block_desc *pbd)939 {940 pbd->h1.block_status = TP_STATUS_KERNEL;941 }942 943 static void teardown_socket(struct ring *ring, int fd)944 {945 munmap(ring->map, ring->req.tp_block_size * ring->req.tp_block_nr);946 free(ring->rd);947 close(fd);948 }949 950 int main(int argc, char **argp)951 {952 int fd, err;953 socklen_t len;954 struct ring ring;955 struct pollfd pfd;956 unsigned int block_num = 0, blocks = 64;957 struct block_desc *pbd;958 struct tpacket_stats_v3 stats;959 960 if (argc != 2) {961 fprintf(stderr, "Usage: %s INTERFACE\n", argp[0]);962 return EXIT_FAILURE;963 }964 965 signal(SIGINT, sighandler);966 967 memset(&ring, 0, sizeof(ring));968 fd = setup_socket(&ring, argp[argc - 1]);969 assert(fd > 0);970 971 memset(&pfd, 0, sizeof(pfd));972 pfd.fd = fd;973 pfd.events = POLLIN | POLLERR;974 pfd.revents = 0;975 976 while (likely(!sigint)) {977 pbd = (struct block_desc *) ring.rd[block_num].iov_base;978 979 if ((pbd->h1.block_status & TP_STATUS_USER) == 0) {980 poll(&pfd, 1, -1);981 continue;982 }983 984 walk_block(pbd, block_num);985 flush_block(pbd);986 block_num = (block_num + 1) % blocks;987 }988 989 len = sizeof(stats);990 err = getsockopt(fd, SOL_PACKET, PACKET_STATISTICS, &stats, &len);991 if (err < 0) {992 perror("getsockopt");993 exit(1);994 }995 996 fflush(stdout);997 printf("\nReceived %u packets, %lu bytes, %u dropped, freeze_q_cnt: %u\n",998 stats.tp_packets, bytes_total, stats.tp_drops,999 stats.tp_freeze_q_cnt);1000 1001 teardown_socket(&ring, fd);1002 return 0;1003 }1004 1005PACKET_QDISC_BYPASS1006===================1007 1008If there is a requirement to load the network with many packets in a similar1009fashion as pktgen does, you might set the following option after socket1010creation::1011 1012 int one = 1;1013 setsockopt(fd, SOL_PACKET, PACKET_QDISC_BYPASS, &one, sizeof(one));1014 1015This has the side-effect, that packets sent through PF_PACKET will bypass the1016kernel's qdisc layer and are forcedly pushed to the driver directly. Meaning,1017packet are not buffered, tc disciplines are ignored, increased loss can occur1018and such packets are also not visible to other PF_PACKET sockets anymore. So,1019you have been warned; generally, this can be useful for stress testing various1020components of a system.1021 1022On default, PACKET_QDISC_BYPASS is disabled and needs to be explicitly enabled1023on PF_PACKET sockets.1024 1025PACKET_TIMESTAMP1026================1027 1028The PACKET_TIMESTAMP setting determines the source of the timestamp in1029the packet meta information for mmap(2)ed RX_RING and TX_RINGs. If your1030NIC is capable of timestamping packets in hardware, you can request those1031hardware timestamps to be used. Note: you may need to enable the generation1032of hardware timestamps with SIOCSHWTSTAMP (see related information from1033Documentation/networking/timestamping.rst).1034 1035PACKET_TIMESTAMP accepts the same integer bit field as SO_TIMESTAMPING::1036 1037 int req = SOF_TIMESTAMPING_RAW_HARDWARE;1038 setsockopt(fd, SOL_PACKET, PACKET_TIMESTAMP, (void *) &req, sizeof(req))1039 1040For the mmap(2)ed ring buffers, such timestamps are stored in the1041``tpacket{,2,3}_hdr`` structure's tp_sec and ``tp_{n,u}sec`` members.1042To determine what kind of timestamp has been reported, the tp_status field1043is binary or'ed with the following possible bits ...1044 1045::1046 1047 TP_STATUS_TS_RAW_HARDWARE1048 TP_STATUS_TS_SOFTWARE1049 1050... that are equivalent to its ``SOF_TIMESTAMPING_*`` counterparts. For the1051RX_RING, if neither is set (i.e. PACKET_TIMESTAMP is not set), then a1052software fallback was invoked *within* PF_PACKET's processing code (less1053precise).1054 1055Getting timestamps for the TX_RING works as follows: i) fill the ring frames,1056ii) call sendto() e.g. in blocking mode, iii) wait for status of relevant1057frames to be updated resp. the frame handed over to the application, iv) walk1058through the frames to pick up the individual hw/sw timestamps.1059 1060Only (!) if transmit timestamping is enabled, then these bits are combined1061with binary | with TP_STATUS_AVAILABLE, so you must check for that in your1062application (e.g. !(tp_status & (TP_STATUS_SEND_REQUEST | TP_STATUS_SENDING))1063in a first step to see if the frame belongs to the application, and then1064one can extract the type of timestamp in a second step from tp_status)!1065 1066If you don't care about them, thus having it disabled, checking for1067TP_STATUS_AVAILABLE resp. TP_STATUS_WRONG_FORMAT is sufficient. If in the1068TX_RING part only TP_STATUS_AVAILABLE is set, then the tp_sec and tp_{n,u}sec1069members do not contain a valid value. For TX_RINGs, by default no timestamp1070is generated!1071 1072See include/linux/net_tstamp.h and Documentation/networking/timestamping.rst1073for more information on hardware timestamps.1074 1075Miscellaneous bits1076==================1077 1078- Packet sockets work well together with Linux socket filters, thus you also1079 might want to have a look at Documentation/networking/filter.rst1080 1081THANKS1082======1083 1084 Jesse Brandeburg, for fixing my grammathical/spelling errors1085