547 lines · plain
1.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)2 3====================4BPF LLVM Relocations5====================6 7This document describes LLVM BPF backend relocation types.8 9Relocation Record10=================11 12LLVM BPF backend records each relocation with the following 16-byte13ELF structure::14 15 typedef struct16 {17 Elf64_Addr r_offset; // Offset from the beginning of section.18 Elf64_Xword r_info; // Relocation type and symbol index.19 } Elf64_Rel;20 21For example, for the following code::22 23 int g1 __attribute__((section("sec")));24 int g2 __attribute__((section("sec")));25 static volatile int l1 __attribute__((section("sec")));26 static volatile int l2 __attribute__((section("sec")));27 int test() {28 return g1 + g2 + l1 + l2;29 }30 31Compiled with ``clang --target=bpf -O2 -c test.c``, the following is32the code with ``llvm-objdump -dr test.o``::33 34 0: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll35 0000000000000000: R_BPF_64_64 g136 2: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)37 3: 18 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r2 = 0 ll38 0000000000000018: R_BPF_64_64 g239 5: 61 20 00 00 00 00 00 00 r0 = *(u32 *)(r2 + 0)40 6: 0f 10 00 00 00 00 00 00 r0 += r141 7: 18 01 00 00 08 00 00 00 00 00 00 00 00 00 00 00 r1 = 8 ll42 0000000000000038: R_BPF_64_64 sec43 9: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)44 10: 0f 10 00 00 00 00 00 00 r0 += r145 11: 18 01 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 r1 = 12 ll46 0000000000000058: R_BPF_64_64 sec47 13: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)48 14: 0f 10 00 00 00 00 00 00 r0 += r149 15: 95 00 00 00 00 00 00 00 exit50 51There are four relocations in the above for four ``LD_imm64`` instructions.52The following ``llvm-readelf -r test.o`` shows the binary values of the four53relocations::54 55 Relocation section '.rel.text' at offset 0x190 contains 4 entries:56 Offset Info Type Symbol's Value Symbol's Name57 0000000000000000 0000000600000001 R_BPF_64_64 0000000000000000 g158 0000000000000018 0000000700000001 R_BPF_64_64 0000000000000004 g259 0000000000000038 0000000400000001 R_BPF_64_64 0000000000000000 sec60 0000000000000058 0000000400000001 R_BPF_64_64 0000000000000000 sec61 62Each relocation is represented by ``Offset`` (8 bytes) and ``Info`` (8 bytes).63For example, the first relocation corresponds to the first instruction64(Offset 0x0) and the corresponding ``Info`` indicates the relocation type65of ``R_BPF_64_64`` (type 1) and the entry in the symbol table (entry 6).66The following is the symbol table with ``llvm-readelf -s test.o``::67 68 Symbol table '.symtab' contains 8 entries:69 Num: Value Size Type Bind Vis Ndx Name70 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND71 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS test.c72 2: 0000000000000008 4 OBJECT LOCAL DEFAULT 4 l173 3: 000000000000000c 4 OBJECT LOCAL DEFAULT 4 l274 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 sec75 5: 0000000000000000 128 FUNC GLOBAL DEFAULT 2 test76 6: 0000000000000000 4 OBJECT GLOBAL DEFAULT 4 g177 7: 0000000000000004 4 OBJECT GLOBAL DEFAULT 4 g278 79The 6th entry is global variable ``g1`` with value 0.80 81Similarly, the second relocation is at ``.text`` offset ``0x18``, instruction 3,82has a type of ``R_BPF_64_64`` and refers to entry 7 in the symbol table.83The second relocation resolves to global variable ``g2`` which has a symbol84value 4. The symbol value represents the offset from the start of ``.data``85section where the initial value of the global variable ``g2`` is stored.86 87The third and fourth relocations refer to static variables ``l1``88and ``l2``. From the ``.rel.text`` section above, it is not clear89to which symbols they really refer as they both refer to90symbol table entry 4, symbol ``sec``, which has ``STT_SECTION`` type91and represents a section. So for a static variable or function,92the section offset is written to the original insn93buffer, which is called ``A`` (addend). Looking at94above insn ``7`` and ``11``, they have section offset ``8`` and ``12``.95From symbol table, we can find that they correspond to entries ``2``96and ``3`` for ``l1`` and ``l2``.97 98In general, the ``A`` is 0 for global variables and functions,99and is the section offset or some computation result based on100section offset for static variables/functions. The non-section-offset101case refers to function calls. See below for more details.102 103Different Relocation Types104==========================105 106Six relocation types are supported. The following is an overview and107``S`` represents the value of the symbol in the symbol table::108 109 Enum ELF Reloc Type Description BitSize Offset Calculation110 0 R_BPF_NONE None111 1 R_BPF_64_64 ld_imm64 insn 32 r_offset + 4 S + A112 2 R_BPF_64_ABS64 normal data 64 r_offset S + A113 3 R_BPF_64_ABS32 normal data 32 r_offset S + A114 4 R_BPF_64_NODYLD32 .BTF[.ext] data 32 r_offset S + A115 10 R_BPF_64_32 call insn 32 r_offset + 4 (S + A) / 8 - 1116 117For example, ``R_BPF_64_64`` relocation type is used for ``ld_imm64`` instruction.118The actual to-be-relocated data (0 or section offset)119is stored at ``r_offset + 4`` and the read/write120data bitsize is 32 (4 bytes). The relocation can be resolved with121the symbol value plus implicit addend. Note that the ``BitSize`` is 32 which122means the section offset must be less than or equal to ``UINT32_MAX`` and this123is enforced by LLVM BPF backend.124 125In another case, ``R_BPF_64_ABS64`` relocation type is used for normal 64-bit data.126The actual to-be-relocated data is stored at ``r_offset`` and the read/write data127bitsize is 64 (8 bytes). The relocation can be resolved with128the symbol value plus implicit addend.129 130Both ``R_BPF_64_ABS32`` and ``R_BPF_64_NODYLD32`` types are for 32-bit data.131But ``R_BPF_64_NODYLD32`` specifically refers to relocations in ``.BTF`` and132``.BTF.ext`` sections. For cases like bcc where llvm ``ExecutionEngine RuntimeDyld``133is involved, ``R_BPF_64_NODYLD32`` types of relocations should not be resolved134to actual function/variable address. Otherwise, ``.BTF`` and ``.BTF.ext``135become unusable by bcc and kernel.136 137Type ``R_BPF_64_32`` is used for call instruction. The call target section138offset is stored at ``r_offset + 4`` (32bit) and calculated as139``(S + A) / 8 - 1``.140 141Examples142========143 144Types ``R_BPF_64_64`` and ``R_BPF_64_32`` are used to resolve ``ld_imm64``145and ``call`` instructions. For example::146 147 __attribute__((noinline)) __attribute__((section("sec1")))148 int gfunc(int a, int b) {149 return a * b;150 }151 static __attribute__((noinline)) __attribute__((section("sec1")))152 int lfunc(int a, int b) {153 return a + b;154 }155 int global __attribute__((section("sec2")));156 int test(int a, int b) {157 return gfunc(a, b) + lfunc(a, b) + global;158 }159 160Compiled with ``clang --target=bpf -O2 -c test.c``, we will have161following code with `llvm-objdump -dr test.o``::162 163 Disassembly of section .text:164 165 0000000000000000 <test>:166 0: bf 26 00 00 00 00 00 00 r6 = r2167 1: bf 17 00 00 00 00 00 00 r7 = r1168 2: 85 10 00 00 ff ff ff ff call -1169 0000000000000010: R_BPF_64_32 gfunc170 3: bf 08 00 00 00 00 00 00 r8 = r0171 4: bf 71 00 00 00 00 00 00 r1 = r7172 5: bf 62 00 00 00 00 00 00 r2 = r6173 6: 85 10 00 00 02 00 00 00 call 2174 0000000000000030: R_BPF_64_32 sec1175 7: 0f 80 00 00 00 00 00 00 r0 += r8176 8: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll177 0000000000000040: R_BPF_64_64 global178 10: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)179 11: 0f 10 00 00 00 00 00 00 r0 += r1180 12: 95 00 00 00 00 00 00 00 exit181 182 Disassembly of section sec1:183 184 0000000000000000 <gfunc>:185 0: bf 20 00 00 00 00 00 00 r0 = r2186 1: 2f 10 00 00 00 00 00 00 r0 *= r1187 2: 95 00 00 00 00 00 00 00 exit188 189 0000000000000018 <lfunc>:190 3: bf 20 00 00 00 00 00 00 r0 = r2191 4: 0f 10 00 00 00 00 00 00 r0 += r1192 5: 95 00 00 00 00 00 00 00 exit193 194The first relocation corresponds to ``gfunc(a, b)`` where ``gfunc`` has a value of 0,195so the ``call`` instruction offset is ``(0 + 0)/8 - 1 = -1``.196The second relocation corresponds to ``lfunc(a, b)`` where ``lfunc`` has a section197offset ``0x18``, so the ``call`` instruction offset is ``(0 + 0x18)/8 - 1 = 2``.198The third relocation corresponds to ld_imm64 of ``global``, which has a section199offset ``0``.200 201The following is an example to show how R_BPF_64_ABS64 could be generated::202 203 int global() { return 0; }204 struct t { void *g; } gbl = { global };205 206Compiled with ``clang --target=bpf -O2 -g -c test.c``, we will see a207relocation below in ``.data`` section with command208``llvm-readelf -r test.o``::209 210 Relocation section '.rel.data' at offset 0x458 contains 1 entries:211 Offset Info Type Symbol's Value Symbol's Name212 0000000000000000 0000000700000002 R_BPF_64_ABS64 0000000000000000 global213 214The relocation says the first 8-byte of ``.data`` section should be215filled with address of ``global`` variable.216 217With ``llvm-readelf`` output, we can see that dwarf sections have a bunch of218``R_BPF_64_ABS32`` and ``R_BPF_64_ABS64`` relocations::219 220 Relocation section '.rel.debug_info' at offset 0x468 contains 13 entries:221 Offset Info Type Symbol's Value Symbol's Name222 0000000000000006 0000000300000003 R_BPF_64_ABS32 0000000000000000 .debug_abbrev223 000000000000000c 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str224 0000000000000012 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str225 0000000000000016 0000000600000003 R_BPF_64_ABS32 0000000000000000 .debug_line226 000000000000001a 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str227 000000000000001e 0000000200000002 R_BPF_64_ABS64 0000000000000000 .text228 000000000000002b 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str229 0000000000000037 0000000800000002 R_BPF_64_ABS64 0000000000000000 gbl230 0000000000000040 0000000400000003 R_BPF_64_ABS32 0000000000000000 .debug_str231 ......232 233The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::234 235 Relocation section '.rel.BTF' at offset 0x538 contains 1 entries:236 Offset Info Type Symbol's Value Symbol's Name237 0000000000000084 0000000800000004 R_BPF_64_NODYLD32 0000000000000000 gbl238 239 Relocation section '.rel.BTF.ext' at offset 0x548 contains 2 entries:240 Offset Info Type Symbol's Value Symbol's Name241 000000000000002c 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text242 0000000000000040 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text243 244.. _btf-co-re-relocations:245 246=================247CO-RE Relocations248=================249 250From object file point of view CO-RE mechanism is implemented as a set251of CO-RE specific relocation records. These relocation records are not252related to ELF relocations and are encoded in .BTF.ext section.253See :ref:`Documentation/bpf/btf.rst <BTF_Ext_Section>` for more254information on .BTF.ext structure.255 256CO-RE relocations are applied to BPF instructions to update immediate257or offset fields of the instruction at load time with information258relevant for target kernel.259 260Field to patch is selected basing on the instruction class:261 262* For BPF_ALU, BPF_ALU64, BPF_LD `immediate` field is patched;263* For BPF_LDX, BPF_STX, BPF_ST `offset` field is patched;264* BPF_JMP, BPF_JMP32 instructions **should not** be patched.265 266Relocation kinds267================268 269There are several kinds of CO-RE relocations that could be split in270three groups:271 272* Field-based - patch instruction with field related information, e.g.273 change offset field of the BPF_LDX instruction to reflect offset274 of a specific structure field in the target kernel.275 276* Type-based - patch instruction with type related information, e.g.277 change immediate field of the BPF_ALU move instruction to 0 or 1 to278 reflect if specific type is present in the target kernel.279 280* Enum-based - patch instruction with enum related information, e.g.281 change immediate field of the BPF_LD_IMM64 instruction to reflect282 value of a specific enum literal in the target kernel.283 284The complete list of relocation kinds is represented by the following enum:285 286.. code-block:: c287 288 enum bpf_core_relo_kind {289 BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */290 BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */291 BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */292 BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */293 BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */294 BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */295 BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */296 BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */297 BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */298 BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */299 BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */300 BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */301 BPF_CORE_TYPE_MATCHES = 12, /* type match in target kernel */302 };303 304Notes:305 306* ``BPF_CORE_FIELD_LSHIFT_U64`` and ``BPF_CORE_FIELD_RSHIFT_U64`` are307 supposed to be used to read bitfield values using the following308 algorithm:309 310 .. code-block:: c311 312 // To read bitfield ``f`` from ``struct s``313 is_signed = relo(s->f, BPF_CORE_FIELD_SIGNED)314 off = relo(s->f, BPF_CORE_FIELD_BYTE_OFFSET)315 sz = relo(s->f, BPF_CORE_FIELD_BYTE_SIZE)316 l = relo(s->f, BPF_CORE_FIELD_LSHIFT_U64)317 r = relo(s->f, BPF_CORE_FIELD_RSHIFT_U64)318 // define ``v`` as signed or unsigned integer of size ``sz``319 v = *({s|u}<sz> *)((void *)s + off)320 v <<= l321 v >>= r322 323* The ``BPF_CORE_TYPE_MATCHES`` queries matching relation, defined as324 follows:325 326 * for integers: types match if size and signedness match;327 * for arrays & pointers: target types are recursively matched;328 * for structs & unions:329 330 * local members need to exist in target with the same name;331 332 * for each member we recursively check match unless it is already behind a333 pointer, in which case we only check matching names and compatible kind;334 335 * for enums:336 337 * local variants have to have a match in target by symbolic name (but not338 numeric value);339 340 * size has to match (but enum may match enum64 and vice versa);341 342 * for function pointers:343 344 * number and position of arguments in local type has to match target;345 * for each argument and the return value we recursively check match.346 347CO-RE Relocation Record348=======================349 350Relocation record is encoded as the following structure:351 352.. code-block:: c353 354 struct bpf_core_relo {355 __u32 insn_off;356 __u32 type_id;357 __u32 access_str_off;358 enum bpf_core_relo_kind kind;359 };360 361* ``insn_off`` - instruction offset (in bytes) within a code section362 associated with this relocation;363 364* ``type_id`` - BTF type ID of the "root" (containing) entity of a365 relocatable type or field;366 367* ``access_str_off`` - offset into corresponding .BTF string section.368 String interpretation depends on specific relocation kind:369 370 * for field-based relocations, string encodes an accessed field using371 a sequence of field and array indices, separated by colon (:). It's372 conceptually very close to LLVM's `getelementptr <GEP_>`_ instruction's373 arguments for identifying offset to a field. For example, consider the374 following C code:375 376 .. code-block:: c377 378 struct sample {379 int a;380 int b;381 struct { int c[10]; };382 } __attribute__((preserve_access_index));383 struct sample *s;384 385 * Access to ``s[0].a`` would be encoded as ``0:0``:386 387 * ``0``: first element of ``s`` (as if ``s`` is an array);388 * ``0``: index of field ``a`` in ``struct sample``.389 390 * Access to ``s->a`` would be encoded as ``0:0`` as well.391 * Access to ``s->b`` would be encoded as ``0:1``:392 393 * ``0``: first element of ``s``;394 * ``1``: index of field ``b`` in ``struct sample``.395 396 * Access to ``s[1].c[5]`` would be encoded as ``1:2:0:5``:397 398 * ``1``: second element of ``s``;399 * ``2``: index of anonymous structure field in ``struct sample``;400 * ``0``: index of field ``c`` in anonymous structure;401 * ``5``: access to array element #5.402 403 * for type-based relocations, string is expected to be just "0";404 405 * for enum value-based relocations, string contains an index of enum406 value within its enum type;407 408* ``kind`` - one of ``enum bpf_core_relo_kind``.409 410.. _GEP: https://llvm.org/docs/LangRef.html#getelementptr-instruction411 412.. _btf_co_re_relocation_examples:413 414CO-RE Relocation Examples415=========================416 417For the following C code:418 419.. code-block:: c420 421 struct foo {422 int a;423 int b;424 unsigned c:15;425 } __attribute__((preserve_access_index));426 427 enum bar { U, V };428 429With the following BTF definitions:430 431.. code-block::432 433 ...434 [2] STRUCT 'foo' size=8 vlen=2435 'a' type_id=3 bits_offset=0436 'b' type_id=3 bits_offset=32437 'c' type_id=4 bits_offset=64 bitfield_size=15438 [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED439 [4] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)440 ...441 [16] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2442 'U' val=0443 'V' val=1444 445Field offset relocations are generated automatically when446``__attribute__((preserve_access_index))`` is used, for example:447 448.. code-block:: c449 450 void alpha(struct foo *s, volatile unsigned long *g) {451 *g = s->a;452 s->a = 1;453 }454 455 00 <alpha>:456 0: r3 = *(s32 *)(r1 + 0x0)457 00: CO-RE <byte_off> [2] struct foo::a (0:0)458 1: *(u64 *)(r2 + 0x0) = r3459 2: *(u32 *)(r1 + 0x0) = 0x1460 10: CO-RE <byte_off> [2] struct foo::a (0:0)461 3: exit462 463 464All relocation kinds could be requested via built-in functions.465E.g. field-based relocations:466 467.. code-block:: c468 469 void bravo(struct foo *s, volatile unsigned long *g) {470 *g = __builtin_preserve_field_info(s->b, 0 /* field byte offset */);471 *g = __builtin_preserve_field_info(s->b, 1 /* field byte size */);472 *g = __builtin_preserve_field_info(s->b, 2 /* field existence */);473 *g = __builtin_preserve_field_info(s->b, 3 /* field signedness */);474 *g = __builtin_preserve_field_info(s->c, 4 /* bitfield left shift */);475 *g = __builtin_preserve_field_info(s->c, 5 /* bitfield right shift */);476 }477 478 20 <bravo>:479 4: r1 = 0x4480 20: CO-RE <byte_off> [2] struct foo::b (0:1)481 5: *(u64 *)(r2 + 0x0) = r1482 6: r1 = 0x4483 30: CO-RE <byte_sz> [2] struct foo::b (0:1)484 7: *(u64 *)(r2 + 0x0) = r1485 8: r1 = 0x1486 40: CO-RE <field_exists> [2] struct foo::b (0:1)487 9: *(u64 *)(r2 + 0x0) = r1488 10: r1 = 0x1489 50: CO-RE <signed> [2] struct foo::b (0:1)490 11: *(u64 *)(r2 + 0x0) = r1491 12: r1 = 0x31492 60: CO-RE <lshift_u64> [2] struct foo::c (0:2)493 13: *(u64 *)(r2 + 0x0) = r1494 14: r1 = 0x31495 70: CO-RE <rshift_u64> [2] struct foo::c (0:2)496 15: *(u64 *)(r2 + 0x0) = r1497 16: exit498 499 500Type-based relocations:501 502.. code-block:: c503 504 void charlie(struct foo *s, volatile unsigned long *g) {505 *g = __builtin_preserve_type_info(*s, 0 /* type existence */);506 *g = __builtin_preserve_type_info(*s, 1 /* type size */);507 *g = __builtin_preserve_type_info(*s, 2 /* type matches */);508 *g = __builtin_btf_type_id(*s, 0 /* type id in this object file */);509 *g = __builtin_btf_type_id(*s, 1 /* type id in target kernel */);510 }511 512 88 <charlie>:513 17: r1 = 0x1514 88: CO-RE <type_exists> [2] struct foo515 18: *(u64 *)(r2 + 0x0) = r1516 19: r1 = 0xc517 98: CO-RE <type_size> [2] struct foo518 20: *(u64 *)(r2 + 0x0) = r1519 21: r1 = 0x1520 a8: CO-RE <type_matches> [2] struct foo521 22: *(u64 *)(r2 + 0x0) = r1522 23: r1 = 0x2 ll523 b8: CO-RE <local_type_id> [2] struct foo524 25: *(u64 *)(r2 + 0x0) = r1525 26: r1 = 0x2 ll526 d0: CO-RE <target_type_id> [2] struct foo527 28: *(u64 *)(r2 + 0x0) = r1528 29: exit529 530Enum-based relocations:531 532.. code-block:: c533 534 void delta(struct foo *s, volatile unsigned long *g) {535 *g = __builtin_preserve_enum_value(*(enum bar *)U, 0 /* enum literal existence */);536 *g = __builtin_preserve_enum_value(*(enum bar *)V, 1 /* enum literal value */);537 }538 539 f0 <delta>:540 30: r1 = 0x1 ll541 f0: CO-RE <enumval_exists> [16] enum bar::U = 0542 32: *(u64 *)(r2 + 0x0) = r1543 33: r1 = 0x1 ll544 108: CO-RE <enumval_value> [16] enum bar::V = 1545 35: *(u64 *)(r2 + 0x0) = r1546 36: exit547