1008 lines · plain
1==================================================2``-fbounds-safety``: Enforcing bounds safety for C3==================================================4 5.. contents::6 :local:7 8Overview9========10 11**NOTE:** This is a design document and the feature is not available for users yet.12Please see :doc:`BoundsSafetyImplPlans` for more details.13 14``-fbounds-safety`` is a C extension to enforce bounds safety to prevent15out-of-bounds (OOB) memory accesses, which remain a major source of security16vulnerabilities in C. ``-fbounds-safety`` aims to eliminate this class of bugs17by turning OOB accesses into deterministic traps.18 19The ``-fbounds-safety`` extension offers bounds annotations that programmers can20use to attach bounds to pointers. For example, programmers can add the21``__counted_by(N)`` annotation to parameter ``ptr``, indicating that the pointer22has ``N`` valid elements:23 24.. code-block:: c25 26 void foo(int *__counted_by(N) ptr, size_t N);27 28Using this bounds information, the compiler inserts bounds checks on every29pointer dereference, ensuring that the program does not access memory outside30the specified bounds. The compiler requires programmers to provide enough bounds31information so that the accesses can be checked at either run time or compile32time — and it rejects code if it cannot.33 34The most important contribution of ``-fbounds-safety`` is how it reduces the35programmer's annotation burden by reconciling bounds annotations at ABI36boundaries with the use of implicit wide pointers (a.k.a. "fat" pointers) that37carry bounds information on local variables without the need for annotations. We38designed this model so that it preserves ABI compatibility with C while39minimizing adoption effort.40 41The ``-fbounds-safety`` extension has been adopted on millions of lines of42production C code and proven to work in a consumer operating system setting. The43extension was designed to enable incremental adoption — a key requirement in44real-world settings where modifying an entire project and its dependencies all45at once is often not possible. It also addresses multiple of other practical46challenges that have made existing approaches to safer C dialects difficult to47adopt, offering these properties that make it widely adoptable in practice:48 49* It is designed to preserve the Application Binary Interface (ABI).50* It interoperates well with plain C code.51* It can be adopted partially and incrementally while still providing safety52 benefits.53* It is a conforming extension to C.54* Consequently, source code that adopts the extension can continue to be55 compiled by toolchains that do not support the extension (CAVEAT: this still56 requires inclusion of a header file macro-defining bounds annotations to57 empty).58* It has a relatively low adoption cost.59 60This document discusses the key designs of ``-fbounds-safety``. The document is61subject to active updates with a more detailed specification.62 63Programming Model64=================65 66Overview67--------68 69``-fbounds-safety`` ensures that pointers are not used to access memory beyond70their bounds by performing bounds checking. If a bounds check fails, the program71will deterministically trap before out-of-bounds memory is accessed.72 73In our model, every pointer has an explicit or implicit bounds attribute that74determines its bounds and ensures guaranteed bounds checking. Consider the75example below where the ``__counted_by(count)`` annotation indicates that76parameter ``p`` points to a buffer of integers containing ``count`` elements. An77off-by-one error is present in the loop condition, leading to ``p[i]`` being78out-of-bounds access during the loop's final iteration. The compiler inserts a79bounds check before ``p`` is dereferenced to ensure that the access remains80within the specified bounds.81 82.. code-block:: c83 84 void fill_array_with_indices(int *__counted_by(count) p, unsigned count) {85 // off-by-one error (i < count)86 for (unsigned i = 0; i <= count; ++i) {87 // bounds check inserted:88 // if (i >= count) trap();89 p[i] = i;90 }91 }92 93A bounds annotation defines an invariant for the pointer type, and the model94ensures that this invariant remains true. In the example below, pointer ``p``95annotated with ``__counted_by(count)`` must always point to a memory buffer96containing at least ``count`` elements of the pointee type. Changing the value97of ``count``, like in the example below, may violate this invariant and permit98out-of-bounds access to the pointer. To avoid this, the compiler employs99compile-time restrictions and emits run-time checks as necessary to ensure the100new count value doesn't exceed the actual length of the buffer. Section101`Maintaining correctness of bounds annotations`_ provides more details about102this programming model.103 104.. code-block:: c105 106 int g;107 108 void foo(int *__counted_by(count) p, size_t count) {109 count++; // may violate the invariant of __counted_by110 count--; // may violate the invariant of __counted_by if count was 0.111 count = g; // may violate the invariant of __counted_by112 // depending on the value of `g`.113 }114 115The requirement to annotate all pointers with explicit bounds information could116present a significant adoption burden. To tackle this issue, the model117incorporates the concept of a "wide pointer" (a.k.a. fat pointer) – a larger118pointer that carries bounds information alongside the pointer value. Utilizing119wide pointers can potentially reduce the adoption burden, as it contains bounds120information internally and eliminates the need for explicit bounds annotations.121However, wide pointers differ from standard C pointers in their data layout,122which may result in incompatibilities with the application binary interface123(ABI). Breaking the ABI complicates interoperability with external code that has124not adopted the same programming model.125 126``-fbounds-safety`` harmonizes the wide pointer and the bounds annotation127approaches to reduce the adoption burden while maintaining the ABI. In this128model, local variables of pointer type are implicitly treated as wide pointers,129allowing them to carry bounds information without requiring explicit bounds130annotations. Please note that this approach doesn't apply to function parameters131which are considered ABI-visible. As local variables are typically hidden from132the ABI, this approach has a marginal impact on it. In addition,133``-fbounds-safety`` employs compile-time restrictions to prevent implicit wide134pointers from silently breaking the ABI (see `ABI implications of default bounds135annotations`_). Pointers associated with any other variables, including function136parameters, are treated as single object pointers (i.e., ``__single``), ensuring137that they always have the tightest bounds by default and offering a strong138bounds safety guarantee.139 140By implementing default bounds annotations based on ABI visibility, a141considerable portion of C code can operate without modifications within this142programming model, reducing the adoption burden.143 144The rest of the section will discuss individual bounds annotations and the145programming model in more detail.146 147Bounds annotations148------------------149 150Annotation for pointers to a single object151^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^152 153The C language allows pointer arithmetic on arbitrary pointers and this has been154a source of many bounds safety issues. In practice, many pointers are merely155pointing to a single object and incrementing or decrementing such a pointer156immediately makes the pointer go out-of-bounds. To prevent this unsafety,157``-fbounds-safety`` provides the annotation ``__single`` that causes pointer158arithmetic on annotated pointers to be a compile time error.159 160* ``__single`` : indicates that the pointer is either pointing to a single161 object or null. Hence, pointers with ``__single`` do not permit pointer162 arithmetic nor being subscripted with a non-zero index. Dereferencing a163 ``__single`` pointer is allowed but it requires a null check. Upper and lower164 bounds checks are not required because the ``__single`` pointer should point165 to a valid object unless it's null.166 167``__single`` is the default annotation for ABI-visible pointers. This168gives strong security guarantees in that these pointers cannot be incremented or169decremented unless they have an explicit, overriding bounds annotation that can170be used to verify the safety of the operation. The compiler issues an error when171a ``__single`` pointer is utilized for pointer arithmetic or array access, as172these operations would immediately cause the pointer to exceed its bounds.173Consequently, this prompts programmers to provide sufficient bounds information174to pointers. In the following example, the pointer on parameter p is175single-by-default, and is employed for array access. As a result, the compiler176generates an error suggesting to add ``__counted_by`` to the pointer.177 178.. code-block:: c179 180 void fill_array_with_indices(int *p, unsigned count) {181 for (unsigned i = 0; i < count; ++i) {182 p[i] = i; // error183 }184 }185 186 187External bounds annotations188^^^^^^^^^^^^^^^^^^^^^^^^^^^189 190"External" bounds annotations provide a way to express a relationship between a191pointer variable and another variable (or expression) containing the bounds192information of the pointer. In the following example, ``__counted_by(count)``193annotation expresses the bounds of parameter p using another parameter count.194This model works naturally with many C interfaces and structs because the bounds195of a pointer is often available adjacent to the pointer itself, e.g., at another196parameter of the same function prototype, or at another field of the same struct197declaration.198 199.. code-block:: c200 201 void fill_array_with_indices(int *__counted_by(count) p, size_t count) {202 // off-by-one error203 for (size_t i = 0; i <= count; ++i)204 p[i] = i;205 }206 207External bounds annotations include ``__counted_by``, ``__sized_by``, and208``__ended_by``. These annotations do not change the pointer representation,209meaning they do not have ABI implications.210 211* ``__counted_by(N)`` : The pointer points to memory that contains ``N``212 elements of pointee type. ``N`` is an expression of integer type which can be213 a simple reference to declaration, a constant including calls to constant214 functions, or an arithmetic expression that does not have side effect. The215 ``__counted_by`` annotation cannot apply to pointers to incomplete types or216 types without size such as ``void *``. Instead, ``__sized_by`` can be used to217 describe the byte count.218* ``__sized_by(N)`` : The pointer points to memory that contains ``N`` bytes.219 Just like the argument of ``__counted_by``, ``N`` is an expression of integer220 type which can be a constant, a simple reference to a declaration, or an221 arithmetic expression that does not have side effects. This is mainly used for222 pointers to incomplete types or types without size such as ``void *``.223* ``__ended_by(P)`` : The pointer has the upper bound of value ``P``, which is224 one past the last element of the pointer. In other words, this annotation225 describes a range that starts with the pointer that has this annotation and226 ends with ``P`` which is the argument of the annotation. ``P`` itself may be227 annotated with ``__ended_by(Q)``. In this case, the end of the range extends228 to the pointer ``Q``. This is used for "iterator" support in C where you're229 iterating from one pointer value to another until a final pointer value is230 reached (and the final pointer value is not dereferenceable).231 232Accessing a pointer outside the specified bounds causes a run-time trap or a233compile-time error. Also, the model maintains correctness of bounds annotations234when the pointer and/or the related value containing the bounds information are235updated or passed as arguments. This is done by compile-time restrictions or236run-time checks (see `Maintaining correctness of bounds annotations`_237for more detail). For instance, initializing ``buf`` with ``null`` while238assigning non-zero value to ``count``, as shown in the following example, would239violate the ``__counted_by`` annotation because a null pointer does not point to240any valid memory location. To avoid this, the compiler produces either a241compile-time error or run-time trap.242 243.. code-block:: c244 245 void null_with_count_10(int *__counted_by(count) buf, unsigned count) {246 buf = 0;247 // This is not allowed as it creates a null pointer with non-zero length248 count = 10;249 }250 251However, there are use cases where a pointer is either a null pointer or is252pointing to memory of the specified size. To support this idiom,253``-fbounds-safety`` provides ``*_or_null`` variants,254``__counted_by_or_null(N)``, ``__sized_by_or_null(N)``, and255``__ended_by_or_null(P)``. Accessing a pointer with any of these bounds256annotations will require an extra null check to avoid a null pointer257dereference.258 259Internal bounds annotations260^^^^^^^^^^^^^^^^^^^^^^^^^^^261 262A wide pointer (sometimes known as a "fat" pointer) is a pointer that carries263additional bounds information internally (as part of its data). The bounds264require additional storage space making wide pointers larger than normal265pointers, hence the name "wide pointer". The memory layout of a wide pointer is266equivalent to a struct with the pointer, upper bound, and (optionally) lower267bound as its fields as shown below.268 269.. code-block:: c270 271 struct wide_pointer_datalayout {272 void* pointer; // Address used for dereferences and pointer arithmetic273 void* upper_bound; // Points one past the highest address that can be274 // accessed275 void* lower_bound; // (Optional) Points to lowest address that can be276 // accessed277 };278 279Even with this representational change, wide pointers act syntactically as280normal pointers to allow standard pointer operations, such as pointer281dereference (``*p``), array subscript (``p[i]``), member access (``p->``), and282pointer arithmetic, with some restrictions on bounds-unsafe uses.283 284``-fbounds-safety`` has a set of "internal" bounds annotations to turn pointers285into wide pointers. These are ``__bidi_indexable`` and ``__indexable``. When a286pointer has either of these annotations, the compiler changes the pointer to the287corresponding wide pointer. This means these annotations will break the ABI and288will not be compatible with plain C, and thus they should generally not be used289in ABI surfaces.290 291* ``__bidi_indexable`` : A pointer with this annotation becomes a wide pointer292 to carry the upper bound and the lower bound, the layout of which is293 equivalent to ``struct { T *ptr; T *upper_bound; T *lower_bound; };``. As the294 name indicates, pointers with this annotation are "bidirectionally indexable",295 meaning that they can be indexed with either a negative or a positive offset296 and the pointers can be incremented or decremented using pointer arithmetic. A297 ``__bidi_indexable`` pointer is allowed to hold an out-of-bounds pointer298 value. While creating an OOB pointer is undefined behavior in C,299 ``-fbounds-safety`` makes it well-defined behavior. That is, pointer300 arithmetic overflow with ``__bidi_indexable`` is defined as equivalent of301 two's complement integer computation, and at the LLVM IR level this means302 ``getelementptr`` won't get ``inbounds`` keyword. Accessing memory using the303 OOB pointer is prevented via a run-time bounds check.304 305* ``__indexable`` : A pointer with this annotation becomes a wide pointer306 carrying the upper bound (but no explicit lower bound), the layout of which is307 equivalent to ``struct { T *ptr; T *upper_bound; };``. Since ``__indexable``308 pointers do not have a separate lower bound, the pointer value itself acts as309 the lower bound. An ``__indexable`` pointer can only be incremented or indexed310 in the positive direction. Indexing it in the negative direction will trigger311 a compile-time error. Otherwise, the compiler inserts a run-time312 check to ensure pointer arithmetic doesn't make the pointer smaller than the313 original ``__indexable`` pointer (Note that ``__indexable`` doesn't have a314 lower bound so the pointer value is effectively the lower bound). As pointer315 arithmetic overflow will make the pointer smaller than the original pointer,316 it will cause a trap at runtime. Similar to ``__bidi_indexable``, an317 ``__indexable`` pointer is allowed to have a pointer value above the upper318 bound and creating such a pointer is well-defined behavior. Dereferencing such319 a pointer, however, will cause a run-time trap.320 321* ``__bidi_indexable`` offers the best flexibility out of all the pointer322 annotations in this model, as ``__bidi_indexable`` pointers can be used for323 any pointer operation. However, this comes with the largest code size and324 memory cost out of the available pointer annotations in this model. In some325 cases, use of the ``__bidi_indexable`` annotation may be duplicating bounds326 information that exists elsewhere in the program. In such cases, using327 external bounds annotations may be a better choice.328 329``__bidi_indexable`` is the default annotation for non-ABI visible pointers,330such as local pointer variables — that is, if the programmer does not specify331another bounds annotation, a local pointer variable is implicitly332``__bidi_indexable``. Since ``__bidi_indexable`` pointers automatically carry333bounds information and have no restrictions on kinds of pointer operations that334can be used with these pointers, most code inside a function works as is without335modification. In the example below, ``int *buf`` doesn't require manual336annotation as it's implicitly ``int *__bidi_indexable buf``, carrying the bounds337information passed from the return value of malloc, which is necessary to insert338bounds checking for ``buf[i]``.339 340.. code-block:: c341 342 void *__sized_by(size) malloc(size_t size);343 344 int *__counted_by(n) get_array_with_0_to_n_1(size_t n) {345 int *buf = malloc(sizeof(int) * n);346 for (size_t i = 0; i < n; ++i)347 buf[i] = i;348 return buf;349 }350 351Annotations for sentinel-delimited arrays352^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^353 354A C string is an array of characters. The null terminator — the first null355character ('\0') element in the array — marks the end of the string.356``-fbounds-safety`` provides ``__null_terminated`` to annotate C strings and the357generalized form ``__terminated_by(T)`` to annotate pointers and arrays with an358end marked by a sentinel value. The model prevents dereferencing a359``__terminated_by`` pointer beyond its end. Calculating the location of the end360(i.e., the address of the sentinel value), requires reading the entire array in361memory and would have some performance costs. To avoid an unintended performance362hit, the model puts some restrictions on how these pointers can be used.363``__terminated_by`` pointers cannot be indexed and can only be incremented one364element at a time. To allow these operations, the pointers must be explicitly365converted to ``__indexable`` pointers using the intrinsic function366``__unsafe_terminated_by_to_indexable(P, T)`` (or367``__unsafe_null_terminated_to_indexable(P)``) which converts the368``__terminated_by`` pointer ``P`` to an ``__indexable`` pointer.369 370* ``__null_terminated`` : The pointer or array is terminated by ``NULL`` or371 ``0``. Modifying the terminator or incrementing the pointer beyond it is372 prevented at run time.373 374* ``__terminated_by(T)`` : The pointer or array is terminated by ``T`` which is375 a constant expression. Accessing or incrementing the pointer beyond the376 terminator is not allowed. This is a generalization of ``__null_terminated``377 which is defined as ``__terminated_by(0)``.378 379Annotation for interoperating with bounds-unsafe code380^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^381 382A pointer with the ``__unsafe_indexable`` annotation behaves the same as a plain383C pointer. That is, the pointer does not have any bounds information and pointer384operations are not checked.385 386``__unsafe_indexable`` can be used to mark pointers from system headers or387pointers from code that has not adopted -fbounds safety. This enables388interoperation between code using ``-fbounds-safety`` and code that does not.389 390Default pointer types391---------------------392 393ABI visibility and default annotations394^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^395 396Requiring ``-fbounds-safety`` adopters to add bounds annotations to all pointers397in the codebase would be a significant adoption burden. To avoid this and to398secure all pointers by default, ``-fbounds-safety`` applies default bounds399annotations to pointer types.400Default annotations apply to pointer types of declarations401 402``-fbounds-safety`` applies default bounds annotations to pointer types used in403declarations. The default annotations are determined by the ABI visibility of404the pointer. A pointer type is ABI-visible if changing its size or405representation affects the ABI. For instance, changing the size of a type used406in a function parameter will affect the ABI and thus pointers used in function407parameters are ABI-visible pointers. On the other hand, changing the types of408local variables won't have such ABI implications. Hence, ``-fbounds-safety``409considers the outermost pointer types of local variables as non-ABI visible. The410rest of the pointers such as nested pointer types, pointer types of global411variables, struct fields, and function prototypes are considered ABI-visible.412 413All ABI-visible pointers are treated as ``__single`` by default unless annotated414otherwise. This default both preserves ABI and makes these pointers safe by415default. This behavior can be controlled with macros, i.e.,416``__ptrcheck_abi_assume_*ATTR*()``, to set the default annotation for417ABI-visible pointers to be either ``__single``, ``__bidi_indexable``,418``__indexable``, or ``__unsafe_indexable``. For instance,419``__ptrcheck_abi_assume_unsafe_indexable()`` will make all ABI-visible pointers420be ``__unsafe_indexable``. Non-ABI visible pointers — the outermost pointer421types of local variables — are ``__bidi_indexable`` by default, so that these422pointers have the bounds information necessary to perform bounds checks without423the need for a manual annotation. All ``const char`` pointers or any typedefs424equivalent to ``const char`` pointers are ``__null_terminated`` by default. This425means that ``char8_t`` is ``unsigned char`` so ``const char8_t *`` won't be426``__null_terminated`` by default. Similarly, ``const wchar_t *`` won't be427``__null_terminated`` by default unless the platform defines it as ``typedef428char wchar_t``. Please note, however, that the programmers can still explicitly429use ``__null_terminated`` in any other pointers, e.g., ``char8_t430*__null_terminated``, ``wchar_t *__null_terminated``, ``int431*__null_terminated``, etc. if they should be treated as ``__null_terminated``.432The same applies to other annotations.433In system headers, the default pointer attribute for ABI-visible pointers is set434to ``__unsafe_indexable`` by default.435 436The ``__ptrcheck_abi_assume_*ATTR*()`` macros are defined as pragmas in the437toolchain header (See `Portability with toolchains that do not support the438extension`_ for more details about the toolchain header):439 440.. code-block:: C441 442#define __ptrcheck_abi_assume_single() \443 _Pragma("clang abi_ptr_attr set(single)")444 445#define __ptrcheck_abi_assume_indexable() \446 _Pragma("clang abi_ptr_attr set(indexable)")447 448#define __ptrcheck_abi_assume_bidi_indexable() \449 _Pragma("clang abi_ptr_attr set(bidi_indexable)")450 451#define __ptrcheck_abi_assume_unsafe_indexable() \452 _Pragma("clang abi_ptr_attr set(unsafe_indexable)")453 454 455ABI implications of default bounds annotations456^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^457 458Although simply modifying types of a local variable doesn't normally impact the459ABI, taking the address of such a modified type could create a pointer type that460has an ABI mismatch. Looking at the following example, ``int *local`` is461implicitly ``int *__bidi_indexable`` and thus the type of ``&local`` is a462pointer to ``int *__bidi_indexable``. On the other hand, in ``void foo(int463**)``, the parameter type is a pointer to ``int *__single`` (i.e., ``void464foo(int *__single *__single)``) (or a pointer to ``int *__unsafe_indexable`` if465it's from a system header). The compiler reports an error for casts between466pointers whose elements have incompatible pointer attributes. This way,467``-fbounds-safety`` prevents pointers that are implicitly ``__bidi_indexable``468from silently escaping thereby breaking the ABI.469 470.. code-block:: c471 472 void foo(int **);473 474 void bar(void) {475 int *local = 0;476 // error: passing 'int *__bidi_indexable*__bidi_indexable' to parameter of477 // incompatible nested pointer type 'int *__single*__single'478 foo(&local);479 }480 481A local variable may still be exposed to the ABI if ``typeof()`` takes the type482of local variable to define an interface as shown in the following example.483 484.. code-block:: C485 486 // bar.c487 void bar(int *) { ... }488 489 // foo.c490 void foo(void) {491 int *p; // implicitly `int *__bidi_indexable p`492 extern void bar(typeof(p)); // creates an interface of type493 // `void bar(int *__bidi_indexable)`494 }495 496Doing this may break the ABI if the parameter is not ``__bidi_indexable`` at the497definition of function ``bar()`` which is likely the case because parameters are498``__single`` by default without an explicit annotation.499 500In order to avoid an implicitly wide pointer from silently breaking the ABI, the501compiler reports a warning when ``typeof()`` is used on an implicit wide pointer502at any ABI visible context (e.g., function prototype, struct definition, etc.).503 504.. _Default pointer types in typeof:505 506Default pointer types in ``typeof()``507^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^508 509When ``typeof()`` takes an expression, it respects the bounds annotation on510the expression type, including the bounds annotation is implicit. For example,511the global variable ``g`` in the following code is implicitly ``__single`` so512``typeof(g)`` gets ``char *__single``. The similar is true for the parameter513``p``, so ``typeof(p)`` returns ``void *__single``. The local variable ``l`` is514implicitly ``__bidi_indexable``, so ``typeof(l)`` becomes515``int *__bidi_indexable``.516 517.. code-block:: C518 519 char *g; // typeof(g) == char *__single520 521 void foo(void *p) {522 // typeof(p) == void *__single523 524 int *l; // typeof(l) == int *__bidi_indexable525 }526 527When the type of expression has an "external" bounds annotation, e.g.,528``__sized_by``, ``__counted_by``, etc., the compiler may report an error on529``typeof`` if the annotation creates a dependency with another declaration or530variable. For example, the compiler reports an error on ``typeof(p1)`` shown in531the following code because allowing it can potentially create another type532dependent on the parameter ``size`` in a different context (Please note that an533external bounds annotation on a parameter may only refer to another parameter of534the same function). On the other hand, ``typeof(p2)`` works resulting in ``int535*__counted_by(10)``, since it doesn't depend on any other declaration.536 537.. TODO: add a section describing constraints on external bounds annotations538 539.. code-block:: C540 541 void foo(int *__counted_by(size) p1, size_t size) {542 // typeof(p1) == int *__counted_by(size)543 // -> a compiler error as it tries to create another type544 // dependent on `size`.545 546 int *__counted_by(10) p2; // typeof(p2) == int *__counted_by(10)547 // -> no error548 549 }550 551When ``typeof()`` takes a type name, the compiler doesn't apply an implicit552bounds annotation on the named pointer types. For example, ``typeof(int*)``553returns ``int *`` without any bounds annotation. A bounds annotation may be554added after the fact depending on the context. In the following example,555``typeof(int *)`` returns ``int *`` so it's equivalent as the local variable is556declared as ``int *l``, so it eventually becomes implicitly557``__bidi_indexable``.558 559.. code-block:: c560 561 void foo(void) {562 typeof(int *) l; // `int *__bidi_indexable` (same as `int *l`)563 }564 565The programmers can still explicitly add a bounds annotation on the types named566inside ``typeof``, e.g., ``typeof(int *__bidi_indexable)``, which evaluates to567``int *__bidi_indexable``.568 569 570Default pointer types in ``sizeof()``571^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^572 573When ``sizeof()`` takes a type name, the compiler doesn't apply an implicit574bounds annotation on the named pointer types. This means if a bounds annotation575is not specified, the evaluated pointer type is treated identically to a plain C576pointer type. Therefore, ``sizeof(int*)`` remains the same with or without577``-fbounds-safety``. That said, programmers can explicitly add attributes to the578types, e.g., ``sizeof(int *__bidi_indexable)``, in which case the sizeof579evaluates to the size of type ``int *__bidi_indexable`` (the value equivalent to580``3 * sizeof(int*)``).581 582When ``sizeof()`` takes an expression, i.e., ``sizeof(expr``, it behaves as583``sizeof(typeof(expr))``, except that ``sizeof(expr)`` does not report an error584with ``expr`` that has a type with an external bounds annotation dependent on585another declaration, whereas ``typeof()`` on the same expression would be an586error as described in :ref:`Default pointer types in typeof`.587The following example describes this behavior.588 589.. code-block:: c590 591 void foo(int *__counted_by(size) p, size_t size) {592 // sizeof(p) == sizeof(int *__counted_by(size)) == sizeof(int *)593 // typeof(p): error594 };595 596Default pointer types in ``alignof()``597^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^598 599``alignof()`` only takes a type name as the argument and it doesn't take an600expression. Similar to ``sizeof()`` and ``typeof``, the compiler doesn't apply601an implicit bounds annotation on the pointer types named inside ``alignof()``.602Therefore, ``alignof(T *)`` remains the same with or without603``-fbounds-safety``, evaluating into the alignment of the raw pointer ``T *``.604The programmers can explicitly add a bounds annotation to the types, e.g.,605``alignof(int *__bidi_indexable)``, which returns the alignment of ``int606*__bidi_indexable``. A bounds annotation including an internal bounds annotation607(i.e., ``__indexable`` and ``__bidi_indexable``) doesn't affect the alignment of608the original pointer. Therefore, ``alignof(int *__bidi_indexable)`` is equal to609``alignof(int *)``.610 611 612Default pointer types used in C-style casts613^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^614 615A pointer type used in a C-style cast (e.g., ``(int *)src``) inherits the same616pointer attribute in the type of src. For instance, if the type of src is ``T617*__single`` (with ``T`` being an arbitrary C type), ``(int *)src`` will be ``int618*__single``. The reasoning behind this behavior is so that a C-style cast619doesn't introduce any unexpected side effects caused by an implicit cast of620bounds attribute.621 622Pointer casts can have explicit bounds annotations. For instance, ``(int623*__bidi_indexable)src`` casts to ``int *__bidi_indexable`` as long as src has a624bounds annotation that can implicitly convert to ``__bidi_indexable``. If625``src`` has type ``int *__single``, it can implicitly convert to ``int626*__bidi_indexable`` which then will have the upper bound pointing to one past627the first element. However, if src has type ``int *__unsafe_indexable``, the628explicit cast ``(int *__bidi_indexable)src`` will cause an error because629``__unsafe_indexable`` cannot cast to ``__bidi_indexable`` as630``__unsafe_indexable`` doesn't have bounds information. `Cast rules`_ describes631in more detail what kinds of casts are allowed between pointers with different632bounds annotations.633 634Default pointer types in typedef635^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^636 637Pointer types in ``typedef``\s do not have implicit default bounds annotations.638Instead, the bounds annotation is determined when the ``typedef`` is used. The639following example shows that no pointer annotation is specified in the ``typedef640pint_t`` while each instance of ``typedef``'ed pointer gets its bounds641annotation based on the context in which the type is used.642 643.. code-block:: c644 645 typedef int * pint_t; // int *646 647 pint_t glob; // int *__single glob;648 649 void foo(void) {650 pint_t local; // int *__bidi_indexable local;651 }652 653Pointer types in a ``typedef`` can still have explicit annotations, e.g.,654``typedef int *__single``, in which case the bounds annotation ``__single`` will655apply to every use of the ``typedef``.656 657Array to pointer promotion to secure arrays (including VLAs)658------------------------------------------------------------659 660Arrays on function prototypes661^^^^^^^^^^^^^^^^^^^^^^^^^^^^^662 663In C, arrays on function prototypes are promoted (or "decayed") to a pointer to664its first element (e.g., ``&arr[0]``). In ``-fbounds-safety``, arrays are also665decayed to pointers, but with the addition of an implicit bounds annotation,666which includes variable-length arrays (VLAs). As shown in the following example,667arrays on function prototypes are decayed to corresponding ``__counted_by``668pointers.669 670.. code-block:: c671 672 // Function prototype: void foo(int n, int *__counted_by(n) arr);673 void foo(int n, int arr[n]);674 675 // Function prototype: void bar(int *__counted_by(10) arr);676 void bar(int arr[10]);677 678This means the array parameters are treated as `__counted_by` pointers within679the function and callers of the function also see them as the corresponding680`__counted_by` pointers.681 682Incomplete arrays on function prototypes will cause a compiler error unless it683has ``__counted_by`` annotation in its bracket.684 685.. code-block:: c686 687 void f1(int n, int arr[]); // error688 689 void f3(int n, int arr[__counted_by(n)]); // ok690 691 void f2(int n, int arr[n]); // ok, decays to int *__counted_by(n)692 693 void f4(int n, int *__counted_by(n) arr); // ok694 695 void f5(int n, int *arr); // ok, but decays to int *__single,696 // and cannot be used for pointer arithmetic697 698Array references699^^^^^^^^^^^^^^^^700 701In C, similar to arrays on the function prototypes, a reference to array is702automatically promoted (or "decayed") to a pointer to its first element (e.g.,703``&arr[0]``).704 705In `-fbounds-safety`, array references are promoted to ``__bidi_indexable``706pointers which contain the upper and lower bounds of the array, with the707equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]``708(or one past the last element) serving as the upper bound. This applies to all709types of arrays including constant-length arrays, variable-length arrays (VLAs),710and flexible array members annotated with `__counted_by`.711 712In the following example, reference to ``vla`` promotes to ``int713*__bidi_indexable``, with ``&vla[n]`` as the upper bound and ``&vla[0]`` as the714lower bound. Then, it's copied to ``int *p``, which is implicitly ``int715*__bidi_indexable p``. Please note that value of ``n`` used to create the upper716bound is ``10``, not ``100``, in this case because ``10`` is the actual length717of ``vla``, the value of ``n`` at the time when the array is being allocated.718 719.. code-block:: c720 721 void foo(void) {722 int n = 10;723 int vla[n];724 n = 100;725 int *p = vla; // { .ptr: &vla[0], .upper: &vla[10], .lower: &vla[0] }726 // it's `&vla[10]` because the value of `n` was 10 at the727 // time when the array is actually allocated.728 // ...729 }730 731By promoting array references to ``__bidi_indexable``, all array accesses are732bounds checked in ``-fbounds-safety``, just as ``__bidi_indexable`` pointers733are.734 735Maintaining correctness of bounds annotations736---------------------------------------------737 738``-fbounds-safety`` maintains correctness of bounds annotations by performing739additional checks when a pointer object and/or its related value containing the740bounds information is updated.741 742For example, ``__single`` expresses an invariant that the pointer must either743point to a single valid object or be a null pointer. To maintain this invariant,744the compiler inserts checks when initializing a ``__single`` pointer, as shown745in the following example:746 747.. code-block:: c748 749 void foo(void *__sized_by(size) vp, size_t size) {750 // Inserted check:751 // if ((int*)upper_bound(vp) - (int*)vp < sizeof(int) && !!vp) trap();752 int *__single ip = (int *)vp;753 }754 755Additionally, an explicit bounds annotation such as ``int *__counted_by(count)756buf`` defines a relationship between two variables, ``buf`` and ``count``:757namely, that ``buf`` has ``count`` number of elements available. This758relationship must hold even after any of these related variables are updated. To759this end, the model requires that assignments to ``buf`` and ``count`` must be760side by side, with no side effects between them. This prevents ``buf`` and761``count`` from temporarily falling out of sync due to updates happening at a762distance.763 764The example below shows a function ``alloc_buf`` that initializes a struct that765members that use the ``__counted_by`` annotation. The compiler allows these766assignments because ``sbuf->buf`` and ``sbuf->count`` are updated side by side767without any side effects in between the assignments.768 769Furthermore, the compiler inserts additional run-time checks to ensure the new770``buf`` has at least as many elements as the new ``count`` indicates as shown in771the transformed pseudo code of function ``alloc_buf()`` in the example below.772 773.. code-block:: c774 775 typedef struct {776 int *__counted_by(count) buf;777 size_t count;778 } sized_buf_t;779 780 void alloc_buf(sized_buf_t *sbuf, size_t nelems) {781 sbuf->buf = (int *)malloc(sizeof(int) * nelems);782 sbuf->count = nelems;783 }784 785 // Transformed pseudo code:786 void alloc_buf(sized_buf_t *sbuf, size_t nelems) {787 // Materialize RHS values:788 int *tmp_ptr = (int *)malloc(sizeof(int) * nelems);789 int tmp_count = nelems;790 // Inserted check:791 // - checks to ensure that `lower <= tmp_ptr <= upper`792 // - if (upper(tmp_ptr) - tmp_ptr < tmp_count) trap();793 sbuf->buf = tmp_ptr;794 sbuf->count = tmp_count;795 }796 797Whether the compiler can optimize such run-time checks depends on how the upper798bound of the pointer is derived. If the source pointer has ``__sized_by``,799``__counted_by``, or a variant of such, the compiler assumes that the upper800bound calculation doesn't overflow, e.g., ``ptr + size`` (where the type of801``ptr`` is ``void *__sized_by(size)``), because when the ``__sized_by`` pointer802is initialized, ``-fbounds-safety`` inserts run-time checks to ensure that ``ptr803+ size`` doesn't overflow and that ``size >= 0``.804 805Assuming the upper bound calculation doesn't overflow, the compiler can simplify806the trap condition ``upper(tmp_ptr) - tmp_ptr < tmp_count`` to ``size <807tmp_count`` so if both ``size`` and ``tmp_count`` values are known at compile808time such that ``0 <= tmp_count <= size``, the optimizer can remove the check.809 810``ptr + size`` may still overflow if the ``__sized_by`` pointer is created from811code that doesn't enable ``-fbounds-safety``, which is undefined behavior.812 813In the previous code example with the transformed ``alloc_buf()``, the upper814bound of ``tmp_ptr`` is derived from ``void *__sized_by_or_null(size)``, which815is the return type of ``malloc()``. Hence, the pointer arithmetic doesn't816overflow or ``tmp_ptr`` is null. Therefore, if ``nelems`` was given as a817compile-time constant, the compiler could remove the checks.818 819Cast rules820----------821 822``-fbounds-safety`` does not enforce overall type safety and bounds invariants823can still be violated by incorrect casts in some cases. That said,824``-fbounds-safety`` prevents type conversions that change bounds attributes in a825way to violate the bounds invariant of the destination's pointer annotation.826Type conversions that change bounds attributes may be allowed if it does not827violate the invariant of the destination or that can be verified at run time.828Here are some of the important cast rules.829 830Two pointers that have different bounds annotations on their nested pointer831types are incompatible and cannot implicitly cast to each other. For example,832``T *__single *__single`` cannot be converted to ``T *__bidi_indexable833*__single``. Such a conversion between incompatible nested bounds annotations834can be allowed using an explicit cast (e.g., C-style cast). Hereafter, the rules835only apply to the top pointer types. ``__unsafe_indexable`` cannot be converted836to any other safe pointer types (``__single``, ``__bidi_indexable``,837``__counted_by``, etc) using a cast. The extension provides builtins to force838this conversion, ``__unsafe_forge_bidi_indexable(type, pointer, char_count)`` to839convert pointer to a ``__bidi_indexable`` pointer of type with ``char_count``840bytes available and ``__unsafe_forge_single(type, pointer)`` to convert pointer841to a single pointer of type type. The following examples show the usage of these842functions. Function ``example_forge_bidi()`` gets an external buffer from an843unsafe library by calling ``get_buf()`` which returns ``void844*__unsafe_indexable.`` Under the type rules, this cannot be directly assigned to845``void *buf`` (implicitly ``void *__bidi_indexable``). Thus,846``__unsafe_forge_bidi_indexable`` is used to manually create a847``__bidi_indexable`` from the unsafe buffer.848 849.. code-block:: c850 851 // unsafe_library.h852 void *__unsafe_indexable get_buf(void);853 size_t get_buf_size(void);854 855 // my_source1.c (enables -fbounds-safety)856 #include "unsafe_library.h"857 void example_forge_bidi(void) {858 void *buf =859 __unsafe_forge_bidi_indexable(void *, get_buf(), get_buf_size());860 // ...861 }862 863 // my_source2.c (enables -fbounds-safety)864 #include <stdio.h>865 void example_forge_single(void) {866 FILE *fp = __unsafe_forge_single(FILE *, fopen("mypath", "rb"));867 // ...868 }869 870* Function ``example_forge_single`` takes a file handle by calling fopen defined871 in system header ``stdio.h``. Assuming ``stdio.h`` did not adopt872 ``-fbounds-safety``, the return type of ``fopen`` would implicitly be ``FILE873 *__unsafe_indexable`` and thus it cannot be directly assigned to ``FILE *fp``874 in the bounds-safe source. To allow this operation, ``__unsafe_forge_single``875 is used to create a ``__single`` from the return value of ``fopen``.876 877* Similar to ``__unsafe_indexable``, any non-pointer type (including ``int``,878 ``intptr_t``, ``uintptr_t``, etc.) cannot be converted to any safe pointer879 type because these don't have bounds information. ``__unsafe_forge_single`` or880 ``__unsafe_forge_bidi_indexable`` must be used to force the conversion.881 882* Any safe pointer types can cast to ``__unsafe_indexable`` because it doesn't883 have any invariant to maintain.884 885* ``__single`` casts to ``__bidi_indexable`` if the pointee type has a known886 size. After the conversion, the resulting ``__bidi_indexable`` has the size of887 a single object of the pointee type of ``__single``. ``__single`` cannot cast888 to ``__bidi_indexable`` if the pointee type is incomplete or sizeless. For889 example, ``void *__single`` cannot convert to ``void *__bidi_indexable``890 because void is an incomplete type and thus the compiler cannot correctly891 determine the upper bound of a single void pointer.892 893* Similarly, ``__single`` can cast to ``__indexable`` if the pointee type has a894 known size. The resulting ``__indexable`` has the size of a single object of895 the pointee type.896 897* ``__single`` casts to ``__counted_by(E)`` only if ``E`` is 0 or 1.898 899* ``__single`` can cast to ``__single`` including when they have different900 pointee types as long as it is allowed in the underlying C standard.901 ``-fbounds-safety`` doesn't guarantee type safety.902 903* ``__bidi_indexable`` and ``__indexable`` can cast to ``__single``. The904 compiler may insert run-time checks to ensure the pointer has at least a905 single element or is a null pointer.906 907* ``__bidi_indexable`` casts to ``__indexable`` if the pointer does not have an908 underflow. The compiler may insert run-time checks to ensure the pointer is909 not below the lower bound.910 911* ``__indexable`` casts to ``__bidi_indexable``. The resulting912 ``__bidi_indexable`` gets the lower bound same as the pointer value.913 914* A type conversion may involve both a bitcast and a bounds annotation cast. For915 example, casting from ``int *__bidi_indexable`` to ``char *__single`` involves916 a bitcast (``int *`` to ``char *``) and a bounds annotation cast917 (``__bidi_indexable`` to ``__single``). In this case, the compiler performs918 the bitcast and then converts the bounds annotation. This means, ``int919 *__bidi_indexable`` will be converted to ``char *__bidi_indexable`` and then920 to ``char *__single``.921 922* ``__terminated_by(T)`` cannot cast to any safe pointer type without the same923 ``__terminated_by(T)`` attribute. To perform the cast, programmers can use an924 intrinsic function such as ``__unsafe_terminated_by_to_indexable(P)`` to force925 the conversion.926 927* ``__terminated_by(T)`` can cast to ``__unsafe_indexable``.928 929* Any type without ``__terminated_by(T)`` cannot cast to ``__terminated_by(T)``930 without explicitly using an intrinsic function to allow it.931 932 + ``__unsafe_terminated_by_from_indexable(T, PTR [, PTR_TO_TERM])`` casts any933 safe pointer PTR to a ``__terminated_by(T)`` pointer. ``PTR_TO_TERM`` is an934 optional argument where the programmer can provide the exact location of the935 terminator. With this argument, the function can skip reading the entire936 array in order to locate the end of the pointer (or the upper bound).937 Providing an incorrect ``PTR_TO_TERM`` causes a run-time trap.938 939 + ``__unsafe_forge_terminated_by(T, P, E)`` creates ``T __terminated_by(E)``940 pointer given any pointer ``P``. Tmust be a pointer type.941 942Portability with toolchains that do not support the extension943-------------------------------------------------------------944 945The language model is designed so that it doesn't alter the semantics of the946original C program, other than introducing deterministic traps where otherwise947the behavior is undefined and/or unsafe. Clang provides a toolchain header948(``ptrcheck.h``) that macro-defines the annotations as type attributes when949``-fbounds-safety`` is enabled and defines them to empty when the extension is950disabled. Thus, the code adopting ``-fbounds-safety`` can compile with951toolchains that do not support this extension, by including the header or adding952macros to define the annotations to empty. For example, the toolchain not953supporting this extension may not have a header defining ``__counted_by``, so954the code using ``__counted_by`` must define it as nothing or include a header955that has the define.956 957.. code-block:: c958 959 #if defined(__has_feature) && __has_feature(bounds_safety)960 #define __counted_by(T) __attribute__((__counted_by__(T)))961 // ... other bounds annotations962 #else963 #define __counted_by(T) // defined as nothing964 // ... other bounds annotations965 #endif966 967 // expands to `void foo(int * ptr, size_t count);`968 // when extension is not enabled or not available969 void foo(int *__counted_by(count) ptr, size_t count);970 971Other potential applications of bounds annotations972==================================================973 974The bounds annotations provided by the ``-fbounds-safety`` programming model975have potential use cases beyond the language extension itself. For example,976static and dynamic analysis tools could use the bounds information to improve977diagnostics for out-of-bounds accesses, even if ``-fbounds-safety`` is not used.978The bounds annotations could be used to improve C interoperability with979bounds-safe languages, providing a better mapping to bounds-safe types in the980safe language interface. The bounds annotations can also serve as documentation981specifying the relationship between declarations.982 983Limitations984===========985 986``-fbounds-safety`` aims to bring the bounds safety guarantee to the C language,987and it does not guarantee other types of memory safety properties. Consequently,988it may not prevent some of the secondary bounds safety violations caused by989other types of safety violations such as type confusion. For instance,990``-fbounds-safety`` does not perform type-safety checks on conversions between991``__single`` pointers of different pointee types (e.g., ``char *__single`` →992``void *__single`` → ``int *__single``) beyond what the foundation languages993(C/C++) already offer.994 995``-fbounds-safety`` heavily relies on run-time checks to keep the bounds safety996and the soundness of the type system. This may incur significant code size997overhead in unoptimized builds and leave some of the adoption mistakes to be998caught only at run time. This is not a fundamental limitation, however, because999incrementally adding necessary static analysis will allow us to catch issues1000early on and remove unnecessary bounds checks in unoptimized builds.1001 1002Try it out1003==========1004 1005Your feedback on the programming model is valuable. You may want to follow the1006instruction in :doc:`BoundsSafetyAdoptionGuide` to play with ``-fbounds-safety``1007and please send your feedback to `Yeoul Na <mailto:yeoul_na@apple.com>`_.1008