183 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3====================4Kernel Testing Guide5====================6 7 8There are a number of different tools for testing the Linux kernel, so knowing9when to use each of them can be a challenge. This document provides a rough10overview of their differences, and how they fit together.11 12 13Writing and Running Tests14=========================15 16The bulk of kernel tests are written using either the kselftest or KUnit17frameworks. These both provide infrastructure to help make running tests and18groups of tests easier, as well as providing helpers to aid in writing new19tests.20 21If you're looking to verify the behaviour of the Kernel — particularly specific22parts of the kernel — then you'll want to use KUnit or kselftest.23 24 25The Difference Between KUnit and kselftest26------------------------------------------27 28KUnit (Documentation/dev-tools/kunit/index.rst) is an entirely in-kernel system29for "white box" testing: because test code is part of the kernel, it can access30internal structures and functions which aren't exposed to userspace.31 32KUnit tests therefore are best written against small, self-contained parts33of the kernel, which can be tested in isolation. This aligns well with the34concept of 'unit' testing.35 36For example, a KUnit test might test an individual kernel function (or even a37single codepath through a function, such as an error handling case), rather38than a feature as a whole.39 40This also makes KUnit tests very fast to build and run, allowing them to be41run frequently as part of the development process.42 43There is a KUnit test style guide which may give further pointers in44Documentation/dev-tools/kunit/style.rst45 46 47kselftest (Documentation/dev-tools/kselftest.rst), on the other hand, is48largely implemented in userspace, and tests are normal userspace scripts or49programs.50 51This makes it easier to write more complicated tests, or tests which need to52manipulate the overall system state more (e.g., spawning processes, etc.).53However, it's not possible to call kernel functions directly from kselftest.54This means that only kernel functionality which is exposed to userspace somehow55(e.g. by a syscall, device, filesystem, etc.) can be tested with kselftest. To56work around this, some tests include a companion kernel module which exposes57more information or functionality. If a test runs mostly or entirely within the58kernel, however, KUnit may be the more appropriate tool.59 60kselftest is therefore suited well to tests of whole features, as these will61expose an interface to userspace, which can be tested, but not implementation62details. This aligns well with 'system' or 'end-to-end' testing.63 64For example, all new system calls should be accompanied by kselftest tests.65 66Code Coverage Tools67===================68 69The Linux Kernel supports two different code coverage measurement tools. These70can be used to verify that a test is executing particular functions or lines71of code. This is useful for determining how much of the kernel is being tested,72and for finding corner-cases which are not covered by the appropriate test.73 74Documentation/dev-tools/gcov.rst is GCC's coverage testing tool, which can be75used with the kernel to get global or per-module coverage. Unlike KCOV, it76does not record per-task coverage. Coverage data can be read from debugfs,77and interpreted using the usual gcov tooling.78 79Documentation/dev-tools/kcov.rst is a feature which can be built in to the80kernel to allow capturing coverage on a per-task level. It's therefore useful81for fuzzing and other situations where information about code executed during,82for example, a single syscall is useful.83 84 85Dynamic Analysis Tools86======================87 88The kernel also supports a number of dynamic analysis tools, which attempt to89detect classes of issues when they occur in a running kernel. These typically90each look for a different class of bugs, such as invalid memory accesses,91concurrency issues such as data races, or other undefined behaviour like92integer overflows.93 94Some of these tools are listed below:95 96* kmemleak detects possible memory leaks. See97 Documentation/dev-tools/kmemleak.rst98* KASAN detects invalid memory accesses such as out-of-bounds and99 use-after-free errors. See Documentation/dev-tools/kasan.rst100* UBSAN detects behaviour that is undefined by the C standard, like integer101 overflows. See Documentation/dev-tools/ubsan.rst102* KCSAN detects data races. See Documentation/dev-tools/kcsan.rst103* KFENCE is a low-overhead detector of memory issues, which is much faster than104 KASAN and can be used in production. See Documentation/dev-tools/kfence.rst105* lockdep is a locking correctness validator. See106 Documentation/locking/lockdep-design.rst107* Runtime Verification (RV) supports checking specific behaviours for a given108 subsystem. See Documentation/trace/rv/runtime-verification.rst109* There are several other pieces of debug instrumentation in the kernel, many110 of which can be found in lib/Kconfig.debug111 112These tools tend to test the kernel as a whole, and do not "pass" like113kselftest or KUnit tests. They can be combined with KUnit or kselftest by114running tests on a kernel with these tools enabled: you can then be sure115that none of these errors are occurring during the test.116 117Some of these tools integrate with KUnit or kselftest and will118automatically fail tests if an issue is detected.119 120Static Analysis Tools121=====================122 123In addition to testing a running kernel, one can also analyze kernel source code124directly (**at compile time**) using **static analysis** tools. The tools125commonly used in the kernel allow one to inspect the whole source tree or just126specific files within it. They make it easier to detect and fix problems during127the development process.128 129Sparse can help test the kernel by performing type-checking, lock checking,130value range checking, in addition to reporting various errors and warnings while131examining the code. See the Documentation/dev-tools/sparse.rst documentation132page for details on how to use it.133 134Smatch extends Sparse and provides additional checks for programming logic135mistakes such as missing breaks in switch statements, unused return values on136error checking, forgetting to set an error code in the return of an error path,137etc. Smatch also has tests against more serious issues such as integer138overflows, null pointer dereferences, and memory leaks. See the project page at139http://smatch.sourceforge.net/.140 141Coccinelle is another static analyzer at our disposal. Coccinelle is often used142to aid refactoring and collateral evolution of source code, but it can also help143to avoid certain bugs that occur in common code patterns. The types of tests144available include API tests, tests for correct usage of kernel iterators, checks145for the soundness of free operations, analysis of locking behavior, and further146tests known to help keep consistent kernel usage. See the147Documentation/dev-tools/coccinelle.rst documentation page for details.148 149Beware, though, that static analysis tools suffer from **false positives**.150Errors and warns need to be evaluated carefully before attempting to fix them.151 152When to use Sparse and Smatch153-----------------------------154 155Sparse does type checking, such as verifying that annotated variables do not156cause endianness bugs, detecting places that use ``__user`` pointers improperly,157and analyzing the compatibility of symbol initializers.158 159Smatch does flow analysis and, if allowed to build the function database, it160also does cross function analysis. Smatch tries to answer questions like where161is this buffer allocated? How big is it? Can this index be controlled by the162user? Is this variable larger than that variable?163 164It's generally easier to write checks in Smatch than it is to write checks in165Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks.166 167Strong points of Smatch and Coccinelle168--------------------------------------169 170Coccinelle is probably the easiest for writing checks. It works before the171pre-processor so it's easier to check for bugs in macros using Coccinelle.172Coccinelle also creates patches for you, which no other tool does.173 174For example, with Coccinelle you can do a mass conversion from175``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and176that's really useful. If you just created a Smatch warning and try to push the177work of converting on to the maintainers they would be annoyed. You'd have to178argue about each warning if can really overflow or not.179 180Coccinelle does no analysis of variable values, which is the strong point of181Smatch. On the other hand, Coccinelle allows you to do simple things in a simple182way.183