831 lines · html
1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"2 "http://www.w3.org/TR/html4/strict.dtd">3<html>4<head>5 <title>Checker Developer Manual</title>6 <link type="text/css" rel="stylesheet" href="menu.css">7 <link type="text/css" rel="stylesheet" href="content.css">8 <script type="text/javascript" src="scripts/menu.js"></script>9</head>10<body>11 12<div id="page">13<!--#include virtual="menu.html.incl"-->14 15<div id="content">16 17<h3 style="color:red">This Page Is Under Construction</h3>18 19<h1>Checker Developer Manual</h1>20 21<p>The static analyzer engine performs path-sensitive exploration of the program and22relies on a set of checkers to implement the logic for detecting and23constructing specific bug reports. Anyone who is interested in implementing their own24checker, should check out the Building a Checker in 24 Hours talk25(<a href="https://llvm.org/devmtg/2012-11/Zaks-Rose-Checker24Hours.pdf">slides</a>26 <a href="https://youtu.be/kdxlsP5QVPw">video</a>)27and refer to this page for additional information on writing a checker. The static analyzer is a28part of the Clang project, so consult <a href="https://clang.llvm.org/hacking.html">Hacking on Clang</a>29and <a href="https://llvm.org/docs/ProgrammersManual.html">LLVM Programmer's Manual</a>30for developer guidelines and post your questions and proposals to the31<a href="https://discourse.llvm.org/c/clang/static-analyzer/"> Static Analyzer</a> subcategory at32the official <a href="https://discourse.llvm.org/"> LLVM Discourse server</a>.33</p>34 35 <ul>36 <li><a href="#start">Getting Started</a></li>37 <li><a href="#analyzer">Static Analyzer Overview</a>38 <ul>39 <li><a href="#interaction">Interaction with Checkers</a></li>40 <li><a href="#values">Representing Values</a></li>41 </ul></li>42 <li><a href="#idea">Idea for a Checker</a></li>43 <li><a href="#registration">Checker Registration</a></li>44 <li><a href="#events_callbacks">Events, Callbacks, and Checker Class Structure</a></li>45 <li><a href="#extendingstates">Custom Program States</a></li>46 <li><a href="#bugs">Bug Reports</a></li>47 <li><a href="#ast">AST Visitors</a></li>48 <li><a href="#testing">Testing</a></li>49 <li><a href="#commands">Useful Commands/Debugging Hints</a>50 <ul>51 <li><a href="#attaching">Attaching the Debugger</a></li>52 <li><a href="#narrowing">Narrowing Down the Problem</a></li>53 <li><a href="#visualizing">Visualizing the Analysis</a></li>54 <li><a href="#debugprints">Debug Prints and Tricks</a></li>55 </ul></li>56 <li><a href="#additioninformation">Additional Sources of Information</a></li>57 <li><a href="#links">Useful Links</a></li>58 </ul>59 60<h2 id=start>Getting Started</h2>61 <ul>62 <li>To check out the source code and build the project, follow steps 1-4 of63 the <a href="https://clang.llvm.org/get_started.html">Clang Getting Started</a>64 page.</li>65 66 <li>The analyzer source code is located under the Clang source tree:67 <br><tt>68 $ <b>cd llvm/tools/clang</b>69 </tt>70 <br>See: <tt>include/clang/StaticAnalyzer</tt>, <tt>lib/StaticAnalyzer</tt>,71 <tt>test/Analysis</tt>.</li>72 73 <li>The analyzer regression tests can be executed from the Clang's build74 directory:75 <br><tt>76 $ <b>cd ../../../; cd build/tools/clang; TESTDIRS=Analysis make test</b>77 </tt></li>78 79 <li>Analyze a file with the specified checker:80 <br><tt>81 $ <b>clang -cc1 -analyze -analyzer-checker=core.DivideZero test.c</b>82 </tt></li>83 84 <li>List the available checkers:85 <br><tt>86 $ <b>clang -cc1 -analyzer-checker-help</b>87 </tt></li>88 89 <li>See the analyzer help for different output formats, fine tuning, and90 debug options:91 <br><tt>92 $ <b>clang -cc1 -help | grep "analyzer"</b>93 </tt></li>94 95 </ul>96 97<h2 id=analyzer>Static Analyzer Overview</h2>98 The analyzer core performs symbolic execution of the given program. All the99 input values are represented with symbolic values; further, the engine deduces100 the values of all the expressions in the program based on the input symbols101 and the path. The execution is path sensitive and every possible path through102 the program is explored. The explored execution traces are represented with103 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1ExplodedGraph.html">ExplodedGraph</a> object.104 Each node of the graph is105 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1ExplodedNode.html">ExplodedNode</a>,106 which consists of a <tt>ProgramPoint</tt> and a <tt>ProgramState</tt>.107 <p>108 <a href="https://clang.llvm.org/doxygen/classclang_1_1ProgramPoint.html">ProgramPoint</a>109 represents the corresponding location in the program (or the CFG).110 <tt>ProgramPoint</tt> is also used to record additional information on111 when/how the state was added. For example, <tt>PostPurgeDeadSymbolsKind</tt>112 kind means that the state is the result of purging dead symbols - the113 analyzer's equivalent of garbage collection.114 <p>115 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1ProgramState.html">ProgramState</a>116 represents abstract state of the program. It consists of:117 <ul>118 <li><tt>Environment</tt> - a mapping from source code expressions to symbolic119 values120 <li><tt>Store</tt> - a mapping from memory locations to symbolic values121 <li><tt>GenericDataMap</tt> - constraints on symbolic values122 </ul>123 124 <h3 id=interaction>Interaction with Checkers</h3>125 126 <p>127 Checkers are not merely passive receivers of the analyzer core changes - they128 actively participate in the <tt>ProgramState</tt> construction through the129 <tt>GenericDataMap</tt> which can be used to store the checker-defined part130 of the state. Each time the analyzer engine explores a new statement, it131 notifies each checker registered to listen for that statement, giving it an132 opportunity to either report a bug or modify the state. (As a rule of thumb,133 the checker itself should be stateless.) The checkers are called one after another134 in the predefined order; thus, calling all the checkers adds a chain to the135 <tt>ExplodedGraph</tt>.136 </p>137 138 <h3 id=values>Representing Values</h3>139 140 <p>141 During symbolic execution, <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1SVal.html">SVal</a>142 objects are used to represent the semantic evaluation of expressions.143 They can represent things like concrete144 integers, symbolic values, or memory locations (which are memory regions).145 They are a discriminated union of "values", symbolic and otherwise.146 If a value isn't symbolic, usually that means there is no symbolic147 information to track. For example, if the value was an integer, such as148 <tt>42</tt>, it would be a <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1nonloc_1_1ConcreteInt.html">ConcreteInt</a>,149 and the checker doesn't usually need to track any state with the concrete150 number. In some cases, <tt>SVal</tt> is not a symbol, but it really should be151 a symbolic value. This happens when the analyzer cannot reason about something152 (yet). An example is floating point numbers. In such cases, the153 <tt>SVal</tt> will evaluate to <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1UnknownVal.html">UnknownVal</a>.154 This represents a case that is outside the realm of the analyzer's reasoning155 capabilities. <tt>SVals</tt> are value objects and their values can be viewed156 using the <tt>.dump()</tt> method. Often they wrap persistent objects such as157 symbols or regions.158 </p>159 160 <p>161 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1SymExpr.html">SymExpr</a> (symbol)162 is meant to represent abstract, but named, symbolic value. Symbols represent163 an actual (immutable) value. We might not know what its specific value is, but164 we can associate constraints with that value as we analyze a path. For165 example, we might record that the value of a symbol is greater than166 <tt>0</tt>, etc.167 </p>168 169 <p>170 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1MemRegion.html">MemRegion</a> is similar to a symbol.171 It is used to provide a lexicon of how to describe abstract memory. Regions can172 layer on top of other regions, providing a layered approach to representing memory.173 For example, a struct object on the stack might be represented by a <tt>VarRegion</tt>,174 but a <tt>FieldRegion</tt> which is a subregion of the <tt>VarRegion</tt> could175 be used to represent the memory associated with a specific field of that object.176 So how do we represent symbolic memory regions? That's what177 <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1SymbolicRegion.html">SymbolicRegion</a>178 is for. It is a <tt>MemRegion</tt> that has an associated symbol. Since the179 symbol is unique and has a unique name; that symbol names the region.180 </p>181 182 <p>183 Let's see how the analyzer processes the expressions in the following example:184 </p>185 186 <p>187 <pre class="code_example">188 int foo(int x) {189 int y = x * 2;190 int z = x;191 ...192 }193 </pre>194 </p>195 196 <p>197Let's look at how <tt>x*2</tt> gets evaluated. When <tt>x</tt> is evaluated,198we first construct an <tt>SVal</tt> that represents the lvalue of <tt>x</tt>, in199this case it is an <tt>SVal</tt> that references the <tt>MemRegion</tt> for <tt>x</tt>.200Afterwards, when we do the lvalue-to-rvalue conversion, we get a new <tt>SVal</tt>,201which references the value <b>currently bound</b> to <tt>x</tt>. That value is202symbolic; it's whatever <tt>x</tt> was bound to at the start of the function.203Let's call that symbol <tt>$0</tt>. Similarly, we evaluate the expression for <tt>2</tt>,204and get an <tt>SVal</tt> that references the concrete number <tt>2</tt>. When205we evaluate <tt>x*2</tt>, we take the two <tt>SVals</tt> of the subexpressions,206and create a new <tt>SVal</tt> that represents their multiplication (which in207this case is a new symbolic expression, which we might call <tt>$1</tt>). When we208evaluate the assignment to <tt>y</tt>, we again compute its lvalue (a <tt>MemRegion</tt>),209and then bind the <tt>SVal</tt> for the RHS (which references the symbolic value <tt>$1</tt>)210to the <tt>MemRegion</tt> in the symbolic store.211<br>212The second line is similar. When we evaluate <tt>x</tt> again, we do the same213dance, and create an <tt>SVal</tt> that references the symbol <tt>$0</tt>. Note, two <tt>SVals</tt>214might reference the same underlying values.215 </p>216 217<p>218To summarize, MemRegions are unique names for blocks of memory. Symbols are219unique names for abstract symbolic values. Some MemRegions represents abstract220symbolic chunks of memory, and thus are also based on symbols. SVals are just221references to values, and can reference either MemRegions, Symbols, or concrete222values (e.g., the number 1).223</p>224 225 <!--226 TODO: Add a picture.227 <br>228 Symbols<br>229 FunctionalObjects are used throughout.230 -->231 232<h2 id=idea>Idea for a Checker</h2>233 Here are several questions which you should consider when evaluating your234 checker idea:235 <ul>236 <li>Can the check be effectively implemented without path-sensitive237 analysis? See <a href="#ast">AST Visitors</a>.</li>238 239 <li>How high the false positive rate is going to be? Looking at the occurrences240 of the issue you want to write a checker for in the existing code bases might241 give you some ideas. </li>242 243 <li>How the current limitations of the analysis will effect the false alarm244 rate? Currently, the analyzer only reasons about one procedure at a time (no245 inter-procedural analysis). Also, it uses a simple range tracking based246 solver to model symbolic execution.</li>247 248 <li>Consult the <a249 href="https://github.com/llvm/llvm-project/labels/clang%3Astatic%20analyzer">GitHub Issues</a>250 to get some ideas for new checkers and consider starting with improving/fixing251 bugs in the existing checkers.</li>252 </ul>253 254<p>Once an idea for a checker has been chosen, there are two key decisions that255need to be made:256 <ul>257 <li> Which events the checker should be tracking. This is discussed in more258 detail in the section <a href="#events_callbacks">Events, Callbacks, and259 Checker Class Structure</a>.260 <li> What checker-specific data needs to be stored as part of the program261 state (if any). This should be minimized as much as possible. More detail about262 implementing custom program state is given in section <a263 href="#extendingstates">Custom Program States</a>.264 </ul>265 266 267<h2 id=registration>Checker Registration</h2>268 All checker implementation files are located in269 <tt>clang/lib/StaticAnalyzer/Checkers</tt> folder. The steps below describe270 how the checker <tt>SimpleStreamChecker</tt>, which checks for misuses of271 stream APIs, was registered with the analyzer.272 Similar steps should be followed for a new checker.273<ol>274 <li>A new checker implementation file, <tt>SimpleStreamChecker.cpp</tt>, was275 created in the directory <tt>lib/StaticAnalyzer/Checkers</tt>.276 <li>The following registration code was added to the implementation file:277<pre class="code_example">278void ento::registerSimpleStreamChecker(CheckerManager &mgr) {279 mgr.registerChecker<SimpleStreamChecker>();280}281</pre>282<li>A package was selected for the checker and the checker was defined in the283table of checkers at <tt>include/clang/StaticAnalyzer/Checkers/Checkers.td</tt>.284Since all checkers should first be developed as "alpha", and the SimpleStreamChecker285performs UNIX API checks, the correct package is "alpha.unix", and the following286was added to the corresponding <tt>UnixAlpha</tt> section of <tt>Checkers.td</tt>:287<pre class="code_example">288let ParentPackage = UnixAlpha in {289...290def SimpleStreamChecker : Checker<"SimpleStream">,291 HelpText<"Check for misuses of stream APIs">,292 DescFile<"SimpleStreamChecker.cpp">;293...294} // end "alpha.unix"295</pre>296 297<li>The source code file was made visible to CMake by adding it to298<tt>lib/StaticAnalyzer/Checkers/CMakeLists.txt</tt>.299 300</ol>301 302After adding a new checker to the analyzer, one can verify that the new checker303was successfully added by seeing if it appears in the list of available checkers:304<br> <tt><b>$clang -cc1 -analyzer-checker-help</b></tt>305 306<h2 id=events_callbacks>Events, Callbacks, and Checker Class Structure</h2>307 308<p> All checkers inherit from the <tt><a309href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1Checker.html">310Checker</a></tt> template class; the template parameter(s) describe the type of311events that the checker is interested in processing. The various types of events312that are available are described in the file <a313href="https://clang.llvm.org/doxygen/CheckerDocumentation_8cpp_source.html">314CheckerDocumentation.cpp</a>315 316<p> For each event type requested, a corresponding callback function must be317defined in the checker class (<a318href="https://clang.llvm.org/doxygen/CheckerDocumentation_8cpp_source.html">319CheckerDocumentation.cpp</a> shows the320correct function name and signature for each event type).321 322<p> As an example, consider <tt>SimpleStreamChecker</tt>. This checker needs to323take action at the following times:324 325<ul>326<li>Before making a call to a function, check if the function is <tt>fclose</tt>.327If so, check the parameter being passed.328<li>After making a function call, check if the function is <tt>fopen</tt>. If329so, process the return value.330<li>When values go out of scope, check whether they are still-open file331descriptors, and report a bug if so. In addition, remove any information about332them from the program state in order to keep the state as small as possible.333<li>When file pointers "escape" (are used in a way that the analyzer can no longer334track them), mark them as such. This prevents false positives in the cases where335the analyzer cannot be sure whether the file was closed or not.336</ul>337 338<p>These events that will be used for each of these actions are, respectively, <a339href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1check_1_1PreCall.html">PreCall</a>,340<a341href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1check_1_1PostCall.html">PostCall</a>,342<a343href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1check_1_1DeadSymbols.html">DeadSymbols</a>,344and <a345href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1check_1_1PointerEscape.html">PointerEscape</a>.346The high-level structure of the checker's class is thus:347 348<pre class="code_example">349class SimpleStreamChecker : public Checker<check::PreCall,350 check::PostCall,351 check::DeadSymbols,352 check::PointerEscape> {353public:354 355 void checkPreCall(const CallEvent &Call, CheckerContext &C) const;356 357 void checkPostCall(const CallEvent &Call, CheckerContext &C) const;358 359 void checkDeadSymbols(SymbolReaper &SR, CheckerContext &C) const;360 361 ProgramStateRef checkPointerEscape(ProgramStateRef State,362 const InvalidatedSymbols &Escaped,363 const CallEvent *Call,364 PointerEscapeKind Kind) const;365};366</pre>367 368<h2 id=extendingstates>Custom Program States</h2>369 370<p> Checkers often need to keep track of information specific to the checks they371perform. However, since checkers have no guarantee about the order in which the372program will be explored, or even that all possible paths will be explored, this373state information cannot be kept within individual checkers. Therefore, if374checkers need to store custom information, they need to add new categories of375data to the <tt>ProgramState</tt>. The preferred way to do so is to use one of376several macros designed for this purpose. They are:377 378<ul>379<li><a380href="https://clang.llvm.org/doxygen/ProgramStateTrait_8h.html#ae4cddb54383cd702a045d7c61b009147">REGISTER_TRAIT_WITH_PROGRAMSTATE</a>:381Used when the state information is a single value. The methods available for382state types declared with this macro are <tt>get</tt>, <tt>set</tt>, and383<tt>remove</tt>.384<li><a385href="https://clang.llvm.org/doxygen/ProgramStateTrait_8h.html#aa27656fa0ce65b0d9ba12eb3c02e8be9">REGISTER_LIST_WITH_PROGRAMSTATE</a>:386Used when the state information is a list of values. The methods available for387state types declared with this macro are <tt>add</tt>, <tt>get</tt>,388<tt>remove</tt>, and <tt>contains</tt>.389<li><a390href="https://clang.llvm.org/doxygen/ProgramStateTrait_8h.html#ad90f9387b94b344eaaf499afec05f4d1">REGISTER_SET_WITH_PROGRAMSTATE</a>:391Used when the state information is a set of values. The methods available for392state types declared with this macro are <tt>add</tt>, <tt>get</tt>,393<tt>remove</tt>, and <tt>contains</tt>.394<li><a395href="https://clang.llvm.org/doxygen/ProgramStateTrait_8h.html#a6d1893bb8c18543337b6c363c1319fcf">REGISTER_MAP_WITH_PROGRAMSTATE</a>:396Used when the state information is a map from a key to a value. The methods397available for state types declared with this macro are <tt>add</tt>,398<tt>set</tt>, <tt>get</tt>, <tt>remove</tt>, and <tt>contains</tt>.399</ul>400 401<p>All of these macros take as parameters the name to be used for the custom402category of state information and the data type(s) to be used for storage. The403data type(s) specified will become the parameter type and/or return type of the404methods that manipulate the new category of state information. Each of these405methods are templated with the name of the custom data type.406 407<p>For example, a common case is the need to track data associated with a408symbolic expression; a map type is the most logical way to implement this. The409key for this map will be a pointer to a symbolic expression410(<tt>SymbolRef</tt>). If the data type to be associated with the symbolic411expression is an integer, then the custom category of state information would be412declared as413 414<pre class="code_example">415REGISTER_MAP_WITH_PROGRAMSTATE(ExampleDataType, SymbolRef, int)416</pre>417 418The data would be accessed with the function419 420<pre class="code_example">421ProgramStateRef state;422SymbolRef Sym;423...424int currentlValue = state->get<ExampleDataType>(Sym);425</pre>426 427and set with the function428 429<pre class="code_example">430ProgramStateRef state;431SymbolRef Sym;432int newValue;433...434ProgramStateRef newState = state->set<ExampleDataType>(Sym, newValue);435</pre>436 437<p>In addition, the macros define a data type used for storing the data of the438new data category; the name of this type is the name of the data category with439"Ty" appended. For <tt>REGISTER_TRAIT_WITH_PROGRAMSTATE</tt>, this will simply440be passed data type; for the other three macros, this will be a specialized441version of the <a442href="https://llvm.org/doxygen/classllvm_1_1ImmutableList.html">llvm::ImmutableList</a>,443<a444href="https://llvm.org/doxygen/classllvm_1_1ImmutableSet.html">llvm::ImmutableSet</a>,445or <a446href="https://llvm.org/doxygen/classllvm_1_1ImmutableMap.html">llvm::ImmutableMap</a>447templated class. For the <tt>ExampleDataType</tt> example above, the type448created would be equivalent to writing the declaration:449 450<pre class="code_example">451using ExampleDataTypeTy = llvm::ImmutableMap<SymbolRef, int>;452</pre>453 454<p>These macros will cover a majority of use cases; however, they still have a455few limitations. They cannot be used inside namespaces (since they expand to456contain top-level namespace references), and the data types that they define457cannot be referenced from more than one file.458 459<p>Note that <tt>ProgramStates</tt> are immutable; instead of modifying an existing460one, functions that modify the state will return a copy of the previous state461with the change applied. This updated state must be then provided to the462analyzer core by calling the <tt>CheckerContext::addTransition</tt> function.463<h2 id=bugs>Bug Reports</h2>464 465 466<p> When a checker detects a mistake in the analyzed code, it needs a way to467report it to the analyzer core so that it can be displayed. The two classes used468to construct this report are <tt><a469href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1BugType.html">BugType</a></tt>470and <tt><a471href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1BugReport.html">472BugReport</a></tt>.473 474<p>475<tt>BugType</tt>, as the name would suggest, represents a type of bug. The476constructor for <tt>BugType</tt> takes two parameters: The name of the bug477type, and the name of the category of the bug. These are used (e.g.) in the478summary page generated by the scan-build tool.479 480<P>481 The <tt>BugReport</tt> class represents a specific occurrence of a bug. In482 the most common case, three parameters are used to form a <tt>BugReport</tt>:483<ol>484<li>The type of bug, specified as an instance of the <tt>BugType</tt> class.485<li>A short descriptive string. This is placed at the location of the bug in486the detailed line-by-line output generated by scan-build.487<li>The context in which the bug occurred. This includes both the location of488the bug in the program and the program's state when the location is reached. These are489both encapsulated in an <tt>ExplodedNode</tt>.490</ol>491 492<p>In order to obtain the correct <tt>ExplodedNode</tt>, a decision must be made493as to whether or not analysis can continue along the current path. This decision494is based on whether the detected bug is one that would prevent the program under495analysis from continuing. For example, leaking of a resource should not stop496analysis, as the program can continue to run after the leak. Dereferencing a497null pointer, on the other hand, should stop analysis, as there is no way for498the program to meaningfully continue after such an error.499 500<p>If analysis can continue, then the most recent <tt>ExplodedNode</tt>501generated by the checker can be passed to the <tt>BugReport</tt> constructor502without additional modification. This <tt>ExplodedNode</tt> will be the one503returned by the most recent call to <a504href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1CheckerContext.html#a9717efea3fbc71523984160ae7ae9d41">CheckerContext::addTransition</a>.505If no transition has been performed during the current callback, the checker should call <a506href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1CheckerContext.html#a9717efea3fbc71523984160ae7ae9d41">CheckerContext::addTransition()</a>507and use the returned node for bug reporting.508 509<p>If analysis can not continue, then the current state should be transitioned510into a so-called <i>sink node</i>, a node from which no further analysis will be511performed. This is done by calling the <a512href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1CheckerContext.html#a249245cdf2384738921f134c8d7d909a">513CheckerContext::generateSink</a> function; this function is the same as the514<tt>addTransition</tt> function, but marks the state as a sink node. Like515<tt>addTransition</tt>, this returns an <tt>ExplodedNode</tt> with the updated516state, which can then be passed to the <tt>BugReport</tt> constructor.517 518<p>519After a <tt>BugReport</tt> is created, it should be passed to the analyzer core520by calling <a href="https://clang.llvm.org/doxygen/classclang_1_1ento_1_1CheckerContext.html#af50a9f46f6ea787a2a8e4ad7f86576e7">CheckerContext::emitReport</a>.521 522<h2 id=ast>AST Visitors</h2>523 Some checks might not require path-sensitivity to be effective. Simple AST walk524 might be sufficient. If that is the case, consider implementing a Clang525 compiler warning. On the other hand, a check might not be acceptable as a compiler526 warning; for example, because of a relatively high false positive rate. In this527 situation, AST callbacks <tt><b>checkASTDecl</b></tt> and528 <tt><b>checkASTCodeBody</b></tt> are your best friends.529 530<h2 id=testing>Testing</h2>531 Every patch should be well tested with Clang regression tests. The checker tests532 live in <tt>clang/test/Analysis</tt> folder. To run all of the analyzer tests,533 execute the following from the <tt>clang</tt> build directory:534 <pre class="code">535 $ <b>bin/llvm-lit -sv ../llvm/tools/clang/test/Analysis</b>536 </pre>537 538<h2 id=commands>Useful Commands/Debugging Hints</h2>539 540<h3 id=attaching>Attaching the Debugger</h3>541 542<p>When your command contains the <tt><b>-cc1</b></tt> flag, you can attach the543debugger to it directly:</p>544 545<pre class="code">546 $ <b>gdb --args clang -cc1 -analyze -analyzer-checker=core test.c</b>547 $ <b>lldb -- clang -cc1 -analyze -analyzer-checker=core test.c</b>548</pre>549 550<p>551Otherwise, if your command line contains <tt><b>--analyze</b></tt>,552the actual clang instance would be run in a separate process. In553order to debug it, use the <tt><b>-###</b></tt> flag for obtaining554the command line of the child process:555</p>556 557<pre class="code">558 $ <b>clang --analyze test.c -\#\#\#</b>559</pre>560 561<p>562Below we describe a few useful command line arguments, all of which assume that563you are running <tt><b>clang -cc1</b></tt>.564</p>565 566<h3 id=narrowing>Narrowing Down the Problem</h3>567 568<p>While investigating a checker-related issue, instruct the analyzer to only569execute a single checker:570</p>571<pre class="code">572 $ <b>clang -cc1 -analyze -analyzer-checker=osx.KeychainAPI test.c</b>573</pre>574 575<p>If you are experiencing a crash, to see which function is failing while576processing a large file use the <tt><b>-analyzer-display-progress</b></tt>577option.</p>578 579<p>To selectively analyze only the given function, use the580<tt><b>-analyze-function</b></tt> option:</p>581<pre class="code">582 $ <b>clang -cc1 -analyze -analyzer-checker=core test.c -analyzer-display-progress</b>583 ANALYZE (Syntax): test.c foo584 ANALYZE (Syntax): test.c bar585 ANALYZE (Path, Inline_Regular): test.c bar586 ANALYZE (Path, Inline_Regular): test.c foo587 $ <b>clang -cc1 -analyze -analyzer-checker=core test.c -analyzer-display-progress -analyze-function=foo</b>588 ANALYZE (Syntax): test.c foo589 ANALYZE (Path, Inline_Regular): test.c foo590</pre>591 592<b>Note: </b> a fully qualified function name has to be used when selecting593C++ functions and methods, Objective-C methods and blocks, e.g.:594 595<pre class="code">596 $ <b>clang -cc1 -analyze -analyzer-checker=core test.cc -analyze-function='foo(int)'</b>597</pre>598 599The fully qualified name can be found from the600<tt><b>-analyzer-display-progress</b></tt> output.601 602<p>The bug reporter mechanism removes path diagnostics inside intermediate603function calls that have returned by the time the bug was found and contain604no interesting pieces. Usually it is up to the checkers to produce more605interesting pieces by adding custom <tt>BugReporterVisitor</tt> objects.606However, you can disable path pruning while debugging with the607<tt><b>-analyzer-config prune-paths=false</b></tt> option.608 609<h3 id=visualizing>Visualizing the Analysis</h3>610 611<p>To dump the AST, which often helps understanding how the program should612behave:</p>613<pre class="code">614 $ <b>clang -cc1 -ast-dump test.c</b>615</pre>616 617<p>To view/dump CFG use <tt>debug.ViewCFG</tt> or <tt>debug.DumpCFG</tt>618checkers:</p>619<pre class="code">620 $ <b>clang -cc1 -analyze -analyzer-checker=debug.ViewCFG test.c</b>621</pre>622 623<p><tt>ExplodedGraph</tt> (the state graph explored by the analyzer) can be624visualized with another debug checker:</p>625<pre class="code">626 $ <b>clang -cc1 -analyze -analyzer-checker=debug.ViewExplodedGraph test.c</b>627</pre>628<p>Or, equivalently, with <tt><b>-analyzer-viz-egraph-graphviz</b></tt>629option, which does the same thing - dumps the exploded graph in graphviz630<tt><b>.dot</b></tt> format.</p>631 632<p>You can convert <tt><b>.dot</b></tt> files into other formats - in633particular, converting to <tt><b>.svg</b></tt> and viewing in your web634browser might be more comfortable than using a <tt><b>.dot</b></tt> viewer:</p>635<pre class="code">636 $ <b>dot -Tsvg ExprEngine-501e2e.dot -o ExprEngine-501e2e.svg</b>637</pre>638 639<p>The <tt><b>-trim-egraph</b></tt> option removes all paths except those640leading to bug reports from the exploded graph dump. This is useful641because exploded graphs are often huge and hard to navigate.</p>642 643<p>Viewing <tt>ExplodedGraph</tt> is your most powerful tool for understanding644the analyzer's false positives, because it gives comprehensive information645on every decision made by the analyzer across all analysis paths.</p>646 647<p>There are more debug checkers available. To see all available debug checkers:648</p>649<pre class="code">650 $ <b>clang -cc1 -analyzer-checker-help | grep "debug"</b>651</pre>652 653<h3 id=debugprints>Debug Prints and Tricks</h3>654 655<p>To view "half-baked" <tt>ExplodedGraph</tt> while debugging, jump to a frame656that has <tt>clang::ento::ExprEngine</tt> object and execute:</p>657<pre class="code">658 (gdb) <b>p ViewGraph(0)</b>659</pre>660 661<p>To see the <tt>ProgramState</tt> while debugging use the following command.662<pre class="code">663 (gdb) <b>p State->dump()</b>664</pre>665 666<p>To see <tt>clang::Expr</tt> while debugging use the following command. If you667pass in a <tt>SourceManager</tt> object, it will also dump the corresponding line in the668source code.</p>669<pre class="code">670 (gdb) <b>p E->dump()</b>671</pre>672 673<p>To dump AST of a method that the current <tt>ExplodedNode</tt> belongs674to:</p>675<pre class="code">676 (gdb) <b>p C.getPredecessor()->getCodeDecl().getBody()->dump()</b>677</pre>678 679<h2 id=links>Making Your Checker Better</h2>680<ul>681<li>User facing documentation is important for adoption! Make sure the <a href="/available_checks.html">checker list </a>is updated682 at the homepage of the analyzer. Also ensure the description is clear to683 non-analyzer-developers in <tt>Checkers.td</tt>.</li>684<li>Warning and note messages should be clear and easy to understand, even if a bit long.</li>685<ul>686 <li>Messages should start with a capital letter (unlike Clang warnings!) and should not687 end with <tt>.</tt>.</li>688 <li>Articles are usually omitted, eg. <tt>Dereference of a null pointer</tt> ->689 <tt>Dereference of null pointer</tt>.</li>690 <li>Introduce <tt>BugReporterVisitor</tt>s to emit additional notes that explain the warning691 to the user better. There are some existing visitors that might be useful for your check,692 e.g. <tt>trackNullOrUndefValue</tt>. For example, SimpleStreamChecker should highlight693 the event of opening the file when reporting a file descriptor leak.</li>694</ul>695<li>If the check tracks anything in the program state, it needs to implement the696 <tt>checkDeadSymbols</tt>callback to clean the state up.</li>697<li>The check should conservatively assume that the program is correct when a tracked symbol698 is passed to a function that is unknown to the analyzer.699 <tt>checkPointerEscape</tt> callback could help you handle that case.</li>700<li>Use safe and convenient APIs!</li>701<ul>702 <li>Always use <tt>CheckerContext::generateErrorNode</tt> and703 <tt>CheckerContext::generateNonFatalErrorNode</tt> for emitting bug reports.704 Most importantly, never emit report against <tt>CheckerContext::getPredecessor</tt>.</li>705 <li>Prefer <tt>checkPreCall</tt> and <tt>checkPostCall</tt> to706 <tt>checkPreStmt<CallExpr></tt> and <tt>checkPostStmt<CallExpr></tt>.</li>707 <li>Use <tt>CallDescription</tt> to detect hardcoded API calls in the program.</li>708 <li>Simplify <tt>C.getState()->getSVal(E, C.getLocationContext())</tt> to <tt>C.getSVal(E)</tt>.</li>709</ul>710<li>Common sources of crashes:</li>711<ul>712 <li><tt>CallEvent::getOriginExpr</tt> is nullable - for example, it returns null for an713 automatic destructor of a variable. The same applies to some values generated while the714 call was modeled, eg. <tt>SymbolConjured::getStmt</tt> is nullable.</li>715 <li><tt>CallEvent::getDecl</tt> is nullable - for example, it returns null for a716 call of symbolic function pointer.</li>717 <li><tt>addTransition</tt>, <tt>generateSink</tt>, <tt>generateNonFatalErrorNode</tt>,718 <tt>generateErrorNode</tt> are nullable because you can transition to a node that you have already visited.</li>719 <li>Methods of <tt>CallExpr</tt>/<tt>FunctionDecl</tt>/<tt>CallEvent</tt> that720 return arguments crash when the argument is out-of-bounds. If you checked the function name,721 it doesn't mean that the function has the expected number of arguments!722 Which is why you should use <tt>CallDescription</tt>.</li>723 <li>Nullability of different entities within different kinds of symbols and regions is usually724 documented via assertions in their constructors.</li>725 <li><tt>NamedDecl::getName</tt> will fail if the name of the declaration is not a single token,726 e.g. for destructors. You could use <tt>NamedDecl::getNameAsString</tt> for those cases.727 Note that this method is much slower and should be used sparringly, e.g. only when generating reports728 but not during analysis.</li>729 <li>Is <tt>-analyzer-checker=core</tt> included in all test <tt>RUN:</tt> lines? It was never supported730 to run the analyzer with the core checks disabled. It might cause unexpected behavior and731 crashes. You should do all your testing with the core checks enabled.</li>732</ul>733</ul>734<li>Patterns that you should most likely avoid even if they're not technically wrong:</li>735<ul>736 <li><tt>BugReporterVisitor</tt> should most likely not match the AST of the current program point737 to decide when to emit a note. It is much easier to determine that by observing changes in738 the program state.</li>739 <li>In <tt>State->getSVal(Region)</tt>, if <tt>Region</tt> is not known to be a <tt>TypedValueRegion</tt>740 and the optional type argument is not specified, the checker may accidentally try to dereference a741 void pointer.</li>742 <li>Checker logic should not depend on whether a certain value is a <tt>Loc</tt> or <tt>NonLoc</tt>.743 It should be immediately obvious whether the <tt>SVal</tt> is a <tt>Loc</tt> or a744 <tt>NonLoc</tt> depending on the AST that is being checked. Checking whether a value745 is <tt>Loc</tt> or <tt>Unknown</tt>/<tt>Undefined</tt> or whether the value is746 <tt>NonLoc</tt> or <tt>Unknown</tt>/<tt>Undefined</tt> is totally fine.</li>747 <li>New symbols should not be constructed in the checker via direct calls to <tt>SymbolManager</tt>,748 unless they are of <tt>SymbolMetadata</tt> class tagged by the checker,749 or they represent newly created values such as the return value in <tt>evalCall</tt>.750 For modeling arithmetic/bitwise/comparison operations, <tt>SValBuilder</tt> should be used.</li>751 <li>Custom <tt>ProgramPointTag</tt>s should not be created within the checker. There is usually752 no good reason for a checker to chain multiple nodes together, because checkers aren't worklists.</li>753</ul>754<li>Checkers are encouraged to actively participate in the analysis by sharing755 their knowledge about the program state with the rest of the analyzer,756 but they should not be disrupting the analysis unnecessarily:</li>757<ul>758 <li>If a checker splits program state, this must be based on knowledge that759 the newly appearing branches are definitely possible and worth exploring760 from the user's perspective. Otherwise the state split should be delayed761 until there's an indication that one of the paths is taken, or one of the762 paths needs to be dropped entirely. For example, it is fine to eagerly split763 paths while modeling <tt>isalpha(x)</tt> as long as <tt>x</tt> is constrained accordingly on764 each path. At the same time, it is not a good idea to split paths over the765 return value of <tt>printf()</tt> while modeling the call because nobody ever checks766 for errors in <tt>printf</tt>; at best, it'd just double the remaining analysis time.767 </li>768 <li>Caution is advised when using <tt>CheckerContext::generateNonFatalErrorNode</tt>769 because it generates an independent transition, much like <tt>addTransition</tt>.770 It is easy to accidentally split paths while using it. Ideally, try to771 structure the code so that it was obvious that every <tt>addTransition</tt> or772 <tt>generateNonFatalErrorNode</tt> (or sequence of such if the split is intended) is773 immediately followed by return from the checker callback.</li>774 <li>Multiple implementations of <tt>evalCall</tt> in different checkers should not conflict.</li>775 <li>When implementing <tt>evalAssume</tt>, the checker should always return a non-null state776 for either the true assumption or the false assumption (or both).</li>777 <li>Checkers shall not mutate values of expressions, i.e. use the <tt>ProgramState::BindExpr</tt> API,778 unless they are fully responsible for computing the value.779 Under no circumstances should they change non-<tt>Unknown</tt> values of expressions.780 Currently the only valid use case for this API in checkers is to model the return value in the <tt>evalCall</tt> callback.781 If expression values are incorrect, <tt>ExprEngine</tt> needs to be fixed instead.</li>782</ul>783 784<h2 id=additioninformation>Additional Sources of Information</h2>785 786Here are some additional resources that are useful when working on the Clang787Static Analyzer:788 789<ul>790<li><a href="https://lcs.ios.ac.cn/~xzx/memmodel.pdf">Xu, Zhongxing &791Kremenek, Ted & Zhang, Jian. (2010). A Memory Model for Static Analysis of C792Programs.</a></li>793<li><a href="https://github.com/llvm/llvm-project/blob/main/clang/lib/StaticAnalyzer/README.txt">794The Clang Static Analyzer README</a></li>795<li><a href="https://github.com/llvm/llvm-project/blob/main/clang/docs/analyzer/developer-docs/RegionStore.rst">796Documentation for how the Store works</a></li>797<li><a href="https://github.com/llvm/llvm-project/blob/main/clang/docs/analyzer/developer-docs/IPA.rst">798Documentation about inlining</a></li>799<li> The "Building a Checker in 24 hours" presentation given at the <a800href="https://llvm.org/devmtg/2012-11">November 2012 LLVM Developer's801meeting</a>. Describes the construction of SimpleStreamChecker. <a802href="https://llvm.org/devmtg/2012-11/Zaks-Rose-Checker24Hours.pdf">Slides</a>803and <a804href="https://youtu.be/kdxlsP5QVPw">video</a>805are available.</li>806<li>807<a href="https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf">808Artem Degrachev: Clang Static Analyzer: A Checker Developer's Guide809</a> (reading the previous items first might be a good idea)</li>810<li>The list of <a href="implicit_checks.html">Implicit Checkers</a></li>811<li> <a href="https://clang.llvm.org/doxygen">Clang doxygen</a>. Contains812up-to-date documentation about the APIs available in Clang. Relevant entries813have been linked throughout this page. Also of use is the814<a href="https://llvm.org/doxygen">LLVM doxygen</a>, when dealing with classes815from LLVM.</li>816<li>817 The <a href="https://discourse.llvm.org/c/clang/"> Clang Frontend Discourse site</a>.818 This is the primary forum discussing ideas and posting questions about Clang development.819 For posting Clang Static Analyzer specific questions, please visit the820 <a href="https://discourse.llvm.org/c/clang/static-analyzer/"> Static Analyzer subcategory</a>821 of the same site. In the past, Static Analyzer discussions took place at the822 <a href="https://lists.llvm.org/pipermail/cfe-dev/"> cfe-dev</a> mailing list, which is now823 archived and superseeded by the mentioned Discourse site.824</li>825</ul>826 827</div>828</div>829</body>830</html>831