228 lines · plain
1====================================2LLVM bugpoint tool: design and usage3====================================4 5.. contents::6 :local:7 8Description9===========10 11``bugpoint`` narrows down the source of problems in LLVM tools and passes. It12can be used to debug three types of failures: optimizer crashes, miscompilations13by optimizers, or bad native code generation (including problems in the static14and JIT compilers). It aims to reduce large test cases to small, useful ones.15For example, if ``opt`` crashes while optimizing a file, it will identify the16optimization (or combination of optimizations) that causes the crash, and reduce17the file down to a small example which triggers the crash.18 19For detailed case scenarios, such as debugging ``opt``, or one of the LLVM code20generators, see :doc:`HowToSubmitABug`.21 22Design Philosophy23=================24 25``bugpoint`` is designed to be a useful tool without requiring any hooks into26the LLVM infrastructure at all. It works with any and all LLVM passes and code27generators, and does not need to "know" how they work. Because of this, it may28appear to do stupid things or miss obvious simplifications. ``bugpoint`` is29also designed to trade off programmer time for computer time in the30compiler-debugging process; consequently, it may take a long period of31(unattended) time to reduce a test case, but we feel it is still worth it. Note32that ``bugpoint`` is generally very quick unless debugging a miscompilation33where each test of the program (which requires executing it) takes a long time.34 35Automatic Debugger Selection36----------------------------37 38``bugpoint`` reads each ``.bc`` or ``.ll`` file specified on the command line39and links them together into a single module, called the test program. If any40LLVM passes are specified on the command line, it runs these passes on the test41program. If any of the passes crash, or if they produce malformed output (which42causes the verifier to abort), ``bugpoint`` starts the `crash debugger`_.43 44Otherwise, if the ``-output`` option was not specified, ``bugpoint`` runs the45test program with the "safe" backend (which is assumed to generate good code) to46generate a reference output. Once ``bugpoint`` has a reference output for the47test program, it tries executing it with the selected code generator. If the48selected code generator crashes, ``bugpoint`` starts the `crash debugger`_ on49the code generator. Otherwise, if the resulting output differs from the50reference output, it assumes the difference resulted from a code generator51failure, and starts the `code generator debugger`_.52 53Finally, if the output of the selected code generator matches the reference54output, ``bugpoint`` runs the test program after all of the LLVM passes have55been applied to it. If its output differs from the reference output, it assumes56the difference resulted from a failure in one of the LLVM passes, and enters the57`miscompilation debugger`_. Otherwise, there is no problem ``bugpoint`` can58debug.59 60.. _crash debugger:61 62Crash debugger63--------------64 65If an optimizer or code generator crashes, ``bugpoint`` will try as hard as it66can to reduce the list of passes (for optimizer crashes) and the size of the67test program. First, ``bugpoint`` figures out which combination of optimizer68passes triggers the bug. This is useful when debugging a problem exposed by69``opt``, for example, because it runs over 38 passes.70 71Next, ``bugpoint`` tries removing functions from the test program, to reduce its72size. Usually it is able to reduce a test program to a single function, when73debugging intraprocedural optimizations. Once the number of functions has been74reduced, it attempts to delete various edges in the control flow graph, to75reduce the size of the function as much as possible. Finally, ``bugpoint``76deletes any individual LLVM instructions whose absence does not eliminate the77failure. At the end, ``bugpoint`` should tell you what passes crash, give you a78bitcode file, and give you instructions on how to reproduce the failure with79``opt`` or ``llc``.80 81.. _code generator debugger:82 83Code generator debugger84-----------------------85 86The code generator debugger attempts to narrow down the amount of code that is87being miscompiled by the selected code generator. To do this, it takes the test88program and partitions it into two pieces: one piece which it compiles with the89"safe" backend (into a shared object), and one piece which it runs with either90the JIT or the static LLC compiler. It uses several techniques to reduce the91amount of code pushed through the LLVM code generator, to reduce the potential92scope of the problem. After it is finished, it emits two bitcode files (called93"test" [to be compiled with the code generator] and "safe" [to be compiled with94the "safe" backend], respectively), and instructions for reproducing the95problem. The code generator debugger assumes that the "safe" backend produces96good code.97 98.. _miscompilation debugger:99 100Miscompilation debugger101-----------------------102 103The miscompilation debugger works similarly to the code generator debugger. It104works by splitting the test program into two pieces, running the optimizations105specified on one piece, linking the two pieces back together, and then executing106the result. It attempts to narrow down the list of passes to the one (or few)107which are causing the miscompilation, then reduce the portion of the test108program which is being miscompiled. The miscompilation debugger assumes that109the selected code generator is working properly.110 111Advice for using bugpoint112=========================113 114``bugpoint`` can be a remarkably useful tool, but it sometimes works in115non-obvious ways. Here are some hints and tips:116 117* In the code generator and miscompilation debuggers, ``bugpoint`` only works118 with programs that have deterministic output. Thus, if the program outputs119 ``argv[0]``, the date, time, or any other "random" data, ``bugpoint`` may120 misinterpret differences in these data, when output, as the result of a121 miscompilation. Programs should be temporarily modified to disable outputs122 that are likely to vary from run to run.123 124* In the `crash debugger`_, ``bugpoint`` does not distinguish different crashes125 during reduction. Thus, if new crash or miscompilation happens, ``bugpoint``126 will continue with the new crash instead. If you would like to stick to127 particular crash, you should write check scripts to validate the error128 message, see ``-compile-command`` in :doc:`CommandGuide/bugpoint`.129 130* In the code generator and miscompilation debuggers, debugging will go faster131 if you manually modify the program or its inputs to reduce the runtime, but132 still exhibit the problem.133 134* ``bugpoint`` is extremely useful when working on a new optimization: it helps135 track down regressions quickly. To avoid having to relink ``bugpoint`` every136 time you change your optimization however, have ``bugpoint`` dynamically load137 your optimization with the ``-load`` option.138 139* ``bugpoint`` can generate a lot of output and run for a long period of time.140 It is often useful to capture the output of the program to file. For example,141 in the C shell, you can run:142 143 .. code-block:: console144 145 $ bugpoint ... |& tee bugpoint.log146 147 to get a copy of ``bugpoint``'s output in the file ``bugpoint.log``, as well148 as on your terminal.149 150* ``bugpoint`` cannot debug problems with the LLVM linker. If ``bugpoint``151 crashes before you see its "All input ok" message, you might try ``llvm-link152 -v`` on the same set of input files. If that also crashes, you may be153 experiencing a linker bug.154 155* ``bugpoint`` is useful for proactively finding bugs in LLVM. Invoking156 ``bugpoint`` with the ``-find-bugs`` option will cause the list of specified157 optimizations to be randomized and applied to the program. This process will158 repeat until a bug is found or the user kills ``bugpoint``.159 160* ``bugpoint`` can produce IR which contains long names. Run ``opt161 -passes=metarenamer`` over the IR to rename everything using easy-to-read,162 metasyntactic names. Alternatively, run ``opt -passes=strip,instnamer`` to163 rename everything with very short (often purely numeric) names.164 165What to do when bugpoint isn't enough166=====================================167 168Sometimes, ``bugpoint`` is not enough. In particular, InstCombine and169TargetLowering both have visitor structured code with lots of potential170transformations. If the process of using bugpoint has left you with still too171much code to figure out and the problem seems to be in instcombine, the172following steps may help. These same techniques are useful with TargetLowering173as well.174 175Turn on ``-debug-only=instcombine`` and see which transformations within176instcombine are firing by selecting out lines with "``IC``" in them.177 178At this point, you have a decision to make. Is the number of transformations179small enough to step through them using a debugger? If so, then try that.180 181If there are too many transformations, then a source modification approach may182be helpful. In this approach, you can modify the source code of instcombine to183disable just those transformations that are being performed on your test input184and perform a binary search over the set of transformations. One set of places185to modify are the "``visit*``" methods of ``InstCombiner`` (*e.g.*186``visitICmpInst``) by adding a "``return false``" as the first line of the187method.188 189If that still doesn't remove enough, then change the caller of190``InstCombiner::DoOneIteration``, ``InstCombiner::runOnFunction`` to limit the191number of iterations.192 193You may also find it useful to use "``-stats``" now to see what parts of194instcombine are firing. This can guide where to put additional reporting code.195 196At this point, if the amount of transformations is still too large, then197inserting code to limit whether or not to execute the body of the code in the198visit function can be helpful. Add a static counter which is incremented on199every invocation of the function. Then add code which simply returns false on200desired ranges. For example:201 202.. code-block:: c++203 204 205 static int calledCount = 0;206 calledCount++;207 LLVM_DEBUG(if (calledCount < 212) return false);208 LLVM_DEBUG(if (calledCount > 217) return false);209 LLVM_DEBUG(if (calledCount == 213) return false);210 LLVM_DEBUG(if (calledCount == 214) return false);211 LLVM_DEBUG(if (calledCount == 215) return false);212 LLVM_DEBUG(if (calledCount == 216) return false);213 LLVM_DEBUG(dbgs() << "visitXOR calledCount: " << calledCount << "\n");214 LLVM_DEBUG(dbgs() << "I: "; I->dump());215 216could be added to ``visitXOR`` to limit ``visitXor`` to being applied only to217calls 212 and 217. This is from an actual test case and raises an important218point---a simple binary search may not be sufficient, as transformations that219interact may require isolating more than one call. In TargetLowering, use220``return SDNode();`` instead of ``return false;``.221 222Now that the number of transformations is down to a manageable number, try223examining the output to see if you can figure out which transformations are224being done. If that can be figured out, then do the usual debugging. If which225code corresponds to the transformation being performed isn't obvious, set a226breakpoint after the call count based disabling and step through the code.227Alternatively, you can use "``printf``" style debugging to report waypoints.228