543 lines · plain
1=======2Remarks3=======4 5.. contents::6 :local:7 8Introduction to the LLVM remark diagnostics9===========================================10 11LLVM is able to emit diagnostics from passes describing whether an optimization12has been performed or missed for a particular reason, which should give more13insight to users about what the compiler did during the compilation pipeline.14 15There are three main remark types:16 17``Passed``18 19 Remarks that describe a successful optimization performed by the compiler.20 21 :Example:22 23 ::24 25 foo inlined into bar with (cost=always): always inline attribute26 27``Missed``28 29 Remarks that describe an attempt to an optimization by the compiler that30 could not be performed.31 32 :Example:33 34 ::35 36 foo not inlined into bar because it should never be inlined37 (cost=never): noinline function attribute38 39``Analysis``40 41 Remarks that describe the result of an analysis, that can bring more42 information to the user regarding the generated code.43 44 :Example:45 46 ::47 48 16 stack bytes in function49 50 ::51 52 10 instructions in function53 54Enabling optimization remarks55=============================56 57There are two modes that are supported for enabling optimization remarks in58LLVM: through remark diagnostics, or through serialized remarks.59 60See also the clang flags61`-Rpass <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_62and63`-fsave-optimization-record <http://clang.llvm.org/docs/UsersManual.html#cmdoption-f-no-save-optimization-record>`_.64 65Remark diagnostics66------------------67 68Optimization remarks can be emitted as diagnostics. These diagnostics will be69propagated to front-ends if desired, or emitted by tools like :doc:`llc70<CommandGuide/llc>` or :doc:`opt <CommandGuide/opt>`.71 72.. option:: -pass-remarks=<regex>73 74 Enables optimization remarks from passes whose name match the given (POSIX)75 regular expression.76 77.. option:: -pass-remarks-missed=<regex>78 79 Enables missed optimization remarks from passes whose name match the given80 (POSIX) regular expression.81 82.. option:: -pass-remarks-analysis=<regex>83 84 Enables optimization analysis remarks from passes whose name match the given85 (POSIX) regular expression.86 87Serialized remarks88------------------89 90While diagnostics are useful during development, it is often more useful to91refer to optimization remarks post-compilation, typically during performance92analysis.93 94For that, LLVM can serialize the remarks produced for each compilation unit to95a file that can be consumed later.96 97By default, the format of the serialized remarks is :ref:`YAML98<yamlremarks>`, and it can be accompanied by a :ref:`section <remarkssection>`99in the object files to easily retrieve it.100 101:doc:`llc <CommandGuide/llc>` and :doc:`opt <CommandGuide/opt>` support the102following options:103 104 105``Basic options``106 107 .. option:: -pass-remarks-output=<filename>108 109 Enables the serialization of remarks to a file specified in <filename>.110 111 By default, the output is serialized to :ref:`YAML <yamlremarks>`.112 113 .. option:: -pass-remarks-format=<format>114 115 Specifies the output format of the serialized remarks.116 117 Supported formats:118 119 * :ref:`yaml <yamlremarks>` (default)120 * :ref:`bitstream <bitstreamremarks>`121 122``Content configuration``123 124 .. option:: -pass-remarks-filter=<regex>125 126 Only passes whose name match the given (POSIX) regular expression will be127 serialized to the final output.128 129 .. option:: -pass-remarks-with-hotness130 131 With PGO, include profile count in optimization remarks.132 133 .. option:: -pass-remarks-hotness-threshold134 135 The minimum profile count required for an optimization remark to be136 emitted.137 138Other tools that support remarks:139 140:program:`llvm-lto`141 142 .. option:: -lto-pass-remarks-output=<filename>143 .. option:: -lto-pass-remarks-filter=<regex>144 .. option:: -lto-pass-remarks-format=<format>145 .. option:: -lto-pass-remarks-with-hotness146 .. option:: -lto-pass-remarks-hotness-threshold147 148:program:`gold-plugin` and :program:`lld`149 150 .. option:: -opt-remarks-filename=<filename>151 .. option:: -opt-remarks-filter=<regex>152 .. option:: -opt-remarks-format=<format>153 .. option:: -opt-remarks-with-hotness154 155.. _yamlremarks:156 157YAML remarks158============159 160A typical remark serialized to YAML looks like this:161 162.. code-block:: yaml163 164 --- !<TYPE>165 Pass: <pass>166 Name: <name>167 DebugLoc: { File: <file>, Line: <line>, Column: <column> }168 Function: <function>169 Hotness: <hotness>170 Args:171 - <key>: <value>172 DebugLoc: { File: <arg-file>, Line: <arg-line>, Column: <arg-column> }173 174The following entries are mandatory:175 176* ``<TYPE>``: can be ``Passed``, ``Missed``, ``Analysis``,177 ``AnalysisFPCommute``, ``AnalysisAliasing``, ``Failure``.178* ``<pass>``: the name of the pass that emitted this remark.179* ``<name>``: the name of the remark coming from ``<pass>``.180* ``<function>``: the mangled name of the function.181 182If a ``DebugLoc`` entry is specified, the following fields are required:183 184* ``<file>``185* ``<line>``186* ``<column>``187 188If an ``arg`` entry is specified, the following fields are required:189 190* ``<key>``191* ``<value>``192 193If a ``DebugLoc`` entry is specified within an ``arg`` entry, the following194fields are required:195 196* ``<arg-file>``197* ``<arg-line>``198* ``<arg-column>``199 200.. _optviewer:201 202YAML metadata203-------------204 205The metadata used together with the YAML format is:206 207* a magic number: "REMARKS\\0"208* the version number: a little-endian uint64_t209* 8 zero bytes. This space was previously used to encode the size of a string210 table. String table support for YAML remarks has been removed, use the211 bitstream format instead.212 213Optional:214 215* the absolute file path to the serialized remark diagnostics: a216 null-terminated string.217 218When the metadata is serialized separately from the remarks, the file path219should be present and point to the file where the remarks are serialized to.220 221In case the metadata only acts as a header to the remarks, the file path can be222omitted.223 224.. _bitstreamremarks:225 226LLVM bitstream remarks227======================228 229This format is using :doc:`LLVM bitstream <BitCodeFormat>` to serialize remarks230and their associated metadata.231 232A bitstream remark stream can be identified by the magic number ``"RMRK"`` that233is placed at the very beginning.234 235The format for serializing remarks is composed of two different block types:236 237.. _bitstreamremarksmetablock:238 239META_BLOCK240----------241 242The block providing information about the rest of the content in the stream.243 244Exactly one block is expected. Having multiple metadata blocks is an error.245 246This block can contain the following records:247 248.. _bitstreamremarksrecordmetacontainerinfo:249 250``RECORD_META_CONTAINER_INFO``251 252 The container version and type.253 254 Version: u32255 256 Type: u2257 258.. _bitstreamremarksrecordmetaremarkversion:259 260``RECORD_META_REMARK_VERSION``261 262 The version of the remark entries. This can change independently from the263 container version.264 265 Version: u32266 267.. _bitstreamremarksrecordmetastrtab:268 269``RECORD_META_STRTAB``270 271 The string table used by the remark entries. The format of the string table272 is a sequence of strings separated by ``\0``.273 274.. _bitstreamremarksrecordmetaexternalfile:275 276``RECORD_META_EXTERNAL_FILE``277 278 The external remark file path that contains the remark blocks associated279 with this metadata. This is an absolute path.280 281.. _bitstreamremarksremarkblock:282 283REMARK_BLOCK284------------285 286The block describing a remark entry.287 2880 or more blocks per file are allowed. Each block will depend on the289:ref:`META_BLOCK <bitstreamremarksmetablock>` in order to be parsed correctly.290 291This block can contain the following records:292 293``RECORD_REMARK_HEADER``294 295 The header of the remark. This contains all the mandatory information about296 a remark.297 298 +---------------+---------------------------+299 | Type | u3 |300 +---------------+---------------------------+301 | Remark name | VBR6 (string table index) |302 +---------------+---------------------------+303 | Pass name | VBR6 (string table index) |304 +---------------+---------------------------+305 | Function name | VBR6 (string table index) |306 +---------------+---------------------------+307 308``RECORD_REMARK_DEBUG_LOC``309 310 The source location for the corresponding remark. This record is optional.311 312 +--------+---------------------------+313 | File | VBR7 (string table index) |314 +--------+---------------------------+315 | Line | u32 |316 +--------+---------------------------+317 | Column | u32 |318 +--------+---------------------------+319 320``RECORD_REMARK_HOTNESS``321 322 The hotness of the remark. This record is optional.323 324 +---------------+---------------------+325 | Hotness | VBR8 (string table index) |326 +---------------+---------------------+327 328``RECORD_REMARK_ARG_WITH_DEBUGLOC``329 330 A remark argument with an associated debug location.331 332 +--------+---------------------------+333 | Key | VBR7 (string table index) |334 +--------+---------------------------+335 | Value | VBR7 (string table index) |336 +--------+---------------------------+337 | File | VBR7 (string table index) |338 +--------+---------------------------+339 | Line | u32 |340 +--------+---------------------------+341 | Column | u32 |342 +--------+---------------------------+343 344``RECORD_REMARK_ARG_WITHOUT_DEBUGLOC``345 346 A remark argument with an associated debug location.347 348 +--------+---------------------------+349 | Key | VBR7 (string table index) |350 +--------+---------------------------+351 | Value | VBR7 (string table index) |352 +--------+---------------------------+353 354The remark container355--------------------356 357The bitstream remark container supports multiple types:358 359.. _bitstreamremarksfileexternal:360 361``RemarksFileExternal: a link to an external remarks file``362 363 This container type expects only a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:364 365 * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`366 * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`367 * :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`368 369 Typically, this is emitted in a section in the object files, allowing370 clients to retrieve remarks and their associated metadata directly from371 intermediate products.372 373 The container versions of the external separate container should match in order to374 have a well-formed file.375 376.. _bitstreamremarksfile:377 378``RemarksFile: a standalone remarks file``379 380 This container type expects a :ref:`META_BLOCK <bitstreamremarksmetablock>` containing only:381 382 * :ref:`RECORD_META_CONTAINER_INFO <bitstreamremarksrecordmetacontainerinfo>`383 * :ref:`RECORD_META_REMARK_VERSION <bitstreamremarksrecordmetaremarkversion>`384 385 Then, this container type expects 1 or more :ref:`REMARK_BLOCK <bitstreamremarksremarkblock>`.386 If no remarks are emitted, the meta blocks are also not emitted, so the file is empty.387 388 After the remark blocks, another :ref:`META_BLOCK <bitstreamremarksmetablock>` is emitted, containing:389 * :ref:`RECORD_META_STRTAB <bitstreamremarksrecordmetastrtab>`390 391 When the parser reads this container type, it jumps to the end of the file392 to read the string table before parsing the individual remarks.393 394 Standalone remarks files can be referenced by the395 :ref:`RECORD_META_EXTERNAL_FILE <bitstreamremarksrecordmetaexternalfile>`396 entry in the :ref:`RemarksFileExternal397 <bitstreamremarksfileexternal>` container.398 399.. FIXME: Add complete output of :program:`llvm-bcanalyzer` on the different container types (once format changes are completed)400 401opt-viewer402==========403 404The ``opt-viewer`` directory contains a collection of tools that visualize and405summarize serialized remarks.406 407The tools only support the ``yaml`` format.408 409.. _optviewerpy:410 411opt-viewer.py412-------------413 414Output a HTML page which gives visual feedback on compiler interactions with415your program.416 417 :Examples:418 419 ::420 421 $ opt-viewer.py my_yaml_file.opt.yaml422 423 ::424 425 $ opt-viewer.py my_build_dir/426 427 428opt-stats.py429------------430 431Output statistics about the optimization remarks in the input set.432 433 :Example:434 435 ::436 437 $ opt-stats.py my_yaml_file.opt.yaml438 439 Total number of remarks 3440 441 442 Top 10 remarks by pass:443 inline 33%444 asm-printer 33%445 prologepilog 33%446 447 Top 10 remarks:448 asm-printer/InstructionCount 33%449 inline/NoDefinition 33%450 prologepilog/StackSize 33%451 452opt-diff.py453-----------454 455Produce a new YAML file which contains all of the changes in optimizations456between two YAML files.457 458Typically, this tool should be used to do diffs between:459 460* new compiler + fixed source vs old compiler + fixed source461* fixed compiler + new source vs fixed compiler + old source462 463This diff file can be displayed using :ref:`opt-viewer.py <optviewerpy>`.464 465 :Example:466 467 ::468 469 $ opt-diff.py my_opt_yaml1.opt.yaml my_opt_yaml2.opt.yaml -o my_opt_diff.opt.yaml470 $ opt-viewer.py my_opt_diff.opt.yaml471 472.. _remarkssection:473 474Emitting remark diagnostics in the object file475==============================================476 477A section containing metadata on remark diagnostics will be emitted for the478following formats:479 480* ``bitstream``481 482This can be overridden by using the flag ``-remarks-section=<bool>``.483 484The section is named:485 486* ``__LLVM,__remarks`` (MachO)487 488C API489=====490 491LLVM provides a library that can be used to parse remarks through a shared492library named ``libRemarks``.493 494The typical usage through the C API is like the following:495 496.. code-block:: c497 498 LLVMRemarkParserRef Parser = LLVMRemarkParserCreateYAML(Buf, Size);499 LLVMRemarkEntryRef Remark = NULL;500 while ((Remark = LLVMRemarkParserGetNext(Parser))) {501 // use Remark502 LLVMRemarkEntryDispose(Remark); // Release memory.503 }504 bool HasError = LLVMRemarkParserHasError(Parser);505 LLVMRemarkParserDispose(Parser);506 507Remark streamers508================509 510The ``RemarkStreamer`` interface is used to unify the serialization511capabilities of remarks across all the components that can generate remarks.512 513All remark serialization should go through the main remark streamer, the514``llvm::remarks::RemarkStreamer`` set up in the ``LLVMContext``. The interface515takes remark objects converted to ``llvm::remarks::Remark``, and takes care of516serializing it to the requested format, using the requested type of metadata,517etc.518 519Typically, a specialized remark streamer will hold a reference to the one set520up in the ``LLVMContext``, and will operate on its own type of diagnostics.521 522For example, LLVM IR passes will emit ``llvm::DiagnosticInfoOptimization*``523that get converted to ``llvm::remarks::Remark`` objects. Then, clang could set524up its own specialized remark streamer that takes ``clang::Diagnostic``525objects. This can allow various components of the frontend to emit remarks526using the same techniques as the LLVM remarks.527 528This gives us the following advantages:529 530* Composition: during the compilation pipeline, multiple components can set up531 their specialized remark streamers that all emit remarks through the same532 main streamer.533* Re-using the remark infrastructure in ``lib/Remarks``.534* Using the same file and format for the remark emitters created throughout the535 compilation.536 537at the cost of an extra layer of abstraction.538 539.. FIXME: add documentation for llvm-opt-report.540.. FIXME: add documentation for Passes supporting optimization remarks541.. FIXME: add documentation for IR Passes542.. FIXME: add documentation for CodeGen Passes543