1209 lines · plain
1llvm-mca - LLVM Machine Code Analyzer2=====================================3 4.. program:: llvm-mca5 6SYNOPSIS7--------8 9:program:`llvm-mca` [*options*] [input]10 11DESCRIPTION12-----------13 14:program:`llvm-mca` is a performance analysis tool that uses information15available in LLVM (e.g. scheduling models) to statically measure the performance16of machine code in a specific CPU.17 18Performance is measured in terms of throughput as well as processor resource19consumption. The tool currently works for processors with a backend for which20there is a scheduling model available in LLVM.21 22The main goal of this tool is not just to predict the performance of the code23when run on the target, but also help with diagnosing potential performance24issues.25 26Given an assembly code sequence, :program:`llvm-mca` estimates the Instructions27Per Cycle (IPC), as well as hardware resource pressure. The analysis and28reporting style were inspired by the IACA tool from Intel.29 30For example, you can compile code with clang, output assembly, and pipe it31directly into :program:`llvm-mca` for analysis:32 33.. code-block:: bash34 35 $ clang foo.c -O2 --target=x86_64 -S -o - | llvm-mca -mcpu=btver236 37Or for Intel syntax:38 39.. code-block:: bash40 41 $ clang foo.c -O2 --target=x86_64 -masm=intel -S -o - | llvm-mca -mcpu=btver242 43(:program:`llvm-mca` detects Intel syntax by the presence of an `.intel_syntax`44directive at the beginning of the input. By default its output syntax matches45that of its input.)46 47Scheduling models are not just used to compute instruction latencies and48throughput, but also to understand what processor resources are available49and how to simulate them.50 51By design, the quality of the analysis conducted by :program:`llvm-mca` is52inevitably affected by the quality of the scheduling models in LLVM.53 54If you see that the performance report is not accurate for a processor,55please `file a bug <https://github.com/llvm/llvm-project/issues>`_56against the appropriate backend.57 58OPTIONS59-------60 61If ``input`` is "``-``" or omitted, :program:`llvm-mca` reads from standard62input. Otherwise, it will read from the specified filename.63 64If the :option:`-o` option is omitted, then :program:`llvm-mca` will send its output65to standard output if the input is from standard input. If the :option:`-o`66option specifies "``-``", then the output will also be sent to standard output.67 68 69.. option:: -help70 71 Print a summary of command line options.72 73.. option:: -o <filename>74 75 Use ``<filename>`` as the output filename. See the summary above for more76 details.77 78.. option:: -mtriple=<target triple>79 80 Specify a target triple string.81 82.. option:: -march=<arch>83 84 Specify the architecture for which to analyze the code. It defaults to the85 host default target.86 87.. option:: -mcpu=<cpuname>88 89 Specify the processor for which to analyze the code. By default, the cpu name90 is autodetected from the host.91 92.. option:: -output-asm-variant=<variant id>93 94 Specify the output assembly variant for the report generated by the tool.95 On x86, possible values are [0, 1]. A value of 0 (vic. 1) for this flag enables96 the AT&T (vic. Intel) assembly format for the code printed out by the tool in97 the analysis report.98 99.. option:: -print-imm-hex100 101 Prefer hex format for numeric literals in the output assembly printed as part102 of the report.103 104.. option:: -dispatch=<width>105 106 Specify a different dispatch width for the processor. The dispatch width107 defaults to field 'IssueWidth' in the processor scheduling model. If width is108 zero, then the default dispatch width is used.109 110.. option:: -register-file-size=<size>111 112 Specify the size of the register file. When specified, this flag limits how113 many physical registers are available for register renaming purposes. A value114 of zero for this flag means "unlimited number of physical registers".115 116.. option:: -iterations=<number of iterations>117 118 Specify the number of iterations to run. If this flag is set to 0, then the119 tool sets the number of iterations to a default value (i.e. 100).120 121.. option:: -noalias=<bool>122 123 If set, the tool assumes that loads and stores don't alias. This is the124 default behavior.125 126.. option:: -lqueue=<load queue size>127 128 Specify the size of the load queue in the load/store unit emulated by the tool.129 By default, the tool assumes an unbound number of entries in the load queue.130 A value of zero for this flag is ignored, and the default load queue size is131 used instead.132 133.. option:: -squeue=<store queue size>134 135 Specify the size of the store queue in the load/store unit emulated by the136 tool. By default, the tool assumes an unbound number of entries in the store137 queue. A value of zero for this flag is ignored, and the default store queue138 size is used instead.139 140.. option:: -timeline141 142 Enable the timeline view.143 144.. option:: -timeline-max-iterations=<iterations>145 146 Limit the number of iterations to print in the timeline view. By default, the147 timeline view prints information for up to 10 iterations.148 149.. option:: -timeline-max-cycles=<cycles>150 151 Limit the number of cycles in the timeline view, or use 0 for no limit. By152 default, the number of cycles is set to 80.153 154.. option:: -resource-pressure155 156 Enable the resource pressure view. This is enabled by default.157 158.. option:: -register-file-stats159 160 Enable register file usage statistics.161 162.. option:: -dispatch-stats163 164 Enable extra dispatch statistics. This view collects and analyzes instruction165 dispatch events, as well as static/dynamic dispatch stall events. This view166 is disabled by default.167 168.. option:: -scheduler-stats169 170 Enable extra scheduler statistics. This view collects and analyzes instruction171 issue events. This view is disabled by default.172 173.. option:: -retire-stats174 175 Enable extra retire control unit statistics. This view is disabled by default.176 177.. option:: -instruction-info178 179 Enable the instruction info view. This is enabled by default.180 181.. option:: -show-encoding182 183 Enable the printing of instruction encodings within the instruction info view.184 185.. option:: -show-barriers186 187 Enable the printing of LoadBarrier and StoreBarrier flags within the188 instruction info view.189 190.. option:: -all-stats191 192 Print all hardware statistics. This enables extra statistics related to the193 dispatch logic, the hardware schedulers, the register file(s), and the retire194 control unit. This option is disabled by default.195 196.. option:: -all-views197 198 Enable all the view.199 200.. option:: -instruction-tables=<level>201 202 Prints resource pressure information based on the static information203 available from the processor model. This differs from the resource pressure204 view because it doesn't require that the code is simulated. It instead prints205 the theoretical uniform distribution of resource pressure for every206 instruction in sequence.207 208 The choice of `<level>` controls number of printed information.209 `<level>` may be `none` (default), `normal`, `full`.210 Note: If the option is used without `<label>`, default is `normal` (legacy).211 212 When `<level>` is `full`, additional information are:213 - `<Bypass Latency>`: Latency when a bypass is implemented between operands214 in pipelines (see SchedReadAdvance).215 - `<LLVM Opcode Name>`: mnemonic plus operands identifier.216 - `<Resources units>`: Used resources associated with LLVM Opcode.217 - `<instruction comment>`: reports comment if any from source assembly.218 219 `<Resources units>` syntax can be:220 - <Resource Name>: ReleaseAtCycle is 1.221 - <Resource Name>[<ReleaseAtCycle>]: ReleaseAtCycle is greater than 1222 and AcquireAtCycle is 0.223 - <Resource Name>[<AcquireAtCycle>,<ReleaseAtCycle>]: ReleaseAtCycle224 is greater than 1 and AcquireAtCycle is greater than 0.225 226.. option:: -bottleneck-analysis227 228 Print information about bottlenecks that affect the throughput. This analysis229 can be expensive, and it is disabled by default. Bottlenecks are highlighted230 in the summary view. Bottleneck analysis is currently not supported for231 processors with an in-order backend.232 233.. option:: -json234 235 Print the requested views in valid JSON format. The instructions and the236 processor resources are printed as members of special top level JSON objects.237 The individual views refer to them by index. However, not all views are238 currently supported. For example, the report from the bottleneck analysis is239 not printed out in JSON. All the default views are currently supported.240 241.. option:: -disable-cb242 243 Force usage of the generic CustomBehaviour and InstrPostProcess classes rather244 than using the target-specific implementation. The generic classes never245 detect any custom hazards or make any post processing modifications to246 instructions.247 248.. option:: -disable-im249 250 Force usage of the generic InstrumentManager rather than using the target251 specific implementation. The generic class creates Instruments that provide252 no extra information, and InstrumentManager never overrides the default253 schedule class for a given instruction.254 255.. option:: -skip-unsupported-instructions=<reason>256 257 Force :program:`llvm-mca` to continue in the presence of instructions which do258 not parse or lack key scheduling information. Note that the resulting analysis259 is impacted since those unsupported instructions are ignored as-if they are260 not supplied as a part of the input.261 262 The choice of `<reason>` controls the when mca will report an error.263 `<reason>` may be `none` (default), `lack-sched`, `parse-failure`, `any`.264 265EXIT STATUS266-----------267 268:program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed269to standard error, and the tool returns 1.270 271USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS272---------------------------------------------273:program:`llvm-mca` allows for the optional usage of special code comments to274mark regions of the assembly code to be analyzed. A comment starting with275substring ``LLVM-MCA-BEGIN`` marks the beginning of an analysis region. A276comment starting with substring ``LLVM-MCA-END`` marks the end of a region.277For example:278 279.. code-block:: none280 281 # LLVM-MCA-BEGIN282 ...283 # LLVM-MCA-END284 285If no user-defined region is specified, then :program:`llvm-mca` assumes a286default region which contains every instruction in the input file. Every region287is analyzed in isolation, and the final performance report is the union of all288the reports generated for every analysis region.289 290Analysis regions can have names. For example:291 292.. code-block:: none293 294 # LLVM-MCA-BEGIN A simple example295 add %eax, %eax296 # LLVM-MCA-END297 298The code from the example above defines a region named "A simple example" with a299single instruction in it. Note how the region name doesn't have to be repeated300in the ``LLVM-MCA-END`` directive. In the absence of overlapping regions,301an anonymous ``LLVM-MCA-END`` directive always ends the currently active user302defined region.303 304Example of nesting regions:305 306.. code-block:: none307 308 # LLVM-MCA-BEGIN foo309 add %eax, %edx310 # LLVM-MCA-BEGIN bar311 sub %eax, %edx312 # LLVM-MCA-END bar313 # LLVM-MCA-END foo314 315Example of overlapping regions:316 317.. code-block:: none318 319 # LLVM-MCA-BEGIN foo320 add %eax, %edx321 # LLVM-MCA-BEGIN bar322 sub %eax, %edx323 # LLVM-MCA-END foo324 add %eax, %edx325 # LLVM-MCA-END bar326 327Note that multiple anonymous regions cannot overlap. Also, overlapping regions328cannot have the same name.329 330There is no support for marking regions from high-level source code, like C or331C++. As a workaround, inline assembly directives may be used:332 333.. code-block:: c++334 335 int foo(int a, int b) {336 __asm volatile("# LLVM-MCA-BEGIN foo":::"memory");337 a += 42;338 __asm volatile("# LLVM-MCA-END":::"memory");339 a *= b;340 return a;341 }342 343However, this interferes with optimizations like loop vectorization and may have344an impact on the code generated. This is because the ``__asm`` statements are345seen as real code having important side effects, which limits how the code346around them can be transformed. If users want to make use of inline assembly347to emit markers, then the recommendation is to always verify that the output348assembly is equivalent to the assembly generated in the absence of markers.349The `Clang options to emit optimization reports <https://clang.llvm.org/docs/UsersManual.html#options-to-emit-optimization-reports>`_350can also help in detecting missed optimizations.351 352INSTRUMENT REGIONS353------------------354 355An InstrumentRegion describes a region of assembly code guarded by356special LLVM-MCA comment directives.357 358.. code-block:: none359 360 # LLVM-MCA-<INSTRUMENT_TYPE> <data>361 ... ## asm362 363where `INSTRUMENT_TYPE` is a type defined by the target and expects364to use `data`.365 366A comment starting with substring `LLVM-MCA-<INSTRUMENT_TYPE>`367brings data into scope for llvm-mca to use in its analysis for368all following instructions.369 370If a comment with the same `INSTRUMENT_TYPE` is found later in the371instruction list, then the original InstrumentRegion will be372automatically ended, and a new InstrumentRegion will begin.373 374If there are comments containing the different `INSTRUMENT_TYPE`,375then both data sets remain available. In contrast with an AnalysisRegion,376an InstrumentRegion does not need a comment to end the region.377 378Comments that are prefixed with `LLVM-MCA-` but do not correspond to379a valid `INSTRUMENT_TYPE` for the target cause an error, except for380`BEGIN` and `END`, since those correspond to AnalysisRegions. Comments381that do not start with `LLVM-MCA-` are ignored by :program:`llvm-mca`.382 383An instruction (a MCInst) is added to an InstrumentRegion R only384if its location is in range [R.RangeStart, R.RangeEnd].385 386There is one instrument that can be used on all targets to explicitly387set instruction latencies. It can be used, for example, to model the388cache misses that impact load latencies. The syntax is like389 390.. code-block:: none391 392 # LLVM-MCA-LATENCY 100393 mov (%edi), %eax394 # LLVM-MCA-LATENCY395 396It sets the latency of mov instruction to 100. LLVM-MCA-LATENCY without397argument ends the region with explicit latency, after it default target398latencies are used.399 400On RISCV targets, vector instructions have different behaviour depending401on the LMUL. Code can be instrumented with a comment that takes the402following form:403 404.. code-block:: none405 406 # LLVM-MCA-RISCV-LMUL <M1|M2|M4|M8|MF2|MF4|MF8>407 408The RISCV InstrumentManager will override the schedule class for vector409instructions to use the scheduling behaviour of its pseudo-instruction410which is LMUL dependent. It makes sense to place RISCV instrument411comments directly after `vset{i}vl{i}` instructions, although412they can be placed anywhere in the program.413 414Example of program with no call to `vset{i}vl{i}`:415 416.. code-block:: none417 418 # LLVM-MCA-RISCV-LMUL M2419 vadd.vv v2, v2, v2420 421Example of program with call to `vset{i}vl{i}`:422 423.. code-block:: none424 425 vsetvli zero, a0, e8, m1, tu, mu426 # LLVM-MCA-RISCV-LMUL M1427 vadd.vv v2, v2, v2428 429Example of program with multiple calls to `vset{i}vl{i}`:430 431.. code-block:: none432 433 vsetvli zero, a0, e8, m1, tu, mu434 # LLVM-MCA-RISCV-LMUL M1435 vadd.vv v2, v2, v2436 vsetvli zero, a0, e8, m8, tu, mu437 # LLVM-MCA-RISCV-LMUL M8438 vadd.vv v2, v2, v2439 440Example of program with call to `vsetvl`:441 442.. code-block:: none443 444 vsetvl rd, rs1, rs2445 # LLVM-MCA-RISCV-LMUL M1446 vadd.vv v12, v12, v12447 vsetvl rd, rs1, rs2448 # LLVM-MCA-RISCV-LMUL M4449 vadd.vv v12, v12, v12450 451HOW LLVM-MCA WORKS452------------------453 454:program:`llvm-mca` takes assembly code as input. The assembly code is parsed455into a sequence of MCInst with the help of the existing LLVM target assembly456parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module457to generate a performance report.458 459The Pipeline module simulates the execution of the machine code sequence in a460loop of iterations (default is 100). During this process, the pipeline collects461a number of execution related statistics. At the end of this process, the462pipeline generates and prints a report from the collected statistics.463 464Here is an example of a performance report generated by the tool for a465dot-product of two packed float vectors of four elements. The analysis is466conducted for target x86, cpu btver2. The following result can be produced via467the following command using the example located at468``test/tools/llvm-mca/X86/BtVer2/dot-product.s``:469 470.. code-block:: bash471 472 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s473 474.. code-block:: none475 476 Iterations: 300477 Instructions: 900478 Total Cycles: 610479 Total uOps: 900480 481 Dispatch Width: 2482 uOps Per Cycle: 1.48483 IPC: 1.48484 Block RThroughput: 2.0485 486 487 Instruction Info:488 [1]: #uOps489 [2]: Latency490 [3]: RThroughput491 [4]: MayLoad492 [5]: MayStore493 [6]: HasSideEffects (U)494 495 [1] [2] [3] [4] [5] [6] Instructions:496 1 2 1.00 vmulps %xmm0, %xmm1, %xmm2497 1 3 1.00 vhaddps %xmm2, %xmm2, %xmm3498 1 3 1.00 vhaddps %xmm3, %xmm3, %xmm4499 500 501 Resources:502 [0] - JALU0503 [1] - JALU1504 [2] - JDiv505 [3] - JFPA506 [4] - JFPM507 [5] - JFPU0508 [6] - JFPU1509 [7] - JLAGU510 [8] - JMul511 [9] - JSAGU512 [10] - JSTC513 [11] - JVALU0514 [12] - JVALU1515 [13] - JVIMUL516 517 518 Resource pressure per iteration:519 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]520 - - - 2.00 1.00 2.00 1.00 - - - - - - -521 522 Resource pressure by instruction:523 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Instructions:524 - - - - 1.00 - 1.00 - - - - - - - vmulps %xmm0, %xmm1, %xmm2525 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm2, %xmm2, %xmm3526 - - - 1.00 - 1.00 - - - - - - - - vhaddps %xmm3, %xmm3, %xmm4527 528According to this report, the dot-product kernel has been executed 300 times,529for a total of 900 simulated instructions. The total number of simulated micro530opcodes (uOps) is also 900.531 532The report is structured in three main sections. The first section collects a533few performance numbers; the goal of this section is to give a very quick534overview of the performance throughput. Important performance indicators are535**IPC**, **uOps Per Cycle**, and **Block RThroughput** (Block Reciprocal536Throughput).537 538Field *DispatchWidth* is the maximum number of micro opcodes that are dispatched539to the out-of-order backend every simulated cycle. For processors with an540in-order backend, *DispatchWidth* is the maximum number of micro opcodes issued541to the backend every simulated cycle.542 543IPC is computed dividing the total number of simulated instructions by the total544number of cycles.545 546Field *Block RThroughput* is the reciprocal of the block throughput. Block547throughput is a theoretical quantity computed as the maximum number of blocks548(i.e. iterations) that can be executed per simulated clock cycle in the absence549of loop carried dependencies. Block throughput is superiorly limited by the550dispatch rate, and the availability of hardware resources.551 552In the absence of loop-carried data dependencies, the observed IPC tends to a553theoretical maximum which can be computed by dividing the number of instructions554of a single iteration by the `Block RThroughput`.555 556Field 'uOps Per Cycle' is computed dividing the total number of simulated micro557opcodes by the total number of cycles. A delta between Dispatch Width and this558field is an indicator of a performance issue. In the absence of loop-carried559data dependencies, the observed 'uOps Per Cycle' should tend to a theoretical560maximum throughput which can be computed by dividing the number of uOps of a561single iteration by the `Block RThroughput`.562 563Field *uOps Per Cycle* is bounded from above by the dispatch width. That is564because the dispatch width limits the maximum size of a dispatch group. Both IPC565and 'uOps Per Cycle' are limited by the amount of hardware parallelism. The566availability of hardware resources affects the resource pressure distribution,567and it limits the number of instructions that can be executed in parallel every568cycle. A delta between Dispatch Width and the theoretical maximum uOps per569Cycle (computed by dividing the number of uOps of a single iteration by the570`Block RThroughput`) is an indicator of a performance bottleneck caused by the571lack of hardware resources.572In general, the lower the Block RThroughput, the better.573 574In this example, ``uOps per iteration/Block RThroughput`` is 1.50. Since there575are no loop-carried dependencies, the observed `uOps Per Cycle` is expected to576approach 1.50 when the number of iterations tends to infinity. The delta between577the Dispatch Width (2.00), and the theoretical maximum uOp throughput (1.50) is578an indicator of a performance bottleneck caused by the lack of hardware579resources, and the *Resource pressure view* can help to identify the problematic580resource usage.581 582The second section of the report is the `instruction info view`. It shows the583latency and reciprocal throughput of every instruction in the sequence. It also584reports extra information related to the number of micro opcodes, and opcode585properties (i.e., 'MayLoad', 'MayStore', and 'HasSideEffects').586 587Field *RThroughput* is the reciprocal of the instruction throughput. Throughput588is computed as the maximum number of instructions of a same type that can be589executed per clock cycle in the absence of operand dependencies. In this590example, the reciprocal throughput of a vector float multiply is 1591cycles/instruction. That is because the FP multiplier JFPM is only available592from pipeline JFPU1.593 594Instruction encodings are displayed within the instruction info view when flag595`-show-encoding` is specified.596 597Below is an example of `-show-encoding` output for the dot-product kernel:598 599.. code-block:: none600 601 Instruction Info:602 [1]: #uOps603 [2]: Latency604 [3]: RThroughput605 [4]: MayLoad606 [5]: MayStore607 [6]: HasSideEffects (U)608 [7]: Encoding Size609 610 [1] [2] [3] [4] [5] [6] [7] Encodings: Instructions:611 1 2 1.00 4 c5 f0 59 d0 vmulps %xmm0, %xmm1, %xmm2612 1 4 1.00 4 c5 eb 7c da vhaddps %xmm2, %xmm2, %xmm3613 1 4 1.00 4 c5 e3 7c e3 vhaddps %xmm3, %xmm3, %xmm4614 615The `Encoding Size` column shows the size in bytes of instructions. The616`Encodings` column shows the actual instruction encodings (byte sequences in617hex).618 619The third section is the *Resource pressure view*. This view reports620the average number of resource cycles consumed every iteration by instructions621for every processor resource unit available on the target. Information is622structured in two tables. The first table reports the number of resource cycles623spent on average every iteration. The second table correlates the resource624cycles to the machine instruction in the sequence. For example, every iteration625of the instruction vmulps always executes on resource unit [6]626(JFPU1 - floating point pipeline #1), consuming an average of 1 resource cycle627per iteration. Note that on AMD Jaguar, vector floating-point multiply can628only be issued to pipeline JFPU1, while horizontal floating-point additions can629only be issued to pipeline JFPU0.630 631The resource pressure view helps with identifying bottlenecks caused by high632usage of specific hardware resources. Situations with resource pressure mainly633concentrated on a few resources should, in general, be avoided. Ideally,634pressure should be uniformly distributed between multiple resources.635 636Timeline View637^^^^^^^^^^^^^638The timeline view produces a detailed report of each instruction's state639transitions through an instruction pipeline. This view is enabled by the640command line option ``-timeline``. As instructions transition through the641various stages of the pipeline, their states are depicted in the view report.642These states are represented by the following characters:643 644* D : Instruction dispatched.645* e : Instruction executing.646* E : Instruction executed.647* R : Instruction retired.648* = : Instruction already dispatched, waiting to be executed.649* \- : Instruction executed, waiting to be retired.650 651Below is the timeline view for a subset of the dot-product example located in652``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by653:program:`llvm-mca` using the following command:654 655.. code-block:: bash656 657 $ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s658 659.. code-block:: none660 661 Timeline view:662 012345663 Index 0123456789664 665 [0,0] DeeER. . . vmulps %xmm0, %xmm1, %xmm2666 [0,1] D==eeeER . . vhaddps %xmm2, %xmm2, %xmm3667 [0,2] .D====eeeER . vhaddps %xmm3, %xmm3, %xmm4668 [1,0] .DeeE-----R . vmulps %xmm0, %xmm1, %xmm2669 [1,1] . D=eeeE---R . vhaddps %xmm2, %xmm2, %xmm3670 [1,2] . D====eeeER . vhaddps %xmm3, %xmm3, %xmm4671 [2,0] . DeeE-----R . vmulps %xmm0, %xmm1, %xmm2672 [2,1] . D====eeeER . vhaddps %xmm2, %xmm2, %xmm3673 [2,2] . D======eeeER vhaddps %xmm3, %xmm3, %xmm4674 675 676 Average Wait times (based on the timeline view):677 [0]: Executions678 [1]: Average time spent waiting in a scheduler's queue679 [2]: Average time spent waiting in a scheduler's queue while ready680 [3]: Average time elapsed from WB until retire stage681 682 [0] [1] [2] [3]683 0. 3 1.0 1.0 3.3 vmulps %xmm0, %xmm1, %xmm2684 1. 3 3.3 0.7 1.0 vhaddps %xmm2, %xmm2, %xmm3685 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4686 3 3.3 0.5 1.4 <total>687 688The timeline view is interesting because it shows instruction state changes689during execution. It also gives an idea of how the tool processes instructions690executed on the target, and how their timing information might be calculated.691 692The timeline view is structured in two tables. The first table shows693instructions changing state over time (measured in cycles); the second table694(named *Average Wait times*) reports useful timing statistics, which should695help diagnose performance bottlenecks caused by long data dependencies and696sub-optimal usage of hardware resources.697 698An instruction in the timeline view is identified by a pair of indices, where699the first index identifies an iteration, and the second index is the700instruction index (i.e., where it appears in the code sequence). Since this701example was generated using 3 iterations: ``-iterations=3``, the iteration702indices range from 0-2 inclusively.703 704Excluding the first and last column, the remaining columns are in cycles.705Cycles are numbered sequentially starting from 0.706 707From the example output above, we know the following:708 709* Instruction [1,0] was dispatched at cycle 1.710* Instruction [1,0] started executing at cycle 2.711* Instruction [1,0] reached the write back stage at cycle 4.712* Instruction [1,0] was retired at cycle 10.713 714Instruction [1,0] (i.e., vmulps from iteration #1) does not have to wait in the715scheduler's queue for the operands to become available. By the time vmulps is716dispatched, operands are already available, and pipeline JFPU1 is ready to717serve another instruction. So the instruction can be immediately issued on the718JFPU1 pipeline. That is demonstrated by the fact that the instruction only719spent 1cy in the scheduler's queue.720 721There is a gap of 5 cycles between the write-back stage and the retire event.722That is because instructions must retire in program order, so [1,0] has to wait723for [0,2] to be retired first (i.e., it has to wait until cycle 10).724 725In the example, all instructions are in a RAW (Read After Write) dependency726chain. Register %xmm2 written by vmulps is immediately used by the first727vhaddps, and register %xmm3 written by the first vhaddps is used by the second728vhaddps. Long data dependencies negatively impact the ILP (Instruction Level729Parallelism).730 731In the dot-product example, there are anti-dependencies introduced by732instructions from different iterations. However, those dependencies can be733removed at register renaming stage (at the cost of allocating register aliases,734and therefore consuming physical registers).735 736Table *Average Wait times* helps diagnose performance issues that are caused by737the presence of long latency instructions and potentially long data dependencies738which may limit the ILP. Last row, ``<total>``, shows a global average over all739instructions measured. Note that :program:`llvm-mca`, by default, assumes at740least 1cy between the dispatch event and the issue event.741 742When the performance is limited by data dependencies and/or long latency743instructions, the number of cycles spent while in the *ready* state is expected744to be very small when compared with the total number of cycles spent in the745scheduler's queue. The difference between the two counters is a good indicator746of how large of an impact data dependencies had on the execution of the747instructions. When performance is mostly limited by the lack of hardware748resources, the delta between the two counters is small. However, the number of749cycles spent in the queue tends to be larger (i.e., more than 1-3cy),750especially when compared to other low latency instructions.751 752Bottleneck Analysis753^^^^^^^^^^^^^^^^^^^754The ``-bottleneck-analysis`` command line option enables the analysis of755performance bottlenecks.756 757This analysis is potentially expensive. It attempts to correlate increases in758backend pressure (caused by pipeline resource pressure and data dependencies) to759dynamic dispatch stalls.760 761Below is an example of ``-bottleneck-analysis`` output generated by762:program:`llvm-mca` for 500 iterations of the dot-product example on btver2.763 764.. code-block:: none765 766 767 Cycles with backend pressure increase [ 48.07% ]768 Throughput Bottlenecks:769 Resource Pressure [ 47.77% ]770 - JFPA [ 47.77% ]771 - JFPU0 [ 47.77% ]772 Data Dependencies: [ 0.30% ]773 - Register Dependencies [ 0.30% ]774 - Memory Dependencies [ 0.00% ]775 776 Critical sequence based on the simulation:777 778 Instruction Dependency Information779 +----< 2. vhaddps %xmm3, %xmm3, %xmm4780 |781 | < loop carried >782 |783 | 0. vmulps %xmm0, %xmm1, %xmm2784 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ]785 +----> 2. vhaddps %xmm3, %xmm3, %xmm4 ## REGISTER dependency: %xmm3786 |787 | < loop carried >788 |789 +----> 1. vhaddps %xmm2, %xmm2, %xmm3 ## RESOURCE interference: JFPA [ probability: 74% ]790 791 792According to the analysis, throughput is limited by resource pressure and not by793data dependencies. The analysis observed increases in backend pressure during79448.07% of the simulated run. Almost all those pressure increase events were795caused by contention on processor resources JFPA/JFPU0.796 797The `critical sequence` is the most expensive sequence of instructions according798to the simulation. It is annotated to provide extra information about critical799register dependencies and resource interferences between instructions.800 801Instructions from the critical sequence are expected to significantly impact802performance. By construction, the accuracy of this analysis is strongly803dependent on the simulation and (as always) by the quality of the processor804model in llvm.805 806Bottleneck analysis is currently not supported for processors with an in-order807backend.808 809Extra Statistics to Further Diagnose Performance Issues810^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^811The ``-all-stats`` command line option enables extra statistics and performance812counters for the dispatch logic, the reorder buffer, the retire control unit,813and the register file.814 815Below is an example of ``-all-stats`` output generated by :program:`llvm-mca`816for 300 iterations of the dot-product example discussed in the previous817sections.818 819.. code-block:: none820 821 Dynamic Dispatch Stall Cycles:822 RAT - Register unavailable: 0823 RCU - Retire tokens unavailable: 0824 SCHEDQ - Scheduler full: 272 (44.6%)825 LQ - Load queue full: 0826 SQ - Store queue full: 0827 GROUP - Static restrictions on the dispatch group: 0828 829 830 Dispatch Logic - number of cycles where we saw N micro opcodes dispatched:831 [# dispatched], [# cycles]832 0, 24 (3.9%)833 1, 272 (44.6%)834 2, 314 (51.5%)835 836 837 Schedulers - number of cycles where we saw N micro opcodes issued:838 [# issued], [# cycles]839 0, 7 (1.1%)840 1, 306 (50.2%)841 2, 297 (48.7%)842 843 Scheduler's queue usage:844 [1] Resource name.845 [2] Average number of used buffer entries.846 [3] Maximum number of used buffer entries.847 [4] Total number of buffer entries.848 849 [1] [2] [3] [4]850 JALU01 0 0 20851 JFPU01 17 18 18852 JLSAGU 0 0 12853 854 855 Retire Control Unit - number of cycles where we saw N instructions retired:856 [# retired], [# cycles]857 0, 109 (17.9%)858 1, 102 (16.7%)859 2, 399 (65.4%)860 861 Total ROB Entries: 64862 Max Used ROB Entries: 35 ( 54.7% )863 Average Used ROB Entries per cy: 32 ( 50.0% )864 865 866 Register File statistics:867 Total number of mappings created: 900868 Max number of mappings used: 35869 870 * Register File #1 -- JFpuPRF:871 Number of physical registers: 72872 Total number of mappings created: 900873 Max number of mappings used: 35874 875 * Register File #2 -- JIntegerPRF:876 Number of physical registers: 64877 Total number of mappings created: 0878 Max number of mappings used: 0879 880If we look at the *Dynamic Dispatch Stall Cycles* table, we see the counter for881SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch882logic is unable to dispatch a full group because the scheduler's queue is full.883 884Looking at the *Dispatch Logic* table, we see that the pipeline was only able to885dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to886one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The887dispatch statistics are displayed by either using the command option888``-all-stats`` or ``-dispatch-stats``.889 890The next table, *Schedulers*, presents a histogram displaying a count,891representing the number of micro opcodes issued on some number of cycles. In892this case, of the 610 simulated cycles, single opcodes were issued 306 times893(50.2%) and there were 7 cycles where no opcodes were issued.894 895The *Scheduler's queue usage* table shows that the average and maximum number of896buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01897reached its maximum (18 of 18 queue entries). Note that AMD Jaguar implements898three schedulers:899 900* JALU01 - A scheduler for ALU instructions.901* JFPU01 - A scheduler floating point operations.902* JLSAGU - A scheduler for address generation.903 904The dot-product is a kernel of three floating point instructions (a vector905multiply followed by two horizontal adds). That explains why only the floating906point scheduler appears to be used.907 908A full scheduler queue is either caused by data dependency chains or by a909sub-optimal usage of hardware resources. Sometimes, resource pressure can be910mitigated by rewriting the kernel using different instructions that consume911different scheduler resources. Schedulers with a small queue are less resilient912to bottlenecks caused by the presence of long data dependencies. The scheduler913statistics are displayed by using the command option ``-all-stats`` or914``-scheduler-stats``.915 916The next table, *Retire Control Unit*, presents a histogram displaying a count,917representing the number of instructions retired on some number of cycles. In918this case, of the 610 simulated cycles, two instructions were retired during the919same cycle 399 times (65.4%) and there were 109 cycles where no instructions920were retired. The retire statistics are displayed by using the command option921``-all-stats`` or ``-retire-stats``.922 923The last table presented is *Register File statistics*. Each physical register924file (PRF) used by the pipeline is presented in this table. In the case of AMD925Jaguar, there are two register files, one for floating-point registers (JFpuPRF)926and one for integer registers (JIntegerPRF). The table shows that of the 900927instructions processed, there were 900 mappings created. Since this dot-product928example utilized only floating point registers, the JFPuPRF was responsible for929creating the 900 mappings. However, we see that the pipeline only used a930maximum of 35 of 72 available register slots at any given time. We can conclude931that the floating point PRF was the only register file used for the example, and932that it was never resource constrained. The register file statistics are933displayed by using the command option ``-all-stats`` or934``-register-file-stats``.935 936In this example, we can conclude that the IPC is mostly limited by data937dependencies, and not by resource pressure.938 939Instruction Flow940^^^^^^^^^^^^^^^^941This section describes the instruction flow through the default pipeline of942:program:`llvm-mca`, as well as the functional units involved in the process.943 944The default pipeline implements the following sequence of stages used to945process instructions.946 947* Dispatch (Instruction is dispatched to the schedulers).948* Issue (Instruction is issued to the processor pipelines).949* Write Back (Instruction is executed, and results are written back).950* Retire (Instruction is retired; writes are architecturally committed).951 952The in-order pipeline implements the following sequence of stages:953 954* InOrderIssue (Instruction is issued to the processor pipelines).955* Retire (Instruction is retired; writes are architecturally committed).956 957:program:`llvm-mca` assumes that instructions have all been decoded and placed958into a queue before the simulation start. Therefore, the instruction fetch and959decode stages are not modeled. Performance bottlenecks in the frontend are not960diagnosed. Also, :program:`llvm-mca` does not model branch prediction.961 962Instruction Dispatch963""""""""""""""""""""964During the dispatch stage, instructions are picked in program order from a965queue of already decoded instructions, and dispatched in groups to the966simulated hardware schedulers.967 968The size of a dispatch group depends on the availability of the simulated969hardware resources. The processor dispatch width defaults to the value970of the ``IssueWidth`` in LLVM's scheduling model.971 972An instruction can be dispatched if:973 974* The size of the dispatch group is smaller than processor's dispatch width.975* There are enough entries in the reorder buffer.976* There are enough physical registers to do register renaming.977* The schedulers are not full.978 979Scheduling models can optionally specify which register files are available on980the processor. :program:`llvm-mca` uses that information to initialize register981file descriptors. Users can limit the number of physical registers that are982globally available for register renaming by using the command option983``-register-file-size``. A value of zero for this option means *unbounded*. By984knowing how many registers are available for renaming, the tool can predict985dispatch stalls caused by the lack of physical registers.986 987The number of reorder buffer entries consumed by an instruction depends on the988number of micro-opcodes specified for that instruction by the target scheduling989model. The reorder buffer is responsible for tracking the progress of990instructions that are "in-flight", and retiring them in program order. The991number of entries in the reorder buffer defaults to the value specified by field992`MicroOpBufferSize` in the target scheduling model.993 994Instructions that are dispatched to the schedulers consume scheduler buffer995entries. :program:`llvm-mca` queries the scheduling model to determine the set996of buffered resources consumed by an instruction. Buffered resources are997treated like scheduler resources.998 999Instruction Issue1000"""""""""""""""""1001Each processor scheduler implements a buffer of instructions. An instruction1002has to wait in the scheduler's buffer until input register operands become1003available. Only at that point, does the instruction becomes eligible for1004execution and may be issued (potentially out-of-order) for execution.1005Instruction latencies are computed by :program:`llvm-mca` with the help of the1006scheduling model.1007 1008:program:`llvm-mca`'s scheduler is designed to simulate multiple processor1009schedulers. The scheduler is responsible for tracking data dependencies, and1010dynamically selecting which processor resources are consumed by instructions.1011It delegates the management of processor resource units and resource groups to a1012resource manager. The resource manager is responsible for selecting resource1013units that are consumed by instructions. For example, if an instruction1014consumes 1cy of a resource group, the resource manager selects one of the1015available units from the group; by default, the resource manager uses a1016round-robin selector to guarantee that resource usage is uniformly distributed1017between all units of a group.1018 1019:program:`llvm-mca`'s scheduler internally groups instructions into three sets:1020 1021* WaitSet: a set of instructions whose operands are not ready.1022* ReadySet: a set of instructions ready to execute.1023* IssuedSet: a set of instructions executing.1024 1025Depending on the operands availability, instructions that are dispatched to the1026scheduler are either placed into the WaitSet or into the ReadySet.1027 1028Every cycle, the scheduler checks if instructions can be moved from the WaitSet1029to the ReadySet, and if instructions from the ReadySet can be issued to the1030underlying pipelines. The algorithm prioritizes older instructions over younger1031instructions.1032 1033Write-Back and Retire Stage1034"""""""""""""""""""""""""""1035Issued instructions are moved from the ReadySet to the IssuedSet. There,1036instructions wait until they reach the write-back stage. At that point, they1037get removed from the queue and the retire control unit is notified.1038 1039When instructions are executed, the retire control unit flags the instruction as1040"ready to retire."1041 1042Instructions are retired in program order. The register file is notified of the1043retirement so that it can free the physical registers that were allocated for1044the instruction during the register renaming stage.1045 1046Load/Store Unit and Memory Consistency Model1047""""""""""""""""""""""""""""""""""""""""""""1048To simulate an out-of-order execution of memory operations, :program:`llvm-mca`1049utilizes a simulated load/store unit (LSUnit) to simulate the speculative1050execution of loads and stores.1051 1052Each load (or store) consumes an entry in the load (or store) queue. Users can1053specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the1054load and store queues respectively. The queues are unbounded by default.1055 1056The LSUnit implements a relaxed consistency model for memory loads and stores.1057The rules are:1058 10591. A younger load is allowed to pass an older load only if there are no1060 intervening stores or barriers between the two loads.10612. A younger load is allowed to pass an older store provided that the load does1062 not alias with the store.10633. A younger store is not allowed to pass an older store.10644. A younger store is not allowed to pass an older load.1065 1066By default, the LSUnit optimistically assumes that loads do not alias1067(`-noalias=true`) store operations. Under this assumption, younger loads are1068always allowed to pass older stores. Essentially, the LSUnit does not attempt1069to run any alias analysis to predict when loads and stores do not alias with1070each other.1071 1072Note that, in the case of write-combining memory, rule 3 could be relaxed to1073allow reordering of non-aliasing store operations. That being said, at the1074moment, there is no way to further relax the memory model (``-noalias`` is the1075only option). Essentially, there is no option to specify a different memory1076type (e.g., write-back, write-combining, write-through; etc.) and consequently1077to weaken, or strengthen, the memory model.1078 1079Other limitations are:1080 1081* The LSUnit does not know when store-to-load forwarding may occur.1082* The LSUnit does not know anything about cache hierarchy and memory types.1083* The LSUnit does not know how to identify serializing operations and memory1084 fences.1085 1086The LSUnit does not attempt to predict if a load or store hits or misses the L11087cache. It only knows if an instruction "MayLoad" and/or "MayStore." For1088loads, the scheduling model provides an "optimistic" load-to-use latency (which1089usually matches the load-to-use latency for when there is a hit in the L1D).1090 1091:program:`llvm-mca` does not (on its own) know about serializing operations or1092memory-barrier like instructions. The LSUnit used to conservatively use an1093instruction's "MayLoad", "MayStore", and unmodeled side effects flags to1094determine whether an instruction should be treated as a memory-barrier. This was1095inaccurate in general and was changed so that now each instruction has an1096IsAStoreBarrier and IsALoadBarrier flag. These flags are mca specific and1097default to false for every instruction. If any instruction should have either of1098these flags set, it should be done within the target's InstrPostProcess class.1099For an example, look at the `X86InstrPostProcess::postProcessInstruction` method1100within `llvm/lib/Target/X86/MCA/X86CustomBehaviour.cpp`.1101 1102A load/store barrier consumes one entry of the load/store queue. A load/store1103barrier enforces ordering of loads/stores. A younger load cannot pass a load1104barrier. Also, a younger store cannot pass a store barrier. A younger load1105has to wait for the memory/load barrier to execute. A load/store barrier is1106"executed" when it becomes the oldest entry in the load/store queue(s). That1107also means, by construction, all of the older loads/stores have been executed.1108 1109In conclusion, the full set of load/store consistency rules are:1110 1111#. A store may not pass a previous store.1112#. A store may not pass a previous load (regardless of ``-noalias``).1113#. A store has to wait until an older store barrier is fully executed.1114#. A load may pass a previous load.1115#. A load may not pass a previous store unless ``-noalias`` is set.1116#. A load has to wait until an older load barrier is fully executed.1117 1118In-order Issue and Execute1119""""""""""""""""""""""""""""""""""""1120In-order processors are modelled as a single ``InOrderIssueStage`` stage. It1121bypasses Dispatch, Scheduler and Load/Store unit. Instructions are issued as1122soon as their operand registers are available and resource requirements are1123met. Multiple instructions can be issued in one cycle according to the value of1124the ``IssueWidth`` parameter in LLVM's scheduling model.1125 1126Once issued, an instruction is moved to ``IssuedInst`` set until it is ready to1127retire. :program:`llvm-mca` ensures that writes are committed in-order. However,1128an instruction is allowed to commit writes and retire out-of-order if1129``RetireOOO`` property is true for at least one of its writes.1130 1131Custom Behaviour1132""""""""""""""""""""""""""""""""""""1133Due to certain instructions not being expressed perfectly within their1134scheduling model, :program:`llvm-mca` isn't always able to simulate them1135perfectly. Modifying the scheduling model isn't always a viable1136option though (maybe because the instruction is modeled incorrectly on1137purpose or the instruction's behaviour is quite complex). The1138CustomBehaviour class can be used in these cases to enforce proper1139instruction modeling (often by customizing data dependencies and detecting1140hazards that :program:`llvm-mca` has no way of knowing about).1141 1142:program:`llvm-mca` comes with one generic and multiple target-specific1143CustomBehaviour classes. The generic class will be used if the ``-disable-cb``1144flag is used or if a target-specific CustomBehaviour class doesn't exist for1145that target. (The generic class does nothing.) Currently, the CustomBehaviour1146class is only a part of the in-order pipeline, but there are plans to add it1147to the out-of-order pipeline in the future.1148 1149CustomBehaviour's main method is `checkCustomHazard()` which uses the1150current instruction and a list of all instructions still executing within1151the pipeline to determine if the current instruction should be dispatched.1152As output, the method returns an integer representing the number of cycles1153that the current instruction must stall for (this can be an underestimate1154if you don't know the exact number and a value of 0 represents no stall).1155 1156If you'd like to add a CustomBehaviour class for a target that doesn't1157already have one, refer to an existing implementation to see how to set it1158up. The classes are implemented within the target-specific backend (for1159example `/llvm/lib/Target/AMDGPU/MCA/`) so that they can access backend symbols.1160 1161Instrument Manager1162""""""""""""""""""""""""""""""""""""1163On certain architectures, scheduling information for certain instructions1164do not contain all of the information required to identify the most precise1165schedule class. For example, data that can have an impact on scheduling can1166be stored in CSR registers.1167 1168One example of this is on RISCV, where values in registers such as `vtype`1169and `vl` change the scheduling behaviour of vector instructions. Since MCA1170does not keep track of the values in registers, instrument comments can1171be used to specify these values.1172 1173InstrumentManager's main function is `getSchedClassID()` which has access1174to the MCInst and all of the instruments that are active for that MCInst.1175This function can use the instruments to override the schedule class of1176the MCInst.1177 1178On RISCV, instrument comments containing LMUL information are used1179by `getSchedClassID()` to map a vector instruction and the active1180LMUL to the scheduling class of the pseudo-instruction that describes1181that base instruction and the active LMUL.1182 1183Custom Views1184""""""""""""""""""""""""""""""""""""1185:program:`llvm-mca` comes with several Views such as the Timeline View and1186Summary View. These Views are generic and can work with most (if not all)1187targets. If you wish to add a new View to :program:`llvm-mca` and it does not1188require any backend functionality that is not already exposed through MC layer1189classes (MCSubtargetInfo, MCInstrInfo, etc.), please add it to the1190`/tools/llvm-mca/View/` directory. However, if your new View is target specific1191AND requires unexposed backend symbols or functionality, you can define it in1192the `/lib/Target/<TargetName>/MCA/` directory.1193 1194To enable this target-specific View, you will have to use this target's1195CustomBehaviour class to override the `CustomBehaviour::getViews()` methods.1196There are 3 variations of these methods based on where you want your View to1197appear in the output: `getStartViews()`, `getPostInstrInfoViews()`, and1198`getEndViews()`. These methods returns a vector of Views so you will want to1199return a vector containing all of the target-specific Views for the target in1200question.1201 1202Because these target specific (and backend dependent) Views require the1203`CustomBehaviour::getViews()` variants, these Views will not be enabled if1204the `-disable-cb` flag is used.1205 1206Enabling these custom Views does not affect the non-custom (generic) Views.1207Continue to use the usual command line arguments to enable / disable those1208Views.1209