745 lines · plain
1=============================================2Machine Learning - Guided Optimization (MLGO)3=============================================4 5Introduction6============7 8MLGO refers to integrating ML techniques (primarily) to replace heuristics within9LLVM with machine learned models.10 11Currently the following heuristics feature such integration:12 13* Inlining for size14* Register allocation (LLVM greedy eviction heuristic) for performance15 16This document is an outline of the tooling and APIs facilitating MLGO.17 18.. note::19 20 The tools for orchestrating ML training are not part of LLVM, as they are21 dependency-heavy - both on the ML infrastructure choice, as well as choices of22 distributed computing. For the training scenario, LLVM only contains facilities23 enabling it, such as corpus extraction, training data extraction, and evaluation24 of models during training.25 26 27.. contents::28 29Corpus Tooling30==============31 32Within the LLVM monorepo, there is the ``mlgo-utils`` Python package that33lives at ``llvm/utils/mlgo-utils``. This package primarily contains tooling34for working with corpora, or collections of LLVM bitcode. We use these corpora35to train and evaluate ML models. Corpora consist of a description in JSON36format at ``corpus_description.json`` in the root of the corpus, and then37a bitcode file and command line flags file for each extracted module. The38corpus structure is designed to contain sufficient information to fully39compile the bitcode to bit-identical object files.40 41.. program:: extract_ir.py42 43Synopsis44--------45 46Extracts a corpus from some form of a structured compilation database. This47tool supports a variety of different scenarios and input types.48 49Options50-------51 52.. option:: --input53 54 The path to the input. This should be a path to a supported structured55 compilation database. Currently only ``compile_commands.json`` files, linker56 parameter files, a directory containing object files (for the local57 ThinLTO case only), or a JSON file containing a bazel aquery result are58 supported.59 60.. option:: --input_type61 62 The type of input that has been passed to the ``--input`` flag.63 64.. option:: --output_dir65 66 The output directory to place the corpus in.67 68.. option:: --num_workers69 70 The number of workers to use for extracting bitcode into the corpus. This71 defaults to the number of hardware threads available on the host system.72 73.. option:: --llvm_objcopy_path74 75 The path to the llvm-objcopy binary to use when extracting bitcode.76 77.. option:: --obj_base_dir78 79 The base directory for object files. Bitcode files that get extracted into80 the corpus will be placed into the output directory based on where their81 source object files are placed relative to this path.82 83.. option:: --cmd_filter84 85 Allows filtering of modules by command line. If set, only modules that match86 the filter will be extracted into the corpus. Regular expressions are87 supported in some instances.88 89.. option:: --thinlto_build90 91 If the build was performed with ThinLTO, this should be set to either92 ``distributed`` or ``local`` depending upon how the build was performed.93 94.. option:: --cmd_section_name95 96 This flag allows specifying the command line section name. This is needed97 on non-ELF platforms where the section name might differ.98 99.. option:: --bitcode_section_name100 101 This flag allows specifying the bitcode section name. This is needed on102 non-ELF platforms where the section name might differ.103 104Example: CMake105--------------106 107CMake can output a ``compilation_commands.json`` compilation database if the108``CMAKE_EXPORT_COMPILE_COMMANDS`` switch is turned on at compile time. It is109also necessary to enable bitcode embedding (done by passing 110``-Xclang -fembed-bitcode=all`` to all C/C++ compilation actions in the111non-ThinLTO case). For example, to extract a corpus from clang, you would112run the following commands (assuming that the system C/C++ compiler is clang):113 114.. code-block:: bash115 116 cmake -GNinja \117 -DCMAKE_BUILD_TYPE=Release \118 -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \119 -DCMAKE_C_FLAGS="-Xclang -fembed-bitcode=all" \120 -DCMAKE_CXX_FLAGS="-Xclang -fembed-bitcode-all"121 ../llvm122 ninja123 124After running CMake and building the project, there should be a125 ``compilation_commands.json`` file within the build directory. You can then126 run the following command to create a corpus:127 128.. code-block:: bash129 130 python3 ./extract_ir.py \131 --input=./build/compile_commands.json \132 --input_type=json \133 --output_dir=./corpus134 135After running the above command, there should be a full136corpus of bitcode within the ``./corpus`` directory.137 138Example: Bazel Aquery139---------------------140 141This tool also supports extracting bitcode from bazel in multiple ways142depending upon the exact configuration. For ThinLTO, a linker parameters file143is preferred. For the non-ThinLTO case, the script will accept the output of144``bazel aquery`` which it will use to find all the object files that are linked145into a specific target and then extract bitcode from them. First, you need146to generate the aquery output:147 148.. code-block:: bash149 150 bazel aquery --output=jsonproto //path/to:target > /path/to/aquery.json151 152Afterwards, assuming that the build is already complete, you can run this153script to create a corpus:154 155.. code-block:: bash156 157 python3 ./extract_ir.py \158 --input=/path/to/aquery.json \159 --input_type=bazel_aqeury \160 --output_dir=./corpus \161 --obj_base_dir=./bazel-bin162 163This will again leave a corpus that contains all the bitcode files. This mode164does not capture all object files in the build however, only the ones that165are involved in the link for the binary passed to the ``bazel aquery``166invocation.167 168.. program:: make_corpus.py169 170Synopsis171--------172 173Creates a corpus from a collection of bitcode files.174 175Options176-------177 178.. option:: --input_dir179 180 The input directory to search for bitcode files in.181 182.. option:: --output_dir183 184 The output directory to place the constructed corpus in.185 186.. option:: --default_args187 188 A list of space separated flags that are put into the corpus description.189 These are used by some tooling when compiling the modules within the corpus.190 191.. program:: combine_training_corpus.py192 193Synopsis194--------195 196Combines two training corpora that share the same parent folder by generating197a new ``corpus_description.json`` that contains all the modules in both corpora.198 199Options200-------201 202.. option:: --root_dir203 204 The root directory that contains subfolders consisting of the corpora that205 should be combined.206 207Interacting with ML models208==========================209 210We interact with ML models in 2 primary scenarios: one is to train such a model.211The other, inference, is to use a model during compilation, to make optimization212decisions.213 214For a specific optimization problem - i.e. inlining, or regalloc eviction - we215first separate correctness - preserving decisions from optimization decisions.216For example, not inlining functions marked "no inline" is an example of the217former. Same is not evicting an unevictable live range. An example of the latter218is deciding to inline a function that will bloat the caller size, just because219we have reason to believe that later, the effect will be some constant220propagation that will actually reduce the size (or dynamic instruction count).221 222ML models can be understood as functions. Their inputs are tensors - buffers of223scalars. The output (in our case, singular) is a scalar. For example, for224inlining, the inputs are properties of the caller, callee, and the callsite225being analyzed for inlining. The output is a boolean.226 227Inputs and outputs are named, have a scalar type (e.g. int32_t) and a shape228(e.g. 3x4). These are the elements that we use to bind to a ML model.229 230In both training and inference, we want to expose to ML (training algorithms or231trained model, respectively) the features we want to make optimization232decisions on. In that regard, the interface from the compiler side to the ML233side is the same: pass features, and get a decision. It's essentially a function234call, where the parameters and result are bound by name and are described by235name, scalar type, and shape tuples.236 237The main types in LLVM are:238 239- ``MLModelRunner`` - an abstraction for the decision making mechanism240- ``TensorSpec`` which describes a tensor.241 242TensorSpec243----------244 245See ``llvm/Analysis/TensorSpec.h``. This is a simple data bag, identifying a246tensor by name (a string), scalar type, and shape (a vector of ints). The scalar247type can only be int (8, 16, 32, or 64), signed or unsigned; float; or double.248 249MLModelRunner250-------------251 252See ``llvm/Analysis/MLModelRunner.h``. The abstraction has a pure virtual,253``evaluateUntyped``, but the contract with implementers is a bit more involved:254 255Implementers256^^^^^^^^^^^^257 258At construction, the implementer is expected to receive a list of ``TensorSpec``259for input features and the ``TensorSpec`` of the output (e.g. 260``std::vector<TensorSpec>``). The list type is not contractual, but it must be261a 0-based indexing array-like container. Given a ``TensorSpec`` at index "I" in262the input list, that has a name "N", shape "D1 x D2x ... Dn", and scalar type263"T", the implementer must:264 265- set up a contiguous buffer sized ``sizeof(T) * D1 * D2 * ... * Dn``. This266 buffer's lifetime must be the same as the lifetime of the implementer object.267- call ``MLModelRunner::setUpBufferForTensor`` passing I, the ``TensorSpec``,268 and the buffer above.269 270Internally, the expectation is that the implementer uses the name (and maybe271shape) of a ``TensorSpec`` for binding (e.g. lookup in an underlying ML model).272 273``MLModelRunner::setUpBufferForTensor`` stores each buffer at the corresponding274index (i.e. its position in the list used at construction). The expectation is275that the user will use that position when calling ``MLModelRunner::getTensor``276to retrieve the underlying buffer (more on that in a bit).277 278The implementation of ``evaluateUntyped`` is expected to use the value in the279buffers described above, carry out whatever computation (e.g. evaluate a ML280model) and then place the outcome in an output buffer which will be returned to281the caller. Importantly, ``evaluateUntyped`` must not reset the input buffers.282This is because during training we may want to log the features and decisions,283and since the data is already buffered, there's no reason to force backing it284up elsewhere.285 286Users287^^^^^288 289The users must pass the input ``TensorSpec`` list at the construction of a290specific ``MLModelRunner`` object. After that, users can be agnostic of the291specific implementation, and would typically follow the following workflow:292 293- call ``getTensor`` or ``getTensorUntyped``, for each input tensor, identified294 by its index (i.e. the index of the corresponding ``TensorSpec`` in the list295 used at construction).296- populate the tensor buffer of each input tensor with values. Users can take297 advantage of the stability of the tensor buffers like set only once those that298 don't change, or cache the buffer address299- call ``evaluate`` and use its result.300 301Versioning302^^^^^^^^^^303 304We support a model "knowing" less inputs than the compiler. This is supported by305``MLModelRunner::setUpBufferForTensor``. If a ``TensorSpec`` requested by the306compiler is not supported by the underlying model, the ``MLModelRunner``307implementer must still call ``setUpBufferForTensor`` with a ``nullptr`` value308for the buffer. In turn, ``MLModelRunner`` will allocate an appropriately - sized309buffer and track its lifetime. The user can safely populate that buffer. Since310the rest of the inputs are still provided, this allows an evolution model where311we first add features to the compiler and continue using older models without312regressing. Then, the new compiler can be used to train new models. Deprecating313features in the compiler involves, then, training first a model without those314features.315 316``MLModelRunner`` implementations317^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^318 319We currently feature 4 implementations:320 321- ``ModelUnderTrainingRunner``. This requires the compiler be built with TFLite322 support. It allows loading a TFLite model dynamically and is primarily323 intended for training scenarios, but it can be used relatively easily in324 production build environments, as it does not change how the compiler operates325 (why this remark is necessary will become clear in a few paragraphs)326 327- ``ReleaseModeModelRunner``. This is intended for inference scenarios. This328 uses the rules defined in ``llvm/cmake/modules/TensorFlowCompile.cmake`` to329 convert, at the time the compiler is built, TensorFlow Saved Models into a330 header (.h) and native object (.o). The latter is a CPU-based implementation of331 the neural network, together with its weights (essentially, loops performing332 matrix multiplications)333 334.. note::335 336 we are actively working on replacing this with an EmitC implementation337 requiring no out of tree build-time dependencies.338 339- ``InteractiveModelRunner``. This is intended for training scenarios where the340 training algorithm drives compilation. This model runner has no special341 dependencies, and relies on I/O pipes to communicate with a separate process,342 presumably a python training algorithm. We do not envision using this in a343 production environment.344 345- ``NoInferenceModelRunner``. This serves as a store for feature values, and its346 ``evaluate`` should never be called. It's used for training scenarios, when we347 want to capture the behavior of the default (non-ML) heuristic.348 349Note that training leaves it to the training infrastructure to handle350distributed computing. The assumed architecture has python processes351communicating remotely between themselves, but managing local communication with352clang.353 354Logging Facility355----------------356 357When training models, we need to expose the features we will want to use during358inference, as well as outcomes, to guide reward-based learning techniques. This359can happen in 2 forms:360 361- when running the compiler on some input, as a capture of the features and362 actions taken by some policy or a model currently being used.363 For example, see ``DevelopmentModeInlineAdvisor`` or ``DevelopmentModeEvictAdvisor``364 in ``MLRegallocEvictAdvisor.cpp``. In more detail, in the former case, if365 ``-training-log`` is specified, the features and actions (inline/no inline)366 from each inlining decision are saved to the specified file. Since367 ``MLModelRunner`` implementations hold on to feature values (they don't get368 cleared by ``evaluate``), logging is easily supported by just looping over the369 model runner's features and passing the tensor buffers to the logger. Note how370 we use the ``NoInferenceModelRunner`` to capture the features observed when371 using the default policy.372 373- as a serialization mechanism for the ``InteractiveModelRunner``. Here, we need374 to pass the observed features over IPC (a file descriptor, likely a named375 pipe).376 377Both cases require serializing the same kind of data and we support both with378``Analysis/Utils/TrainingLogger``.379 380The goal of the logger design was avoiding any new dependency, and optimizing381for the tensor scenario - i.e. exchanging potentially large buffers of fixed382size, containing scalars. We explicitly assume the reader of the format has the383same endianness as the compiler host, and we further expect the reader and the384compiler run on the same host. This is because we expect the training scenarios385have a (typically python) process managing the compiler process, and we leave to386the training side to handle remoting.387 388The logger produces the following sequence:389 390- a header describing the structure of the log. This is a one-line textual JSON391 dictionary with the following elements:392 393 - ``features``: a list of JSON-serialized ``TensorSpec`` values. The position394 in the list matters, as it will be the order in which values will be395 subsequently recorded. If we are just logging (i.e. not using the396 ``InteractiveModelRunner``), the last feature should be that of the action397 (e.g. "inline/no inline", or "index of evicted live range")398 - (optional) ``score``: a ``TensorSpec`` describing a value we will include to399 help formulate a reward. This could be a size estimate or a latency estimate.400 - (optional) ``advice``: a ``TensorSpec`` describing the action. This is used401 for the ``InteractiveModelRunner``, in which case it shouldn't be in the 402 ``features`` list.403- a sequence of ``contexts``. Contexts are independent traces of the optimization404 problem. For module passes, there is only one context, for function passes,405 there is a context per function. The start of a context is marked with a406 one-line JSON dictionary of the form ``{"context": <context name, a string>}``407 408 Each context has a sequence of:409 410 - ``observations``. An observation is:411 412 - one-line JSON ``{"observation": <observation number. 0-indexed>}``413 - a binary dump of the tensor buffers, in the order in which they were414 specified in the header.415 - a new line character416 - if ``score`` was specified in the header:417 418 - a one-line JSON object ``{"outcome": <value>}``, where the ``value``419 conforms to the ``TensorSpec`` in defined for the ``score`` in the header.420 - the outcome value, as a binary dump421 - a new line character.422 423The format uses a mix of textual JSON (for headers) and binary dumps (for tensors)424because the headers are not expected to dominate the payload - the tensor values425are. We wanted to avoid overburdening the log reader - likely python - from426additional dependencies; and the one-line JSON makes it rudimentarily possible427to inspect a log without additional tooling.428 429A python utility for reading logs, used for tests, is available at430``Analysis/models/log_reader.py``. A utility showcasing the ``InteractiveModelRunner``,431which uses this reader as well, is at ``Analysis/models/interactive_host.py``.432The latter is also used in tests.433 434There is no C++ implementation of a log reader. We do not have a scenario435motivating one.436 437Embeddings438==========439 440LLVM provides embedding frameworks to generate vector representations of code441at different abstraction levels. These embeddings capture syntactic, semantic,442and structural properties of the code and can be used as features for machine443learning models in various compiler optimization tasks.444 445Two embedding frameworks are available:446 447- **IR2Vec**: Generates embeddings for LLVM IR448- **MIR2Vec**: Generates embeddings for Machine IR449 450Both frameworks follow a similar architecture with vocabulary-based embedding451generation, where a vocabulary maps code entities to n-dimensional floating452point vectors. These embeddings can be computed at multiple granularity levels453(instruction, basic block, and function) and used for ML-guided compiler454optimizations.455 456IR2Vec457------458 459IR2Vec is a program embedding approach designed specifically for LLVM IR. It460is implemented as a function analysis pass in LLVM. The IR2Vec embeddings461capture syntactic, semantic, and structural properties of the IR through 462learned representations. These representations are obtained as a JSON 463vocabulary that maps the entities of the IR (opcodes, types, operands) to 464n-dimensional floating point vectors (embeddings). 465 466With IR2Vec, representation at different granularities of IR, such as467instructions, functions, and basic blocks, can be obtained. Representations 468of loops and regions can be derived from these representations, which can be469useful in different scenarios. The representations can be useful for various470downstream tasks, including ML-guided compiler optimizations.471 472The core components are:473 - **Vocabulary**: A mapping from IR entities (opcodes, types, etc.) to their474 vector representations. This is managed by ``IR2VecVocabAnalysis``. The 475 vocabulary (.json file) contains three sections -- Opcodes, Types, and 476 Arguments, each containing the representations of the corresponding 477 entities.478 479 .. note::480 481 It is mandatory to have these three sections present in the vocabulary file 482 for it to be valid; order in which they appear does not matter.483 484 - **Embedder**: A class (``ir2vec::Embedder``) that uses the vocabulary to485 compute embeddings for instructions, basic blocks, and functions.486 487Using IR2Vec488^^^^^^^^^^^^489 490.. note::491 492 This section describes how to use IR2Vec within LLVM passes. A standalone 493 tool :doc:`CommandGuide/llvm-ir2vec` is available for generating the494 embeddings and triplets from LLVM IR files, which can be useful for495 training vocabularies and generating embeddings outside of compiler passes.496 497For generating embeddings, first the vocabulary should be obtained. Then, the 498embeddings can be computed and accessed via an ``ir2vec::Embedder`` instance.499 5001. **Get the Vocabulary**:501 In a ModulePass, get the vocabulary analysis result:502 503 .. code-block:: c++504 505 auto &VocabRes = MAM.getResult<IR2VecVocabAnalysis>(M);506 if (!VocabRes.isValid()) {507 // Handle error: vocabulary is not available or invalid508 return;509 }510 const ir2vec::Vocab &Vocabulary = VocabRes.getVocabulary();511 512 Note that ``IR2VecVocabAnalysis`` pass is immutable.513 5142. **Create Embedder instance**:515 With the vocabulary, create an embedder for a specific function:516 517 .. code-block:: c++518 519 // Assuming F is an llvm::Function&520 // For example, using IR2VecKind::Symbolic:521 std::unique_ptr<ir2vec::Embedder> Emb =522 ir2vec::Embedder::create(IR2VecKind::Symbolic, F, Vocabulary);523 524 5253. **Compute and Access Embeddings**:526 Call ``getFunctionVector()`` to get the embedding for the function. 527 528 .. code-block:: c++529 530 ir2vec::Embedding FuncVector = Emb->getFunctionVector();531 532 Currently, ``Embedder`` can generate embeddings at three levels: Instructions,533 Basic Blocks, and Functions. Appropriate getters are provided to access the534 embeddings at these levels.535 536 .. note::537 538 The validity of ``Embedder`` instance (and the embeddings it generates) is539 tied to the function it is associated with remains unchanged. If the function540 is modified, the embeddings may become stale and should be recomputed accordingly.541 5424. **Working with Embeddings:**543 Embeddings are represented as ``std::vector<double>``. These544 vectors as features for machine learning models, compute similarity scores545 between different code snippets, or perform other analyses as needed.546 547Further Details548^^^^^^^^^^^^^^^549 550For more detailed information about the IR2Vec algorithm, its parameters, and551advanced usage, please refer to the original paper:552`IR2Vec: LLVM IR Based Scalable Program Embeddings <https://doi.org/10.1145/3418463>`_.553 554For information about using IR2Vec tool for generating embeddings and555triplets from LLVM IR, see :doc:`CommandGuide/llvm-ir2vec`.556 557The LLVM source code for ``IR2Vec`` can also be explored to understand the 558implementation details.559 560MIR2Vec561-------562 563MIR2Vec is an extension of IR2Vec designed specifically for LLVM Machine IR 564(MIR). It generates embeddings for machine-level instructions, basic blocks, 565and functions. MIR2Vec operates on the target-specific machine representation,566capturing machine instruction semantics including opcodes, operands, and 567register information at the machine level.568 569MIR2Vec extends the vocabulary to include:570 571- **Machine Opcodes**: Target-specific instruction opcodes derived from the572 TargetInstrInfo, grouped by instruction semantics.573 574- **Common Operands**: All common operand types (excluding register operands),575 defined by the ``MachineOperand::MachineOperandType`` enum.576 577- **Physical Register Classes**: Register classes defined by the target,578 specialized for physical registers.579 580- **Virtual Register Classes**: Register classes defined by the target,581 specialized for virtual registers.582 583The core components are:584 585- **Vocabulary**: A mapping from machine IR entities (opcodes, operands, register586 classes) to their vector representations. This is managed by 587 ``MIR2VecVocabLegacyAnalysis`` for the legacy pass manager, with a 588 ``MIR2VecVocabProvider`` that can be used standalone or wrapped by pass 589 managers. The vocabulary (.json file) contains sections for opcodes, common 590 operands, physical register classes, and virtual register classes.591 592 .. note::593 594 The vocabulary file should contain these sections for it to be valid.595 596- **Embedder**: A class (``mir2vec::MIREmbedder``) that uses the vocabulary to597 compute embeddings for machine instructions, machine basic blocks, and 598 machine functions. Currently, ``SymbolicMIREmbedder`` is the available 599 implementation.600 601Using MIR2Vec602^^^^^^^^^^^^^603 604.. note::605 606 This section describes how to use MIR2Vec within LLVM passes. `llvm-ir2vec`607 tool ` :doc:`CommandGuide/llvm-ir2vec` can be used for generating MIR2Vec608 embeddings from Machine IR files (.mir), which can be useful for generating609 embeddings outside of compiler passes.610 611To generate MIR2Vec embeddings in a compiler pass, first obtain the vocabulary,612then create an embedder instance to compute and access embeddings.613 6141. **Get the Vocabulary**:615 In a MachineFunctionPass, get the vocabulary from the analysis:616 617 .. code-block:: c++618 619 auto &VocabAnalysis = getAnalysis<MIR2VecVocabLegacyAnalysis>();620 auto VocabOrErr = VocabAnalysis.getMIR2VecVocabulary(*MF.getFunction().getParent());621 if (!VocabOrErr) {622 // Handle error: vocabulary is not available or invalid623 return;624 }625 const mir2vec::MIRVocabulary &Vocabulary = *VocabOrErr;626 627 Note that ``MIR2VecVocabLegacyAnalysis`` is an immutable pass.628 6292. **Create Embedder instance**:630 With the vocabulary, create an embedder for a specific machine function:631 632 .. code-block:: c++633 634 // Assuming MF is a MachineFunction&635 // For example, using MIR2VecKind::Symbolic:636 std::unique_ptr<mir2vec::MIREmbedder> Emb =637 mir2vec::MIREmbedder::create(MIR2VecKind::Symbolic, MF, Vocabulary);638 639 6403. **Compute and Access Embeddings**:641 Call ``getMFunctionVector()`` to get the embedding for the machine function.642 643 .. code-block:: c++644 645 mir2vec::Embedding FuncVector = Emb->getMFunctionVector();646 647 Currently, ``MIREmbedder`` can generate embeddings at three levels: Machine648 Instructions, Machine Basic Blocks, and Machine Functions. Appropriate 649 getters are provided to access the embeddings at these levels.650 651 .. note::652 653 The validity of the ``MIREmbedder`` instance (and the embeddings it 654 generates) is tied to the machine function it is associated with. If the 655 machine function is modified, the embeddings may become stale and should 656 be recomputed accordingly.657 6584. **Working with Embeddings:**659 Embeddings are represented as ``std::vector<double>``. These vectors can be660 used as features for machine learning models, compute similarity scores661 between different code snippets, or perform other analyses as needed.662 663Further Details664^^^^^^^^^^^^^^^665 666For more detailed information about the MIR2Vec algorithm, its parameters, and667advanced usage, please refer to the original paper:668`RL4ReAl: Reinforcement Learning for Register Allocation <https://doi.org/10.1145/3578360.3580273>`_.669 670For information about using MIR2Vec tool for generating embeddings from671Machine IR, see :doc:`CommandGuide/llvm-ir2vec`.672 673The LLVM source code for ``MIR2Vec`` can be explored to understand the 674implementation details. See ``llvm/include/llvm/CodeGen/MIR2Vec.h`` and 675``llvm/lib/CodeGen/MIR2Vec.cpp``.676 677Building with ML support678========================679 680.. note::681 682 For up to date information on custom builds, see the ``ml-*``683 `build bots <http://lab.llvm.org>`_. They are set up using 684 `like this <https://github.com/google/ml-compiler-opt/blob/main/buildbot/buildbot_init.sh>`_.685 686Embed pre-trained models (aka "release" mode)687---------------------------------------------688 689This supports the ``ReleaseModeModelRunner`` model runners.690 691You need a tensorflow pip package for the AOT (ahead-of-time) Saved Model compiler692and a thin wrapper for the native function generated by it. We currently support693TF 2.15. We recommend using a Python virtual env (in which case, remember to694pass ``-DPython3_ROOT_DIR`` to ``cmake``).695 696Once you install the pip package, find where it was installed:697 698.. code-block:: console699 700 TF_PIP=$(sudo -u buildbot python3 -c "import tensorflow as tf; import os; print(os.path.dirname(tf.__file__))")``701 702Then build LLVM:703 704.. code-block:: console705 706 cmake -DTENSORFLOW_AOT_PATH=$TF_PIP \707 -DLLVM_INLINER_MODEL_PATH=<path to inliner saved model dir> \708 -DLLVM_RAEVICT_MODEL_PATH=<path to regalloc eviction saved model dir> \709 <...other options...> 710 711The example shows the flags for both inlining and regalloc, but either may be712omitted.713 714You can also specify a URL for the path, and it is also possible to pre-compile715the header and object and then just point to the precompiled artifacts. See for716example ``LLVM_OVERRIDE_MODEL_HEADER_INLINERSIZEMODEL``.717 718.. note::719 720 We are transitioning away from the AOT compiler shipping with the721 tensorflow package, and to a EmitC, in-tree solution, so these details will722 change soon.723 724Using TFLite (aka "development" mode)725-------------------------------------726 727This supports the ``ModelUnderTrainingRunner`` model runners.728 729Build the TFLite package using `this script <https://raw.githubusercontent.com/google/ml-compiler-opt/refs/heads/main/buildbot/build_tflite.sh>`_.730Then, assuming you ran that script in ``/tmp/tflitebuild``, just pass731``-C /tmp/tflitebuild/tflite.cmake`` to the ``cmake`` for LLVM.732 733Interactive Mode (for training / research)734------------------------------------------ 735 736The ``InteractiveModelRunner`` is available with no extra dependencies. For the737optimizations that are currently MLGO-enabled, it may be used as follows:738 739- for inlining: ``-mllvm -enable-ml-inliner=release -mllvm -inliner-interactive-channel-base=<name>``740- for regalloc eviction: ``-mllvm -regalloc-evict-advisor=release -mllvm -regalloc-evict-interactive-channel-base=<name>``741 742where the ``name`` is a path fragment. We will expect to find 2 files,743``<name>.in`` (readable, data incoming from the managing process) and744``<name>.out`` (writable, the model runner sends data to the managing process)745