296 lines · plain
1=======2ThinLTO3=======4 5.. contents::6 :local:7 8Introduction9============10 11*ThinLTO* compilation is a new type of LTO that is both scalable and12incremental. *LTO* (Link Time Optimization) achieves better13runtime performance through whole-program analysis and cross-module14optimization. However, monolithic LTO implements this by merging all15input into a single module, which is not scalable16in time or memory, and also prevents fast incremental compiles.17 18In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the19compile phase. The ThinLTO bitcode is augmented with a compact summary20of the module. During the link step, only the summaries are read and21merged into a combined summary index, which includes an index of function22locations for later cross-module function importing. Fast and efficient23whole-program analysis is then performed on the combined summary index.24 25However, all transformations, including function importing, occur26later when the modules are optimized in fully parallel backends.27By default, linkers_ that support ThinLTO are set up to launch28the ThinLTO backends in threads. So the usage model is not affected29as the distinction between the fast serial thin link step and the backends30is transparent to the user.31 32For more information on the ThinLTO design and current performance,33see the LLVM blog post `ThinLTO: Scalable and Incremental LTO34<http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_.35While tuning is still in progress, results in the blog post show that36ThinLTO already performs well compared to LTO, in many cases matching37the performance improvement.38 39Current Status40==============41 42Clang/LLVM43----------44.. _compiler:45 46The 3.9 release of clang includes ThinLTO support. However, ThinLTO47is under active development, and new features, improvements and bugfixes48are being added for the next release. For the latest ThinLTO support,49`build a recent version of clang and LLVM50<https://llvm.org/docs/CMake.html>`_.51 52Linkers53-------54.. _linkers:55.. _linker:56 57ThinLTO is currently supported for the following linkers:58 59- **gold (via the gold-plugin)**:60 Similar to monolithic LTO, this requires using61 a `gold linker configured with plugins enabled62 <https://llvm.org/docs/GoldPlugin.html>`_.63- **ld64**:64 Starting with `Xcode 8 <https://developer.apple.com/xcode/>`_.65- **lld**:66 Starting with r284050 for ELF, r298942 for COFF.67 68Usage69=====70 71Basic72-----73 74To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g.75 76.. code-block:: console77 78 % clang -flto=thin -O2 file1.c file2.c -c79 % clang -flto=thin -O2 file1.o file2.o -o a.out80 81When using lld-link, the -flto option need only be added to the compile step:82 83.. code-block:: console84 85 % clang-cl -flto=thin -O2 -c file1.c file2.c86 % lld-link /out:a.exe file1.obj file2.obj87 88As mentioned earlier, by default the linkers will launch the ThinLTO backend89threads in parallel, passing the resulting native object files back to the90linker for the final native link. As such, the usage model is the same as91non-LTO.92 93With gold, if you see an error during the link of the form:94 95.. code-block:: console96 97 /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory98 99Then either gold was not configured with plugins enabled, or clang100was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See101the instructions for the102`LLVM gold plugin <https://llvm.org/docs/GoldPlugin.html#how-to-build-it>`_.103 104Controlling Backend Parallelism105-------------------------------106.. _parallelism:107 108By default, the ThinLTO link step will launch as many109threads in parallel as there are cores. If the number of110cores can't be computed for the architecture, then it will launch111``std::thread::hardware_concurrency`` number of threads in parallel.112For machines with hyper-threading, this is the total number of113virtual cores. For some applications and machine configurations this114may be too aggressive, in which case the amount of parallelism can115be reduced to ``N`` via:116 117- gold:118 ``-Wl,-plugin-opt,jobs=N``119- ld64:120 ``-Wl,-mllvm,-threads=N``121- ld.lld, ld64.lld:122 ``-Wl,--thinlto-jobs=N``123- lld-link:124 ``/opt:lldltojobs=N``125 126Other possible values for ``N`` are:127 128- 0:129 Use one thread per physical core (default)130- 1:131 Use a single thread only (disable multi-threading)132- all:133 Use one thread per logical core (uses all hyper-threads)134 135Incremental136-----------137.. _incremental:138 139ThinLTO supports fast incremental builds through the use of a cache,140which currently must be enabled through a linker option.141 142- gold (as of LLVM 4.0):143 ``-Wl,-plugin-opt,cache-dir=/path/to/cache``144- ld64 (supported since clang 3.9 and Xcode 8) and Mach-O ld64.lld (as of LLVM145 15.0):146 ``-Wl,-cache_path_lto,/path/to/cache``147- ELF ld.lld (as of LLVM 5.0):148 ``-Wl,--thinlto-cache-dir=/path/to/cache``149- COFF lld-link (as of LLVM 6.0):150 ``/lldltocache:/path/to/cache``151 152Cache Pruning153-------------154 155To help keep the size of the cache under control, ThinLTO supports cache156pruning. Cache pruning is supported with gold, ld64, and lld, but currently only157gold and lld allow you to control the policy with a policy string. The cache158policy must be specified with a linker option.159 160- gold (as of LLVM 6.0):161 ``-Wl,-plugin-opt,cache-policy=POLICY``162- ELF ld.lld (as of LLVM 5.0), Mach-O ld64.lld (as of LLVM 15.0):163 ``-Wl,--thinlto-cache-policy=POLICY``164- COFF lld-link (as of LLVM 6.0):165 ``/lldltocachepolicy:POLICY``166 167A policy string is a series of key-value pairs separated by ``:`` characters.168Possible key-value pairs are:169 170- ``cache_size=X%``: The maximum size for the cache directory is ``X`` percent171 of the available space on the disk. Set to 100 to indicate no limit,172 50 to indicate that the cache size will not be left over half the available173 disk space. A value over 100 is invalid. A value of 0 disables the percentage174 size-based pruning. The default is 75%.175 176- ``cache_size_bytes=X``, ``cache_size_bytes=Xk``, ``cache_size_bytes=Xm``,177 ``cache_size_bytes=Xg``:178 Sets the maximum size for the cache directory to ``X`` bytes (or KB, MB,179 GB respectively). A value over the amount of available space on the disk180 will be reduced to the amount of available space. A value of 0 disables181 the byte size-based pruning. The default is no byte size-based pruning.182 183 Note that ThinLTO will apply both size-based pruning policies simultaneously,184 and changing one does not affect the other. For example, a policy of185 ``cache_size_bytes=1g`` on its own will cause both the 1GB and default 75%186 policies to be applied unless the default ``cache_size`` is overridden.187 188- ``cache_size_files=X``:189 Set the maximum number of files in the cache directory. Set to 0 to indicate190 no limit. The default is 1000000 files.191 192- ``prune_after=Xs``, ``prune_after=Xm``, ``prune_after=Xh``: Sets the193 expiration time for cache files to ``X`` seconds (or minutes, hours194 respectively). When a file hasn't been accessed for ``prune_after`` seconds,195 it is removed from the cache. A value of 0 disables the expiration-based196 pruning. The default is 1 week.197 198- ``prune_interval=Xs``, ``prune_interval=Xm``, ``prune_interval=Xh``:199 Sets the pruning interval to ``X`` seconds (or minutes, hours200 respectively). This is intended to be used to avoid scanning the directory201 too often. It does not impact the decision of which files to prune. A202 value of 0 forces the scan to occur. The default is every 20 minutes.203 204Clang Bootstrap205---------------206 207To `bootstrap clang/LLVM <https://llvm.org/docs/AdvancedBuilds.html#bootstrap-builds>`_208with ThinLTO, follow these steps:209 2101. The host compiler_ must be a version of clang that supports ThinLTO.211#. The host linker_ must support ThinLTO (and in the case of gold, must be212 `configured with plugins enabled <https://llvm.org/docs/GoldPlugin.html>`_).213#. Use the following additional `CMake variables214 <https://llvm.org/docs/CMake.html#options-and-variables>`_215 when configuring the bootstrap compiler build:216 217 * ``-DLLVM_ENABLE_LTO=Thin``218 * ``-DCMAKE_C_COMPILER=/path/to/host/clang``219 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang++``220 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib``221 * ``-DCMAKE_AR=/path/to/host/llvm-ar``222 223 Or, on Windows:224 225 * ``-DLLVM_ENABLE_LTO=Thin``226 * ``-DCMAKE_C_COMPILER=/path/to/host/clang-cl.exe``227 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang-cl.exe``228 * ``-DCMAKE_LINKER=/path/to/host/lld-link.exe``229 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib.exe``230 * ``-DCMAKE_AR=/path/to/host/llvm-ar.exe``231 232#. To use additional linker arguments for controlling the backend233 parallelism_ or enabling incremental_ builds of the bootstrap compiler,234 after configuring the build, modify the resulting CMakeCache.txt file in the235 build directory. Specify any additional linker options after236 ``CMAKE_EXE_LINKER_FLAGS:STRING=``. Note the configure may fail if237 linker plugin options are instead specified directly in the previous step.238 239The ``BOOTSTRAP_LLVM_ENABLE_LTO=Thin`` will enable ThinLTO for stage 2 and240stage 3 in case the compiler used for stage 1 does not support the ThinLTO241option.242 243Integrated Distributed ThinLTO (DTLTO)244--------------------------------------245 246Integrated Distributed ThinLTO (DTLTO) enables the distribution of backend247ThinLTO compilations via external distribution systems, such as Incredibuild,248during the traditional link step.249 250The implementation is documented here: https://llvm.org/docs/DTLTO.html.251 252Command-Line Options253^^^^^^^^^^^^^^^^^^^^254 255DTLTO requires the LLD linker (``-fuse-ld=lld``).256 257``-fthinlto-distributor=<path>``258 - Specifies the ``<path>`` to the distributor process executable for DTLTO.259 - If specified, ThinLTO backend compilations will be distributed by LLD.260 261``-Xthinlto-distributor=<arg>``262 - Passes ``<arg>`` to the distributor process (see ``-fthinlto-distributor=``).263 - Can be specified multiple times to pass multiple options.264 - Multiple options can also be specified by separating them with commas.265 266If ``-fthinlto-distributor=`` is specified, Clang supplies the path to a267compiler to be executed remotely to perform the ThinLTO backend268compilations. Currently, this is Clang itself.269 270Usage271^^^^^272 273Compilation is unchanged from ThinLTO. DTLTO options need to supplied for the link step:274 275.. code-block:: console276 277 % clang -flto=thin -fthinlto-distributor=distribute.sh -Xthinlto-distributor=--verbose,--j10 -fuse-ld=lld file1.o file2.o278 % clang -flto=thin -fthinlto-distributor=$(which python) -Xthinlto-distributor=distribute.py -fuse-ld=lld file1.o file2.o279 280When using lld-link:281 282.. code-block:: console283 284 % lld-link /out:a.exe file1.obj file2.obj /thinlto-distributor:distribute.exe /thinlto-remote-compiler:${LLVM}\bin\clang.exe /thinlto-distributor-arg:--verbose285 286Note that currently, DTLTO is only supported in some LLD flavors. Support can287be added to other LLD flavours in the future.288See `DTLTO <https://lld.llvm.org/DTLTO.html>`_ for more information.289 290More Information291================292 293* From LLVM project blog:294 `ThinLTO: Scalable and Incremental LTO295 <http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_296