407 lines · plain
1========2Multilib3========4 5Introduction6============7 8This document describes how multilib is implemented in Clang.9 10What is multilib and why might you care?11If you're :doc:`cross compiling<CrossCompilation>` then you can't use native12system headers and libraries. To address this, you can use a combination of13``--sysroot``, ``-isystem`` and ``-L`` options to point Clang at suitable14directories for your target.15However, when there are many possible directories to choose from, it's not16necessarily obvious which one to pick.17Multilib allows a toolchain designer to imbue the toolchain with the ability to18pick a suitable directory automatically, based on the options the user provides19to Clang. For example, if the user specifies20``--target=arm-none-eabi -mcpu=cortex-m4`` the toolchain can choose a directory21containing headers and libraries suitable for Armv7E-M, because it knows that's22a suitable architecture for Arm Cortex-M4.23Multilib can also choose between libraries for the same architecture based on24other options. For example if the user specifies ``-fno-exceptions`` then a25toolchain could select libraries built without exception support, thereby26reducing the size of the resulting binary.27 28Design29======30 31Clang supports GCC's ``-print-multi-lib`` and ``-print-multi-directory``32options. These are described in33`GCC Developer Options <https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/Developer-Options.html>`_.34 35There are two ways to configure multilib in Clang: hard-coded or via a36configuration file.37 38Hard-coded Multilib39===================40 41The available libraries can be hard-coded in Clang. Typically this is done42using the ``MultilibBuilder`` interface in43``clang/include/clang/Driver/MultilibBuilder.h``.44There are many examples of this in ``lib/Driver/ToolChains/Gnu.cpp``.45The remainder of this document will not focus on this type of multilib.46 47EXPERIMENTAL Multilib via configuration file48============================================49 50Some Clang toolchains support loading multilib configuration from a51``multilib.yaml`` configuration file.52 53A ``multilib.yaml`` configuration file specifies which multilib variants are54available, their relative location, what compilation options were used to build55them, and the criteria by which they are selected.56 57Multilib processing58===================59 60Clang goes through the following steps to use multilib from a configuration61file:62 63#. Normalize command line options. Clang can accept the same64 information via different options - for example,65 ``--target=arm-none-eabi -march=armv7-m`` and66 ``--target=armv7m-none-eabi`` are equivalent.67 Clang normalizes the command line before passing them to the multilib system.68 To see what flags are emitted for a given set of command line options, use69 the ``-print-multi-flags-experimental`` command line option70 along with the rest of the options you want to use.71#. Load ``multilib.yaml`` from sysroot.72#. Generate additional flags. ``multilib.yaml`` contains a ``Mappings`` section,73 which specifies how to generate additional flags based on the flags derived74 from command line options. Flags are matched using regular expressions.75 These regular expressions shall use the POSIX extended regular expression76 syntax.77#. Match flags against multilib variants. If the generated flags are a superset78 of the flags specified for a multilib variant then the variant is considered79 a match.80 If more than one variant matches then a toolchain may opt to either use only81 the *last* matching multilib variant, or may use all matching variants,82 thereby :ref:`layering<multilib-layering>` them.83#. Generate ``-isystem`` and ``-L`` options. Iterate in reverse order over84 the matching multilib variants, and generate ``-isystem`` and ``-L``85 options based on the multilib variant's directory.86 87.. _multilib-layering:88 89Multilib layering90=================91 92When Clang selects multilib variants, it may find that more than one variant93matches.94 95It is up to the ToolChain subclass to decide what to do in this case.96There are two options permitted:97 98#. Use only the *last* matching multilib variant. This option exists primarily99 for compatibility with the previous multilib design.100#. Use all matching variants, thereby layering them.101 102This decision is hard-coded per ToolChain subclass. The latter option is103preferred for ToolChain subclasses without backwards compatibility104requirements.105 106If the latter option is chosen then ``-isystem`` and ``-L`` options will be107generated for each matching multilib variant, in reverse order.108 109This means that the compiler or linker will find files in the last matching110multilib variant that has the given file.111This behaviour permits multilib variants with only a partial set of files.112This means a toolchain can be distributed with one base multilib variant113containing all system headers and includes, and more specialised multilib114variants containing only files that are different to those in the base variant.115 116For example, a multilib variant could be compiled with ``-fno-exceptions``.117This option doesn't affect the content of header files, nor does it affect the118C libraries. Therefore if multilib layering is supported by the ToolChain119subclass and a suitable base multilib variant is present then the120``-fno-exceptions`` multilib variant need only contain C++ libraries.121 122It is the responsibility of layered multilib authors to ensure that headers and123libraries in each layer are complete enough to mask any incompatibilities.124 125Multilib custom flags126=====================127 128Introduction129------------130 131The multilib mechanism supports library variants that correspond to target,132code generation or language command-line flags. Examples include ``--target``,133``-mcpu``, ``-mfpu``, ``-mbranch-protection``, ``-fno-rtti``. However, some library134variants are particular to features that do not correspond to any command-line135option. Multithreading and semihosting, for instance, have no associated136compiler option.137 138In order to support the selection of variants for which no compiler option139exists, the multilib specification includes the concept of *custom flags*.140These flags have no impact on code generation and are only used in the multilib141processing.142 143Multilib custom flags follow this format in the driver invocation:144 145::146 147 -fmultilib-flag=<value>148 149They are fed into the multilib system alongside the remaining flags.150 151Custom flag declarations152------------------------153 154Custom flags can be declared in the YAML file under the *Flags* section.155 156.. code-block:: yaml157 158 Flags:159 - Name: multithreaded160 Values:161 - Name: no-multithreaded162 MacroDefines: [__SINGLE_THREAD__]163 - Name: multithreaded164 Default: no-multithreaded165 166* Name: the name to categorize a flag.167* Values: a list of flag Values (defined below).168* Default: it specifies the name of the value this flag should take if not169 specified in the command-line invocation. It must be one value from the Values170 field.171 172Each flag *Value* is defined as:173 174* Name: name of the value. This is the string to be used in175 ``-fmultilib-flag=<string>``.176* MacroDefines: a list of strings to be used as macro definitions. Each string177 is fed into the driver as ``-D<string>``.178 179The namespace of flag values is common across all flags. This means that flag180value names must be unique.181 182Usage of custom flags in the *Variants* specifications183------------------------------------------------------184 185Library variants should list their requirement on one or more custom flags like186they do for any other flag. Each requirement must be listed as187``-fmultilib-flag=<value>``.188 189A variant that does not specify a requirement on one particular flag can be190matched against any value of that flag.191 192Stability193=========194 195Multilib via configuration file shall be considered an experimental feature196until LLVM 18, at which point ``-print-multi-flags-experimental``197should be renamed to ``-print-multi-flags``.198A toolchain can opt in to using this feature by including a ``multilib.yaml``199file in its distribution, once support for it is added in relevant ToolChain200subclasses.201Once stability is reached, flags emitted by ``-print-multi-flags``202should not be removed or changed, although new flags may be added.203 204Restrictions205============206 207Despite the name, multilib is used to locate both ``include`` and ``lib``208directories. Therefore it is important that consistent options are passed to209the Clang driver when both compiling and linking. Otherwise inconsistent210``include`` and ``lib`` directories may be used, and the results will be211undefined.212 213EXPERIMENTAL multilib.yaml214==========================215 216The below example serves as a small of a possible multilib, and documents217the available options.218 219For a more comprehensive example see220``clang/test/Driver/baremetal-multilib.yaml`` in the ``llvm-project`` sources.221 222.. code-block:: yaml223 224 # multilib.yaml225 226 # This format is experimental and is likely to change!227 228 # Syntax is YAML 1.2229 230 # This required field defines the version of the multilib.yaml format.231 # Clang will emit an error if this number is greater than its current multilib232 # version or if its major version differs, but will accept lesser minor233 # versions.234 MultilibVersion: 1.0235 236 # The rest of this file is in two parts:237 # 1. A list of multilib variants.238 # 2. A list of regular expressions that may match flags generated from239 # command line options, and further flags that shall be added if the240 # regular expression matches.241 # It is acceptable for the file to contain properties not documented here,242 # and these will be ignored by Clang.243 244 # List of multilib variants. Required.245 # The ordering of items in the variants list is important if more than one246 # variant can match the same set of flags. See the docs on multilib layering247 # for more info.248 Variants:249 250 # Example of a multilib variant targeting Arm v6-M.251 # Dir is the relative location of the directory containing the headers252 # and/or libraries.253 # Exactly how Dir is used is left up to the ToolChain subclass to define, but254 # typically it will be joined to the sysroot.255 - Dir: thumb/v6-m256 # List of one or more normalized command line options, as generated by Clang257 # from the command line options or from Mappings below.258 # Here, if the flags are a superset of {target=thumbv6m-unknown-none-eabi}259 # then this multilib variant will be considered a match.260 Flags: [--target=thumbv6m-unknown-none-eabi]261 262 # Similarly, a multilib variant targeting Arm v7-M with an FPU (floating263 # point unit).264 - Dir: thumb/v7-m265 # Here, the flags generated by Clang must be a superset of266 # {--target=thumbv7m-none-eabi, -mfpu=fpv4-sp-d16} for this multilib variant267 # to be a match.268 Flags: [--target=thumbv7m-none-eabi, -mfpu=fpv4-sp-d16]269 270 # If there is no multilib available for a particular set of flags, and the271 # other multilibs are not adequate fallbacks, then you can define a variant272 # record with an Error key in place of the Dir key.273 - Error: this multilib collection has no hard-float ABI support274 Flags: [--target=thumbv7m-none-eabi, -mfloat-abi=hard]275 276 277 # The second section of the file is a list of regular expressions that are278 # used to map from flags generated from command line options to custom flags.279 # This is optional.280 # Each regular expression must match a whole flag string.281 # Flags in the "Flags" list will be added if any flag generated from command282 # line options matches the regular expression.283 Mappings:284 285 # Set a "--target=thumbv7m-none-eabi" flag if the regular expression matches286 # any of the flags generated from the command line options.287 # Match is a POSIX extended regular expression string.288 - Match: --target=thumbv([7-9]|[1-9][0-9]+).*289 # Flags is a list of one or more strings.290 Flags: [--target=thumbv7m-none-eabi]291 292 # Custom flag declarations. Each item is a different declaration.293 Flags:294 # Name of the flag295 - Name: multithreaded296 # List of custom flag values297 Values:298 # Name of the custom flag value. To be used in -fmultilib-flag=<string>.299 - Name: no-multithreaded300 # Macro definitions. Useful for defining extra macros for building the301 # associated library variant(s).302 MacroDefines: [__SINGLE_THREAD__]303 - Name: multithreaded304 # Default flag value. If no value for this flag declaration is used in the305 # command-line, the multilib system will use this one. Must be equal to one306 # of the flag value names from this flag declaration.307 Default: no-multithreaded308 309Design principles310=================311 312Stable interface313----------------314 315``multilib.yaml`` and ``-print-multi-flags-experimental`` are new316interfaces to Clang. In order for them to be usable over time and across LLVM317versions their interfaces should be stable.318The new multilib system will be considered experimental in LLVM 17, but in319LLVM 18 it will be stable. In particular this is important to which multilib320selection flags Clang generates from command line options. Once a flag is321generated by a released version of Clang it may be used in ``multilib.yaml``322files that exist independently of the LLVM release cycle, and therefore323ceasing to generate the flag would be a breaking change and should be324avoided.325 326However, an exception is the normalization of ``-march``.327``-march`` for Arm architectures contains a list of enabled and disabled328extensions and this list is likely to grow. Therefore ``-march`` flags are329unstable.330 331Incomplete interface332--------------------333 334The new multilib system does multilib selection based on only a limited set of335command line options, and limits which flags can be used for multilib336selection. This is in order to avoid committing to too large an interface.337Later LLVM versions can add support for multilib selection from more command338line options as needed.339 340Extensible341----------342 343It is likely that the configuration format will need to evolve in future to344adapt to new requirements.345Using a format like YAML that supports key-value pairs helps here as it's346trivial to add new keys alongside existing ones.347 348Backwards compatibility349-----------------------350 351New versions of Clang should be able to use configuration written for earlier352Clang versions.353To avoid behaving in a way that may be subtly incorrect, Clang should be able354to detect if the configuration is too new and emit an error.355 356Forwards compatibility357----------------------358 359As an author of a multilib configuration, it should be possible to design the360configuration in such a way that it is likely to work well with future Clang361versions. For example, if a future version of Clang is likely to add support362for newer versions of an architecture and the architecture is known to be363designed for backwards compatibility then it should be possible to express364compatibility for such architecture versions in the multilib configuration.365 366Not GNU spec files367------------------368 369The GNU spec files standard is large and complex and there's little desire to370import that complexity to LLVM. It's also heavily oriented towards processing371command line argument strings which is hard to do correctly, hence the large372amount of logic dedicated to that task in the Clang driver. While compatibility373with GNU would bring benefits, the cost in this case is deemed too high.374 375Avoid re-inventing feature detection in the configuration376---------------------------------------------------------377 378A large amount of logic in the Clang driver is dedicated to inferring which379architectural features are available based on the given command line options.380It is neither desirable nor practical to repeat such logic in each multilib381configuration. Instead the configuration should be able to benefit from the382heavy lifting Clang already does to detect features.383 384Low maintenance385---------------386 387Multilib is a relatively small feature in the scheme of things so supporting it388should accordingly take little time. Where possible this should be achieved by389implementing it in terms of existing features in the LLVM codebase.390 391Minimal additional API surface392------------------------------393 394The greater the API surface, the greater the difficulty of keeping it stable.395Where possible the additional API surface should be kept small by defining it396in relation to existing APIs. An example of this is keeping a simple397relationship between flag names and command line options where possible.398Since the command line options are part of a stable API they are unlikely399to change, and therefore the flag names get the same stability.400 401Low compile-time overhead402-------------------------403 404If the process of selecting multilib directories must be done on every405invocation of the Clang driver then it must have a negligible impact on406overall compile time.407