173 lines · plain
1===============================================2Architecture and Design of DXIL Support in LLVM3===============================================4 5.. contents::6 :local:7 8.. toctree::9 :hidden:10 11Introduction12============13 14LLVM supports reading and writing the `DirectX Intermediate Language.15<https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst>`_,16or DXIL. DXIL is essentially LLVM 3.7 era bitcode with some17restrictions and various semantically important operations and18metadata.19 20LLVM's implementation philosophy for DXIL support is to treat DXIL as21merely a representation format as much as possible. When reading DXIL,22we should translate everything to generic LLVM constructs when23possible. Similarly, we should introduce DXIL-specific constructs as24late as possible in the process of lowering to the format.25 26There are three places to look for DXIL related code in LLVM: The27`DirectX` backend, for writing DXIL; The `DXILUpgrade` pass, for28reading; and in library code that is shared between writing and29reading. We'll describe these in reverse order.30 31Common Code for Reading and Writing32===================================33 34There's quite a bit of logic that needs to be shared between reading35and writing DXIL in order to avoid code duplication. While we don't36have a hard and fast rule about where such code should live, there are37generally three sensible places. Simple definitions of enums and38values that must stay fixed to match DXIL's ABI can be found in39`Support/DXILABI.h`, utilities to translate bidirectionally between40DXIL and modern LLVM constructs live in `lib/Transforms/Utils`, and41more analyses that are needed to derive or preserve information are42implemented as typical `lib/Analysis` passes.43 44The DXILUpgrade Pass45====================46 47Translating DXIL to LLVM IR takes advantage of the fact that DXIL is48compatible with LLVM 3.7 bitcode, and that modern LLVM is capable of49"upgrading" older bitcode into modern IR. Simply relying on the50bitcode upgrade process isn't sufficient though, since that leaves a51number of DXIL specific constructs around. Thus, we have the52`DXILUpgrade` pass to transform DXIL operations to LLVM operations and53smooth over differences in metadata representation. We call this pass54"upgrade" to reflect that it follows LLVM's standard bitcode upgrade55process and simply finishes the job for DXIL constructs - while56"reader" or "lifting" might also be reasonable names, they could be a57bit misleading.58 59The `DXILUpgrade` pass itself is fairly lightweight. It mostly relies60on the utilities described in "Common Code" above in order to share61logic with both the DirectX backend and with Clang's codegen of HLSL62support as much as possible.63 64The DirectX Intrinsic Expansion Pass65====================================66There are intrinsics that don't map directly to DXIL Ops. In some cases67an intrinsic needs to be expanded to a set of LLVM IR instructions. In68other cases an intrinsic needs modifications to the arguments or return69values of a DXIL Op. The `DXILIntrinsicExpansion` pass handles all 70the cases where our intrinsics don't have a one to one mapping. This 71pass may also be used when the expansion is specific to DXIL to keep 72implementation details out of CodeGen. Finally, there is an expectation 73that we maintain vector types through this pass. Therefore, best 74practice would be to avoid scalarization in this pass.75 76 77The DirectX Backend78===================79 80The DirectX backend lowers LLVM IR into DXIL. As we're transforming to81an intermediate format rather than a specific ISA, this backend does82not follow the instruction selection patterns you might be familiar83with from other backends. There are two parts to lowering DXIL - a set84of passes that mutate various constructs into a form that matches how85DXIL represents those constructs, followed by a limited bitcode86"downgrader pass".87 88Before emitting DXIL, the DirectX backend needs to modify the LLVM IR89such that external operations, types, and metadata is represented in90the way that DXIL expects. For example, `DXILOpLowering` translates91intrinsics into `dx.op` calls. These passes are essentially the92inverse of the `DXILUpgrade` pass. It's best to do this downgrading93process as IR to IR passes when possible, as that means that they can94be easily tested with `opt` and `FileCheck` without the need for95external tooling.96 97The second part of DXIL emission is more or less an LLVM bitcode98downgrader. We need to emit bitcode that matches the LLVM 3.799representation. For this, we have `DXILWriter`, which is an alternate100version of LLVM's `BitcodeWriter`. At present, this is able to101leverage LLVM's current bitcode libraries to do a lot of the work, but102it's possible that at some point in the future it will need to be103completely separate as modern LLVM bitcode evolves.104 105DirectX Backend Flow106--------------------107 108The code generation flow for DXIL is broken into a series of passes. The passes109are grouped into two flows:110 111#. Generating DXIL IR.112#. Generating DXIL Binary.113 114The passes to generate DXIL IR follow the flow:115 116 DXILOpLowering -> DXILPrepare -> DXILTranslateMetadata117 118Each of these passes has a defined responsibility:119 120#. DXILOpLowering translates LLVM intrinsic calls to dx.op calls.121#. DXILPrepare updates functions in the DXIL IR to be compatible with LLVM 3.7,122 namely removing attributes, and inserting bitcasts to allow typed pointers123 to be inserted.124#. DXILTranslateMetadata transforms and emits all recognized DXIL Metadata.125 126The passes to encode DXIL to binary in the DX Container follow the flow:127 128 DXILEmbedder -> DXContainerGlobals -> AsmPrinter129 130Each of these passes have the following defined responsibilities:131 132#. DXILEmbedder runs the DXIL bitcode writer to generate a bitcode stream and133 embeds the binary data inside a global in the original module.134#. DXContainerGlobals generates binary data globals for the other DX Container135 parts based on computed analysis passes.136#. AsmPrinter is the standard LLVM infrastructure for emitting object files.137 138When emitting DXIL into a DX Container file the MC layer is used in a similar139way to how the Clang ``-fembed-bitcode`` option operates. The DX Container140object writer knows how to construct the headers and structural fields of the141container, and reads global variables from the module to fill in the remaining142part data.143 144DirectX Container145-----------------146 147The DirectX container format is treated in LLVM as an object file format.148Reading is implemented between the BinaryFormat and Object libraries, and149writing is implemented in the MC layer. Additional testing and inspection150support are implemented in the ObjectYAML library and tools.151 152Testing153=======154 155A lot of DXIL testing can be done with typical IR to IR tests using156`opt` and `FileCheck`, since a lot of the support is implemented in157terms of IR level passes as described in the previous sections. You158can see examples of this in `llvm/test/CodeGen/DirectX` as well as159`llvm/test/Transforms/DXILUpgrade`, and this type of testing should be160leveraged as much as possible.161 162However, when it comes to testing the DXIL format itself, IR passes163are insufficient for testing. For now, the best option we have164available is using the DXC project's tools in order to round trip.165These tests are currently found in `test/tools/dxil-dis` and are only166available if the `LLVM_INCLUDE_DXIL_TESTS` cmake option is set. Note167that we do not currently have the equivalent testing set up for the168DXIL reading path.169 170As soon as we are able, we will also want to round trip using the DXIL171writing and reading paths in order to ensure self consistency and to172get test coverage when `dxil-dis` isn't available.173