298 lines · plain
1============2HLSL Support3============4 5.. contents::6 :local:7 8Introduction9============10 11HLSL Support is under active development in the Clang codebase. This document12describes the high level goals of the project, the guiding principles, as well13as some idiosyncrasies of the HLSL language and how we intend to support them in14Clang.15 16Project Goals17=============18 19The long term goal of this project is to enable Clang to function as a20replacement for the `DirectXShaderCompiler (DXC)21<https://github.com/microsoft/DirectXShaderCompiler/>`_ in all its supported22use cases. Accomplishing that goal will require Clang to be able to process most23existing HLSL programs with a high degree of source compatibility.24 25Non-Goals26---------27 28HLSL ASTs do not need to be compatible between DXC and Clang. We do not expect29identical code generation or that features will resemble DXC's implementation or30architecture. In fact, we explicitly expect to deviate from DXC's implementation31in key ways.32 33Guiding Principles34==================35 36This document lacks details for architectural decisions that are not yet37finalized. Our top priorities are quality, maintainability, and flexibility. In38accordance with community standards we are expecting a high level of test39coverage, and we will engineer our solutions with long term maintenance in mind.40We are also working to limit modifications to the Clang C++ code paths and41share as much functionality as possible.42 43Architectural Direction44=======================45 46HLSL support in Clang is expressed as C++ minus unsupported C and C++ features.47This is different from how other Clang languages are implemented. Most languages48in Clang are additive on top of C.49 50HLSL is not a formally or fully specified language, and while our goals require51a high level of source compatibility, implementations can vary and we have some52flexibility to be more or less permissive in some cases. For modern HLSL DXC is53the reference implementation.54 55The HLSL effort prioritizes following similar patterns for other languages,56drivers, runtimes and targets. Specifically, We will maintain separation between57HSLS-specific code and the rest of Clang as much as possible following patterns58in use in Clang code today (i.e. ParseHLSL.cpp, SemaHLSL.cpp, CGHLSL*.cpp...).59We will use inline checks on language options where the code is simple and60isolated, and prefer HLSL-specific implementation files for any code of61reasonable complexity.62 63In places where the HLSL language is in conflict with C and C++, we will seek to64make minimally invasive changes guarded under the HLSL language options. We will65seek to make HLSL language support as minimal a maintenance burden as possible.66 67DXC Driver68----------69 70A DXC driver mode will provide command-line compatibility with DXC, supporting71DXC's options and flags. The DXC driver is HLSL-specific and will create an72HLSLToolchain which will provide the basis to support targeting both DirectX and73Vulkan.74 75Parser76------77 78Following the examples of other parser extensions HLSL will add a ParseHLSL.cpp79file to contain the implementations of HLSL-specific extensions to the Clang80parser. The HLSL grammar shares most of its structure with C and C++, so we will81use the existing C/C++ parsing code paths.82 83Sema84----85 86HLSL's Sema implementation will also provide an ``ExternalSemaSource``. In DXC,87an ``ExternalSemaSource`` is used to provide definitions for HLSL built-in data88types and built-in templates. Clang is already designed to allow an attached89``ExternalSemaSource`` to lazily complete data types, which is a **huge**90performance win for HLSL.91 92If precompiled headers are used when compiling HLSL, the ``ExternalSemaSource``93will be a ``MultiplexExternalSemaSource`` which includes both the ``ASTReader``94and -. For Built-in declarations that are already95completed in the serialized AST, the ``HLSLExternalSemaSource`` will reuse the96existing declarations and not introduce new declarations. If the built-in types97are not completed in the serialized AST, the ``HLSLExternalSemaSource`` will98create new declarations and connect the de-serialized decls as the previous99declaration.100 101CodeGen102-------103 104Like OpenCL, HLSL relies on capturing a lot of information into IR metadata.105*hand wave* *hand wave* *hand wave* As a design principle here we want our IR to106be idiomatic Clang IR as much as possible. We will use IR attributes wherever we107can, and use metadata as sparingly as possible. One example of a difference from108DXC already implemented in Clang is the use of target triples to communicate109shader model versions and shader stages.110 111Our HLSL CodeGen implementation should also have an eye toward generating IR112that will map directly to targets other than DXIL. While IR itself is generally113not re-targetable, we want to share the Clang CodeGen implementation for HLSL114with other GPU graphics targets like SPIR-V and possibly other GPU and even CPU115targets.116 117hlsl.h118------119 120HLSL has a library of standalone functions. This is similar to OpenCL and CUDA,121and is analogous to C's standard library. The implementation approach for the122HLSL library functionality draws from patterns in use by OpenCL and other Clang123resource headers. All of the clang resource headers are part of the124``ClangHeaders`` component found in the source tree under125`clang/lib/Headers <https://github.com/llvm/llvm-project/tree/main/clang/lib/Headers>`_.126 127.. note::128 129 HLSL's complex data types are not defined in HLSL's header because many of130 the semantics of those data types cannot be expressed in HLSL due to missing131 language features. Data types that can't be expressed in HLSL are defined in132 code in the ``HLSLExternalSemaSource``.133 134Similar to OpenCL, the HLSL library functionality is implicitly declared in135translation units without needing to include a header to provide declarations.136In Clang this is handled by making ``hlsl.h`` an implicitly included header137distributed as part of the Clang resource directory.138 139Similar to OpenCL, HLSL's implicit header will explicitly declare all overloads,140and each overload will map to a corresponding ``__builtin*`` compiler intrinsic141that is handled in ClangCodeGen. CUDA uses a similar pattern although many CUDA142functions have full definitions in the included headers which in turn call143corresponding ``__builtin*`` compiler intrinsics. By not having bodies HLSL144avoids the need for the inliner to clean up and inline large numbers of small145library functions.146 147HLSL's implicit headers also define some of HLSL's typedefs. This is consistent148with how the AVX vector header is implemented.149 150Concerns have been expressed that this approach may result in slower compile151times than the approach DXC uses where library functions are treated more like152Clang ``__builtin*`` intrinsics. No real world use cases have been identified153where parsing is a significant compile-time overhead, but the HLSL implicit154headers can be compiled into a module for performance if needed.155 156Further, by treating these as functions rather than ``__builtin*`` compiler157intrinsics, the language behaviors are more consistent and aligned with user158expectation because normal overload resolution rules and implicit conversions159apply as expected.160 161It is a feature of this design that clangd-powered "go to declaration" for162library functions will jump to a valid header declaration and all overloads will163be user readable.164 165HLSL Language166=============167 168The HLSL language is insufficiently documented, and not formally specified.169Documentation is available on `Microsoft's website170<https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl>`_.171The language syntax is similar enough to C and C++ that carefully written C and172C++ code is valid HLSL. HLSL has some key differences from C & C++ which we will173need to handle in Clang.174 175HLSL is not a conforming or valid extension or superset of C or C++. The176language has key incompatibilities with C and C++, both syntactically and177semantically.178 179An Aside on GPU Languages180-------------------------181 182Due to HLSL being a GPU targeted language HLSL is a Single Program Multiple Data183(SPMD) language relying on the implicit parallelism provided by GPU hardware.184Some language features in HLSL enable programmers to take advantage of the185parallel nature of GPUs in a hardware abstracted language.186 187HLSL also prohibits some features of C and C++ which can have catastrophic188performance or are not widely supportable on GPU hardware or drivers. As an189example, register spilling is often excessively expensive on GPUs, so HLSL190requires all functions to be inlined during code generation, and does not191support a runtime calling convention.192 193Pointers & References194---------------------195 196HLSL does not support referring to values by address. Semantically all variables197are value-types and behave as such. HLSL disallows the pointer dereference198operators (unary ``*``, and ``->``), as well as the address of operator (unary199&). While HLSL disallows pointers and references in the syntax, HLSL does use200reference types in the AST, and we intend to use pointer decay in the AST in201the Clang implementation.202 203HLSL ``this`` Keyword204---------------------205 206HLSL does support member functions, and (in HLSL 2021) limited operator207overloading. With member function support, HLSL also has a ``this`` keyword. The208``this`` keyword is an example of one of the places where HLSL relies on209references in the AST, because ``this`` is a reference.210 211Bitshifts212---------213 214In deviation from C, HLSL bitshifts are defined to mask the shift count by the215size of the type. In DXC, the semantics of LLVM IR were altered to accommodate216this, in Clang we intend to generate the mask explicitly in the IR. In cases217where the shift value is constant, this will be constant folded appropriately,218in other cases we can clean it up in the DXIL target.219 220Non-short Circuiting Logical Operators221--------------------------------------222 223In HLSL 2018 and earlier, HLSL supported logical operators (and the ternary224operator) on vector types. This behavior required that operators not short225circuit. The non-short circuiting behavior applies to all data types until HLSL2262021. In HLSL 2021, logical and ternary operators do not support vector types227instead builtin functions ``and``, ``or`` and ``select`` are available, and228operators short circuit matching C behavior.229 230Precise Qualifier231-----------------232 233HLSL has a ``precise`` qualifier that behaves unlike anything else in the C234language. The support for this qualifier in DXC is buggy, so our bar for235compatibility is low.236 237The ``precise`` qualifier applies in the inverse direction from normal238qualifiers. Rather than signifying that the declaration containing ``precise``239qualifier be precise, it signifies that the operations contributing to the240declaration's value be ``precise``. Additionally, ``precise`` is a misnomer:241values attributed as ``precise`` comply with IEEE-754 floating point semantics,242and are prevented from optimizations which could decrease *or increase*243precision.244 245Differences in Templates246------------------------247 248HLSL uses templates to define builtin types and methods, but disallowed249user-defined templates until HLSL 2021. HLSL also allows omitting empty template250parameter lists when all template parameters are defaulted. This is an ambiguous251syntax in C++, but Clang detects the case and issues a diagnostic. This makes252supporting the case in Clang minimally invasive.253 254Vector Extensions255-----------------256 257HLSL uses the OpenCL vector extensions, and also provides C++-style constructors258for vectors that are not supported by Clang.259 260Standard Library261----------------262 263HLSL does not support the C or C++ standard libraries. Like OpenCL, HLSL264describes its own library of built in types, complex data types, and functions.265 266Unsupported C & C++ Features267----------------------------268 269HLSL does not support all features of C and C++. In implementing HLSL in Clang270use of some C and C++ features will produce diagnostics under HLSL, and others271will be supported as language extensions. In general, any C or C++ feature that272can be supported by the DXIL and SPIR-V code generation targets could be treated273as a clang HLSL extension. Features that cannot be lowered to DXIL or SPIR-V,274must be diagnosed as errors.275 276HLSL does not support the following C features:277 278* Pointers279* References280* ``goto`` or labels281* Variable Length Arrays282* ``_Complex`` and ``_Imaginary``283* C Threads or Atomics (or Obj-C blocks)284* ``union`` types `(in progress for HLSL 202x) <https://github.com/microsoft/DirectXShaderCompiler/pull/4132>`_285* Most features C11 and later286 287HLSL does not support the following C++ features:288 289* RTTI290* Exceptions291* Multiple inheritance292* Access specifiers293* Anonymous or inline namespaces294* ``new`` & ``delete`` operators in all of their forms (array, placement, etc)295* Constructors and destructors296* Any use of the ``virtual`` keyword297* Most features C++11 and later298