233 lines · plain
1.. _libc_gpu_building:2 3======================4Building libs for GPUs5======================6 7.. contents:: Table of Contents8 :depth: 49 :local:10 11Building the GPU C library12==========================13 14This document will present recipes to build the LLVM C library targeting a GPU15architecture. The GPU build uses the same :ref:`cross build<full_cross_build>`16support as the other targets. However, the GPU target has the restriction that17it *must* be built with an up-to-date ``clang`` compiler. This is because the18GPU target uses several compiler extensions to target GPU architectures.19 20The LLVM C library currently supports two GPU targets. This is either21``nvptx64-nvidia-cuda`` for NVIDIA GPUs or ``amdgcn-amd-amdhsa`` for AMD GPUs.22Targeting these architectures is done through ``clang``'s cross-compiling23support using the ``--target=<triple>`` flag. The following sections will24describe how to build the GPU support specifically.25 26Once you have finished building, refer to :ref:`libc_gpu_usage` to get started27with the newly built C library.28 29Standard runtimes build30-----------------------31 32The simplest way to build the GPU libc is to use the existing LLVM runtimes33support. This will automatically handle bootstrapping an up-to-date ``clang``34compiler and using it to build the C library. The following CMake invocation35will instruct it to build the ``libc`` runtime targeting both AMD and NVIDIA36GPUs. The ``LIBC_GPU_BUILD`` option can also be enabled to add the relevant37arguments automatically.38 39.. code-block:: sh40 41 $> cd llvm-project # The llvm-project checkout42 $> mkdir build43 $> cd build44 $> cmake ../llvm -G Ninja \45 -DLLVM_ENABLE_PROJECTS="clang;lld" \46 -DLLVM_ENABLE_RUNTIMES="openmp;offload" \47 -DCMAKE_BUILD_TYPE=<Debug|Release> \ # Select build type48 -DCMAKE_INSTALL_PREFIX=<PATH> \ # Where the libraries will live49 -DRUNTIMES_nvptx64-nvidia-cuda_LLVM_ENABLE_RUNTIMES=libc \50 -DRUNTIMES_amdgcn-amd-amdhsa_LLVM_ENABLE_RUNTIMES=libc \51 -DLLVM_RUNTIME_TARGETS="default;amdgcn-amd-amdhsa;nvptx64-nvidia-cuda"52 $> ninja install53 54We need ``clang`` to build the GPU C library and ``lld`` to link AMDGPU55executables, so we enable them in ``LLVM_ENABLE_PROJECTS``. We add ``openmp`` to56``LLVM_ENABLED_RUNTIMES`` so it is built for the default target and provides57OpenMP support. We then set ``RUNTIMES_<triple>_LLVM_ENABLE_RUNTIMES`` to enable58``libc`` for the GPU targets. The ``LLVM_RUNTIME_TARGETS`` sets the enabled59targets to build, in this case we want the default target and the GPU targets.60Note that if ``libc`` were included in ``LLVM_ENABLE_RUNTIMES`` it would build61targeting the default host environment as well. Alternatively, you can point62your build towards the ``libc/cmake/caches/gpu.cmake`` cache file with ``-C``.63 64Runtimes cross build65--------------------66 67For users wanting more direct control over the build process, the build steps68can be done manually instead. This build closely follows the instructions in the69:ref:`main documentation<full_cross_build>` but is specialized for the GPU70build. We follow the same steps to first build the libc tools and a suitable71compiler. These tools must all be up-to-date with the libc source.72 73.. code-block:: sh74 75 $> cd llvm-project # The llvm-project checkout76 $> mkdir build-libc-tools # A different build directory for the build tools77 $> cd build-libc-tools78 $> HOST_C_COMPILER=<C compiler for the host> # For example "clang"79 $> HOST_CXX_COMPILER=<C++ compiler for the host> # For example "clang++"80 $> cmake ../llvm \81 -G Ninja \82 -DLLVM_ENABLE_PROJECTS="clang" \83 -DCMAKE_C_COMPILER=$HOST_C_COMPILER \84 -DCMAKE_CXX_COMPILER=$HOST_CXX_COMPILER \85 -DLLVM_LIBC_FULL_BUILD=ON \86 -DCMAKE_BUILD_TYPE=Release # Release suggested to make "clang" fast87 $> ninja # Build the 'clang' compiler88 89Once this has finished the build directory should contain the ``clang``90compiler executable. We will use the ``clang`` compiler to build the GPU code.91We use these tools to bootstrap the build out of the runtimes directory92targeting a GPU architecture.93 94.. code-block:: sh95 96 $> cd llvm-project # The llvm-project checkout97 $> mkdir build # A different build directory for the build tools98 $> cd build99 $> TARGET_TRIPLE=<amdgcn-amd-amdhsa or nvptx64-nvidia-cuda>100 $> TARGET_C_COMPILER=</path/to/clang>101 $> TARGET_CXX_COMPILER=</path/to/clang++>102 $> cmake ../runtimes \ # Point to the runtimes build103 -G Ninja \104 -DLLVM_ENABLE_RUNTIMES=libc \105 -DCMAKE_C_COMPILER=$TARGET_C_COMPILER \106 -DCMAKE_CXX_COMPILER=$TARGET_CXX_COMPILER \107 -DLLVM_LIBC_FULL_BUILD=ON \108 -DLLVM_RUNTIMES_TARGET=$TARGET_TRIPLE \109 -DCMAKE_BUILD_TYPE=Release110 $> ninja install111 112The above steps will result in a build targeting one of the supported GPU113architectures. Building for multiple targets requires separate CMake114invocations.115 116Standalone cross build117----------------------118 119The GPU build can also be targeted directly as long as the compiler used is a120supported ``clang`` compiler. This method is generally not recommended as it can121only target a single GPU architecture.122 123.. code-block:: sh124 125 $> cd llvm-project # The llvm-project checkout126 $> mkdir build # A different build directory for the build tools127 $> cd build128 $> CLANG_C_COMPILER=</path/to/clang> # Must be a trunk build129 $> CLANG_CXX_COMPILER=</path/to/clang++> # Must be a trunk build130 $> TARGET_TRIPLE=<amdgcn-amd-amdhsa or nvptx64-nvidia-cuda>131 $> cmake ../llvm \ # Point to the llvm directory132 -G Ninja \133 -DLLVM_ENABLE_PROJECTS=libc \134 -DCMAKE_C_COMPILER=$CLANG_C_COMPILER \135 -DCMAKE_CXX_COMPILER=$CLANG_CXX_COMPILER \136 -DLLVM_LIBC_FULL_BUILD=ON \137 -DLIBC_TARGET_TRIPLE=$TARGET_TRIPLE \138 -DCMAKE_BUILD_TYPE=Release139 $> ninja install140 141This will build and install the GPU C library along with all the other LLVM142libraries.143 144Build overview145==============146 147Once installed, the GPU build will create several files used for different148targets. This section will briefly describe their purpose.149 150**include/<target-triple>**151 The include directory where all of the generated headers for the target will152 go. These definitions are strictly for the GPU when being targeted directly.153 154**lib/clang/<llvm-major-version>/include/llvm-libc-wrappers/llvm-libc-decls**155 These are wrapper headers created for offloading languages like CUDA, HIP, or156 OpenMP. They contain functions supported in the GPU libc along with attributes157 and metadata that declare them on the target device and make them compatible158 with the host headers.159 160**lib/<target-triple>/libc.a**161 The main C library static archive containing LLVM-IR targeting the given GPU.162 It can be linked directly or inspected depending on the target support.163 164**lib/<target-triple>/libm.a**165 The C library static archive providing implementations of the standard math166 functions.167 168**lib/<target-triple>/libc.bc**169 An alternate form of the library provided as a single LLVM-IR bitcode blob.170 This can be used similarly to NVIDIA's or AMD's device libraries.171 172**lib/<target-triple>/libm.bc**173 An alternate form of the library provided as a single LLVM-IR bitcode blob174 containing the standard math functions.175 176**lib/<target-triple>/crt1.o**177 An LLVM-IR file containing startup code to call the ``main`` function on the178 GPU. This is used similarly to the standard C library startup object.179 180**bin/amdhsa-loader**181 A binary utility used to launch executables compiled targeting the AMD GPU.182 This will be included if the build system found the ``hsa-runtime64`` library183 either in ``/opt/rocm`` or the current CMake installation directory. This is184 required to build the GPU tests .See the :ref:`libc GPU usage<libc_gpu_usage>`185 for more information.186 187**bin/nvptx-loader**188 A binary utility used to launch executables compiled targeting the NVIDIA GPU.189 This will be included if the build system found the CUDA driver API. This is190 required for building tests.191 192**include/llvm-libc-rpc-server.h**193 A header file containing definitions that can be used to interface with the194 :ref:`RPC server<libc_gpu_rpc>`.195 196**lib/libllvmlibc_rpc_server.a**197 The static library containing the implementation of the RPC server. This can198 be used to enable host services for anyone looking to interface with the199 :ref:`RPC client<libc_gpu_rpc>`.200 201.. _gpu_cmake_options:202 203CMake options204=============205 206This section briefly lists a few of the CMake variables that specifically207control the GPU build of the C library. These options can be passed individually208to each target using ``-DRUNTIMES_<target>_<variable>=<value>`` when using a209standard runtime build.210 211**LLVM_LIBC_FULL_BUILD**:BOOL212 This flag controls whether or not the libc build will generate its own213 headers. This must always be on when targeting the GPU.214 215**LIBC_GPU_BUILD**:BOOL216 Shorthand for enabling GPU support. Equivalent to enabling support for both217 AMDGPU and NVPTX builds for ``libc``.218 219**LIBC_GPU_TEST_ARCHITECTURE**:STRING220 Sets the architecture used to build the GPU tests for, such as ``gfx90a`` or221 ``sm_80`` for AMD and NVIDIA GPUs respectively. The default behavior is to222 detect the system's GPU architecture using the ``native`` option. If this223 option is not set and a GPU was not detected the tests will not be built.224 225**LIBC_GPU_TEST_JOBS**:STRING226 Sets the number of threads used to run GPU tests. The GPU test suite will227 commonly run out of resources if this is not constrained so it is recommended228 to keep it low. The default value is a single thread.229 230**LIBC_GPU_LOADER_EXECUTABLE**:STRING231 Overrides the default loader used for running GPU tests. If this is not232 provided the standard one will be built.233