227 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3============4x86 Topology5============6 7This documents and clarifies the main aspects of x86 topology modelling and8representation in the kernel. Update/change when doing changes to the9respective code.10 11The architecture-agnostic topology definitions are in12Documentation/admin-guide/cputopology.rst. This file holds x86-specific13differences/specialities which must not necessarily apply to the generic14definitions. Thus, the way to read up on Linux topology on x86 is to start15with the generic one and look at this one in parallel for the x86 specifics.16 17Needless to say, code should use the generic functions - this file is *only*18here to *document* the inner workings of x86 topology.19 20Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>.21 22The main aim of the topology facilities is to present adequate interfaces to23code which needs to know/query/use the structure of the running system wrt24threads, cores, packages, etc.25 26The kernel does not care about the concept of physical sockets because a27socket has no relevance to software. It's an electromechanical component. In28the past a socket always contained a single package (see below), but with the29advent of Multi Chip Modules (MCM) a socket can hold more than one package. So30there might be still references to sockets in the code, but they are of31historical nature and should be cleaned up.32 33The topology of a system is described in the units of:34 35 - packages36 - cores37 - threads38 39Package40=======41Packages contain a number of cores plus shared resources, e.g. DRAM42controller, shared caches etc.43 44Modern systems may also use the term 'Die' for package.45 46AMD nomenclature for package is 'Node'.47 48Package-related topology information in the kernel:49 50 - topology_num_threads_per_package()51 52 The number of threads in a package.53 54 - topology_num_cores_per_package()55 56 The number of cores in a package.57 58 - topology_max_dies_per_package()59 60 The maximum number of dies in a package.61 62 - cpuinfo_x86.topo.die_id:63 64 The physical ID of the die.65 66 - cpuinfo_x86.topo.pkg_id:67 68 The physical ID of the package. This information is retrieved via CPUID69 and deduced from the APIC IDs of the cores in the package.70 71 Modern systems use this value for the socket. There may be multiple72 packages within a socket. This value may differ from topo.die_id.73 74 - cpuinfo_x86.topo.logical_pkg_id:75 76 The logical ID of the package. As we do not trust BIOSes to enumerate the77 packages in a consistent way, we introduced the concept of logical package78 ID so we can sanely calculate the number of maximum possible packages in79 the system and have the packages enumerated linearly.80 81 - topology_max_packages():82 83 The maximum possible number of packages in the system. Helpful for per84 package facilities to preallocate per package information.85 86 - cpuinfo_x86.topo.llc_id:87 88 - On Intel, the first APIC ID of the list of CPUs sharing the Last Level89 Cache90 91 - On AMD, the Node ID or Core Complex ID containing the Last Level92 Cache. In general, it is a number identifying an LLC uniquely on the93 system.94 95Cores96=====97A core consists of 1 or more threads. It does not matter whether the threads98are SMT- or CMT-type threads.99 100AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses101"core".102 103Threads104=======105A thread is a single scheduling unit. It's the equivalent to a logical Linux106CPU.107 108AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always109uses "thread".110 111Thread-related topology information in the kernel:112 113 - topology_core_cpumask():114 115 The cpumask contains all online threads in the package to which a thread116 belongs.117 118 The number of online threads is also printed in /proc/cpuinfo "siblings."119 120 - topology_sibling_cpumask():121 122 The cpumask contains all online threads in the core to which a thread123 belongs.124 125 - topology_logical_package_id():126 127 The logical package ID to which a thread belongs.128 129 - topology_physical_package_id():130 131 The physical package ID to which a thread belongs.132 133 - topology_core_id();134 135 The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo136 "core_id."137 138 139 140System topology examples141========================142 143.. note::144 The alternative Linux CPU enumeration depends on how the BIOS enumerates the145 threads. Many BIOSes enumerate all threads 0 first and then all threads 1.146 That has the "advantage" that the logical Linux CPU numbers of threads 0 stay147 the same whether threads are enabled or not. That's merely an implementation148 detail and has no practical impact.149 1501) Single Package, Single Core::151 152 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0153 1542) Single Package, Dual Core155 156 a) One thread per core::157 158 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0159 -> [core 1] -> [thread 0] -> Linux CPU 1160 161 b) Two threads per core::162 163 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0164 -> [thread 1] -> Linux CPU 1165 -> [core 1] -> [thread 0] -> Linux CPU 2166 -> [thread 1] -> Linux CPU 3167 168 Alternative enumeration::169 170 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0171 -> [thread 1] -> Linux CPU 2172 -> [core 1] -> [thread 0] -> Linux CPU 1173 -> [thread 1] -> Linux CPU 3174 175 AMD nomenclature for CMT systems::176 177 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0178 -> [Compute Unit Core 1] -> Linux CPU 1179 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2180 -> [Compute Unit Core 1] -> Linux CPU 3181 1824) Dual Package, Dual Core183 184 a) One thread per core::185 186 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0187 -> [core 1] -> [thread 0] -> Linux CPU 1188 189 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2190 -> [core 1] -> [thread 0] -> Linux CPU 3191 192 b) Two threads per core::193 194 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0195 -> [thread 1] -> Linux CPU 1196 -> [core 1] -> [thread 0] -> Linux CPU 2197 -> [thread 1] -> Linux CPU 3198 199 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4200 -> [thread 1] -> Linux CPU 5201 -> [core 1] -> [thread 0] -> Linux CPU 6202 -> [thread 1] -> Linux CPU 7203 204 Alternative enumeration::205 206 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0207 -> [thread 1] -> Linux CPU 4208 -> [core 1] -> [thread 0] -> Linux CPU 1209 -> [thread 1] -> Linux CPU 5210 211 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2212 -> [thread 1] -> Linux CPU 6213 -> [core 1] -> [thread 0] -> Linux CPU 3214 -> [thread 1] -> Linux CPU 7215 216 AMD nomenclature for CMT systems::217 218 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0219 -> [Compute Unit Core 1] -> Linux CPU 1220 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2221 -> [Compute Unit Core 1] -> Linux CPU 3222 223 [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4224 -> [Compute Unit Core 1] -> Linux CPU 5225 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6226 -> [Compute Unit Core 1] -> Linux CPU 7227