214 lines · plain
1====================2High Memory Handling3====================4 5By: Peter Zijlstra <a.p.zijlstra@chello.nl>6 7.. contents:: :local:8 9What Is High Memory?10====================11 12High memory (highmem) is used when the size of physical memory approaches or13exceeds the maximum size of virtual memory. At that point it becomes14impossible for the kernel to keep all of the available physical memory mapped15at all times. This means the kernel needs to start using temporary mappings of16the pieces of physical memory that it wants to access.17 18The part of (physical) memory not covered by a permanent mapping is what we19refer to as 'highmem'. There are various architecture dependent constraints on20where exactly that border lies.21 22In the i386 arch, for example, we choose to map the kernel into every process's23VM space so that we don't have to pay the full TLB invalidation costs for24kernel entry/exit. This means the available virtual memory space (4GiB on25i386) has to be divided between user and kernel space.26 27The traditional split for architectures using this approach is 3:1, 3GiB for28userspace and the top 1GiB for kernel space::29 30 +--------+ 0xffffffff31 | Kernel |32 +--------+ 0xc000000033 | |34 | User |35 | |36 +--------+ 0x0000000037 38This means that the kernel can at most map 1GiB of physical memory at any one39time, but because we need virtual address space for other things - including40temporary maps to access the rest of the physical memory - the actual direct41map will typically be less (usually around ~896MiB).42 43Other architectures that have mm context tagged TLBs can have separate kernel44and user maps. Some hardware (like some ARMs), however, have limited virtual45space when they use mm context tags.46 47 48Temporary Virtual Mappings49==========================50 51The kernel contains several ways of creating temporary mappings. The following52list shows them in order of preference of use.53 54* kmap_local_page(), kmap_local_folio() - These functions are used to create55 short term mappings. They can be invoked from any context (including56 interrupts) but the mappings can only be used in the context which acquired57 them. The only differences between them consist in the first taking a pointer58 to a struct page and the second taking a pointer to struct folio and the byte59 offset within the folio which identifies the page.60 61 These functions should always be used, whereas kmap_atomic() and kmap() have62 been deprecated.63 64 These mappings are thread-local and CPU-local, meaning that the mapping65 can only be accessed from within this thread and the thread is bound to the66 CPU while the mapping is active. Although preemption is never disabled by67 this function, the CPU can not be unplugged from the system via68 CPU-hotplug until the mapping is disposed.69 70 It's valid to take pagefaults in a local kmap region, unless the context71 in which the local mapping is acquired does not allow it for other reasons.72 73 As said, pagefaults and preemption are never disabled. There is no need to74 disable preemption because, when context switches to a different task, the75 maps of the outgoing task are saved and those of the incoming one are76 restored.77 78 kmap_local_page(), as well as kmap_local_folio() always returns valid virtual79 kernel addresses and it is assumed that kunmap_local() will never fail.80 81 On CONFIG_HIGHMEM=n kernels and for low memory pages they return the82 virtual address of the direct mapping. Only real highmem pages are83 temporarily mapped. Therefore, users may call a plain page_address()84 for pages which are known to not come from ZONE_HIGHMEM. However, it is85 always safe to use kmap_local_{page,folio}() / kunmap_local().86 87 While they are significantly faster than kmap(), for the highmem case they88 come with restrictions about the pointers validity. Contrary to kmap()89 mappings, the local mappings are only valid in the context of the caller90 and cannot be handed to other contexts. This implies that users must91 be absolutely sure to keep the use of the return address local to the92 thread which mapped it.93 94 Most code can be designed to use thread local mappings. User should95 therefore try to design their code to avoid the use of kmap() by mapping96 pages in the same thread the address will be used and prefer97 kmap_local_page() or kmap_local_folio().98 99 Nesting kmap_local_page() and kmap_atomic() mappings is allowed to a certain100 extent (up to KMAP_TYPE_NR) but their invocations have to be strictly ordered101 because the map implementation is stack based. See kmap_local_page() kdocs102 (included in the "Functions" section) for details on how to manage nested103 mappings.104 105* kmap_atomic(). This function has been deprecated; use kmap_local_page().106 107 NOTE: Conversions to kmap_local_page() must take care to follow the mapping108 restrictions imposed on kmap_local_page(). Furthermore, the code between109 calls to kmap_atomic() and kunmap_atomic() may implicitly depend on the side110 effects of atomic mappings, i.e. disabling page faults or preemption, or both.111 In that case, explicit calls to pagefault_disable() or preempt_disable() or112 both must be made in conjunction with the use of kmap_local_page().113 114 [Legacy documentation]115 116 This permits a very short duration mapping of a single page. Since the117 mapping is restricted to the CPU that issued it, it performs well, but118 the issuing task is therefore required to stay on that CPU until it has119 finished, lest some other task displace its mappings.120 121 kmap_atomic() may also be used by interrupt contexts, since it does not122 sleep and the callers too may not sleep until after kunmap_atomic() is123 called.124 125 Each call of kmap_atomic() in the kernel creates a non-preemptible section126 and disable pagefaults. This could be a source of unwanted latency. Therefore127 users should prefer kmap_local_page() instead of kmap_atomic().128 129 It is assumed that k[un]map_atomic() won't fail.130 131* kmap(). This function has been deprecated; use kmap_local_page().132 133 NOTE: Conversions to kmap_local_page() must take care to follow the mapping134 restrictions imposed on kmap_local_page(). In particular, it is necessary to135 make sure that the kernel virtual memory pointer is only valid in the thread136 that obtained it.137 138 [Legacy documentation]139 140 This should be used to make short duration mapping of a single page with no141 restrictions on preemption or migration. It comes with an overhead as mapping142 space is restricted and protected by a global lock for synchronization. When143 mapping is no longer needed, the address that the page was mapped to must be144 released with kunmap().145 146 Mapping changes must be propagated across all the CPUs. kmap() also147 requires global TLB invalidation when the kmap's pool wraps and it might148 block when the mapping space is fully utilized until a slot becomes149 available. Therefore, kmap() is only callable from preemptible context.150 151 All the above work is necessary if a mapping must last for a relatively152 long time but the bulk of high-memory mappings in the kernel are153 short-lived and only used in one place. This means that the cost of154 kmap() is mostly wasted in such cases. kmap() was not intended for long155 term mappings but it has morphed in that direction and its use is156 strongly discouraged in newer code and the set of the preceding functions157 should be preferred.158 159 On 64-bit systems, calls to kmap_local_page(), kmap_atomic() and kmap() have160 no real work to do because a 64-bit address space is more than sufficient to161 address all the physical memory whose pages are permanently mapped.162 163* vmap(). This can be used to make a long duration mapping of multiple164 physical pages into a contiguous virtual space. It needs global165 synchronization to unmap.166 167 168Cost of Temporary Mappings169==========================170 171The cost of creating temporary mappings can be quite high. The arch has to172manipulate the kernel's page tables, the data TLB and/or the MMU's registers.173 174If CONFIG_HIGHMEM is not set, then the kernel will try and create a mapping175simply with a bit of arithmetic that will convert the page struct address into176a pointer to the page contents rather than juggling mappings about. In such a177case, the unmap operation may be a null operation.178 179If CONFIG_MMU is not set, then there can be no temporary mappings and no180highmem. In such a case, the arithmetic approach will also be used.181 182 183i386 PAE184========185 186The i386 arch, under some circumstances, will permit you to stick up to 64GiB187of RAM into your 32-bit machine. This has a number of consequences:188 189* Linux needs a page-frame structure for each page in the system and the190 pageframes need to live in the permanent mapping, which means:191 192* you can have 896M/sizeof(struct page) page-frames at most; with struct193 page being 32-bytes that would end up being something in the order of 112G194 worth of pages; the kernel, however, needs to store more than just195 page-frames in that memory...196 197* PAE makes your page tables larger - which slows the system down as more198 data has to be accessed to traverse in TLB fills and the like. One199 advantage is that PAE has more PTE bits and can provide advanced features200 like NX and PAT.201 202The general recommendation is that you don't use more than 8GiB on a 32-bit203machine - although more might work for you and your workload, you're pretty204much on your own - don't expect kernel developers to really care much if things205come apart.206 207 208Functions209=========210 211.. kernel-doc:: include/linux/highmem.h212.. kernel-doc:: mm/highmem.c213.. kernel-doc:: include/linux/highmem-internal.h214