163 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=============4Multi-Gen LRU5=============6The multi-gen LRU is an alternative LRU implementation that optimizes7page reclaim and improves performance under memory pressure. Page8reclaim decides the kernel's caching policy and ability to overcommit9memory. It directly impacts the kswapd CPU usage and RAM efficiency.10 11Quick start12===========13Build the kernel with the following configurations.14 15* ``CONFIG_LRU_GEN=y``16* ``CONFIG_LRU_GEN_ENABLED=y``17 18All set!19 20Runtime options21===============22``/sys/kernel/mm/lru_gen/`` contains stable ABIs described in the23following subsections.24 25Kill switch26-----------27``enabled`` accepts different values to enable or disable the28following components. Its default value depends on29``CONFIG_LRU_GEN_ENABLED``. All the components should be enabled30unless some of them have unforeseen side effects. Writing to31``enabled`` has no effect when a component is not supported by the32hardware, and valid values will be accepted even when the main switch33is off.34 35====== ===============================================================36Values Components37====== ===============================================================380x0001 The main switch for the multi-gen LRU.390x0002 Clearing the accessed bit in leaf page table entries in large40 batches, when MMU sets it (e.g., on x86). This behavior can41 theoretically worsen lock contention (mmap_lock). If it is42 disabled, the multi-gen LRU will suffer a minor performance43 degradation for workloads that contiguously map hot pages,44 whose accessed bits can be otherwise cleared by fewer larger45 batches.460x0004 Clearing the accessed bit in non-leaf page table entries as47 well, when MMU sets it (e.g., on x86). This behavior was not48 verified on x86 varieties other than Intel and AMD. If it is49 disabled, the multi-gen LRU will suffer a negligible50 performance degradation.51[yYnN] Apply to all the components above.52====== ===============================================================53 54E.g.,55::56 57 echo y >/sys/kernel/mm/lru_gen/enabled58 cat /sys/kernel/mm/lru_gen/enabled59 0x000760 echo 5 >/sys/kernel/mm/lru_gen/enabled61 cat /sys/kernel/mm/lru_gen/enabled62 0x000563 64Thrashing prevention65--------------------66Personal computers are more sensitive to thrashing because it can67cause janks (lags when rendering UI) and negatively impact user68experience. The multi-gen LRU offers thrashing prevention to the69majority of laptop and desktop users who do not have ``oomd``.70 71Users can write ``N`` to ``min_ttl_ms`` to prevent the working set of72``N`` milliseconds from getting evicted. The OOM killer is triggered73if this working set cannot be kept in memory. In other words, this74option works as an adjustable pressure relief valve, and when open, it75terminates applications that are hopefully not being used.76 77Based on the average human detectable lag (~100ms), ``N=1000`` usually78eliminates intolerable janks due to thrashing. Larger values like79``N=3000`` make janks less noticeable at the risk of premature OOM80kills.81 82The default value ``0`` means disabled.83 84Experimental features85=====================86``/sys/kernel/debug/lru_gen`` accepts commands described in the87following subsections. Multiple command lines are supported, so does88concatenation with delimiters ``,`` and ``;``.89 90``/sys/kernel/debug/lru_gen_full`` provides additional stats for91debugging. ``CONFIG_LRU_GEN_STATS=y`` keeps historical stats from92evicted generations in this file.93 94Working set estimation95----------------------96Working set estimation measures how much memory an application needs97in a given time interval, and it is usually done with little impact on98the performance of the application. E.g., data centers want to99optimize job scheduling (bin packing) to improve memory utilizations.100When a new job comes in, the job scheduler needs to find out whether101each server it manages can allocate a certain amount of memory for102this new job before it can pick a candidate. To do so, the job103scheduler needs to estimate the working sets of the existing jobs.104 105When it is read, ``lru_gen`` returns a histogram of numbers of pages106accessed over different time intervals for each memcg and node.107``MAX_NR_GENS`` decides the number of bins for each histogram. The108histograms are noncumulative.109::110 111 memcg memcg_id memcg_path112 node node_id113 min_gen_nr age_in_ms nr_anon_pages nr_file_pages114 ...115 max_gen_nr age_in_ms nr_anon_pages nr_file_pages116 117Each bin contains an estimated number of pages that have been accessed118within ``age_in_ms``. E.g., ``min_gen_nr`` contains the coldest pages119and ``max_gen_nr`` contains the hottest pages, since ``age_in_ms`` of120the former is the largest and that of the latter is the smallest.121 122Users can write the following command to ``lru_gen`` to create a new123generation ``max_gen_nr+1``:124 125 ``+ memcg_id node_id max_gen_nr [can_swap [force_scan]]``126 127``can_swap`` defaults to the swap setting and, if it is set to ``1``,128it forces the scan of anon pages when swap is off, and vice versa.129``force_scan`` defaults to ``1`` and, if it is set to ``0``, it130employs heuristics to reduce the overhead, which is likely to reduce131the coverage as well.132 133A typical use case is that a job scheduler runs this command at a134certain time interval to create new generations, and it ranks the135servers it manages based on the sizes of their cold pages defined by136this time interval.137 138Proactive reclaim139-----------------140Proactive reclaim induces page reclaim when there is no memory141pressure. It usually targets cold pages only. E.g., when a new job142comes in, the job scheduler wants to proactively reclaim cold pages on143the server it selected, to improve the chance of successfully landing144this new job.145 146Users can write the following command to ``lru_gen`` to evict147generations less than or equal to ``min_gen_nr``.148 149 ``- memcg_id node_id min_gen_nr [swappiness [nr_to_reclaim]]``150 151``min_gen_nr`` should be less than ``max_gen_nr-1``, since152``max_gen_nr`` and ``max_gen_nr-1`` are not fully aged (equivalent to153the active list) and therefore cannot be evicted. ``swappiness``154overrides the default value in ``/proc/sys/vm/swappiness``.155``nr_to_reclaim`` limits the number of pages to evict.156 157A typical use case is that a job scheduler runs this command before it158tries to land a new job on a server. If it fails to materialize enough159cold pages because of the overestimation, it retries on the next160server according to the ranking result obtained from the working set161estimation step. This less forceful approach limits the impacts on the162existing jobs.163