302 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=======================4DAMON-based Reclamation5=======================6 7DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to8be used for proactive and lightweight reclamation under light memory pressure.9It doesn't aim to replace the LRU-list based page_granularity reclamation, but10to be selectively used for different level of memory pressure and requirements.11 12Where Proactive Reclamation is Required?13========================================14 15On general memory over-committed systems, proactively reclaiming cold pages16helps saving memory and reducing latency spikes that incurred by the direct17reclaim of the process or CPU consumption of kswapd, while incurring only18minimal performance degradation [1]_ [2]_ .19 20Free Pages Reporting [3]_ based memory over-commit virtualization systems are21good example of the cases. In such systems, the guest VMs reports their free22memory to host, and the host reallocates the reported memory to other guests.23As a result, the memory of the systems are fully utilized. However, the24guests could be not so memory-frugal, mainly because some kernel subsystems and25user-space applications are designed to use as much memory as available. Then,26guests could report only small amount of memory as free to host, results in27memory utilization drop of the systems. Running the proactive reclamation in28guests could mitigate this problem.29 30How It Works?31=============32 33DAMON_RECLAIM finds memory regions that didn't accessed for specific time34duration and page out. To avoid it consuming too much CPU for the paging out35operation, a speed limit can be configured. Under the speed limit, it pages36out memory regions that didn't accessed longer time first. System37administrators can also configure under what situation this scheme should38automatically activated and deactivated with three memory pressure watermarks.39 40Interface: Module Parameters41============================42 43To use this feature, you should first ensure your system is running on a kernel44that is built with ``CONFIG_DAMON_RECLAIM=y``.45 46To let sysadmins enable or disable it and tune for the given system,47DAMON_RECLAIM utilizes module parameters. That is, you can put48``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write49proper values to ``/sys/module/damon_reclaim/parameters/<parameter>`` files.50 51Below are the description of each parameter.52 53enabled54-------55 56Enable or disable DAMON_RECLAIM.57 58You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``.59Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do60no real monitoring and reclamation due to the watermarks-based activation61condition. Refer to below descriptions for the watermarks parameter for this.62 63commit_inputs64-------------65 66Make DAMON_RECLAIM reads the input parameters again, except ``enabled``.67 68Input parameters that updated while DAMON_RECLAIM is running are not applied69by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values70of parametrs except ``enabled`` again. Once the re-reading is done, this71parameter is set as ``N``. If invalid parameters are found while the72re-reading, DAMON_RECLAIM will be disabled.73 74min_age75-------76 77Time threshold for cold memory regions identification in microseconds.78 79If a memory region is not accessed for this or longer time, DAMON_RECLAIM80identifies the region as cold, and reclaims it.81 82120 seconds by default.83 84quota_ms85--------86 87Limit of time for the reclamation in milliseconds.88 89DAMON_RECLAIM tries to use only up to this time within a time window90(quota_reset_interval_ms) for trying reclamation of cold pages. This can be91used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the92limit is disabled.93 9410 ms by default.95 96quota_sz97--------98 99Limit of size of memory for the reclamation in bytes.100 101DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time102window (quota_reset_interval_ms) and makes no more than this limit is tried.103This can be used for limiting consumption of CPU and IO. If this value is104zero, the limit is disabled.105 106128 MiB by default.107 108quota_reset_interval_ms109-----------------------110 111The time/size quota charge reset interval in milliseconds.112 113The charget reset interval for the quota of time (quota_ms) and size114(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than115quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms116milliseconds.117 1181 second by default.119 120quota_mem_pressure_us121---------------------122 123Desired level of memory pressure-stall time in microseconds.124 125While keeping the caps that set by other quotas, DAMON_RECLAIM automatically126increases and decreases the effective level of the quota aiming this level of127memory pressure is incurred. System-wide ``some`` memory PSI in microseconds128per quota reset interval (``quota_reset_interval_ms``) is collected and129compared to this value to see if the aim is satisfied. Value zero means130disabling this auto-tuning feature.131 132Disabled by default.133 134quota_autotune_feedback135-----------------------136 137User-specifiable feedback for auto-tuning of the effective quota.138 139While keeping the caps that set by other quotas, DAMON_RECLAIM automatically140increases and decreases the effective level of the quota aiming receiving this141feedback of value ``10,000`` from the user. DAMON_RECLAIM assumes the feedback142value and the quota are positively proportional. Value zero means disabling143this auto-tuning feature.144 145Disabled by default.146 147wmarks_interval148---------------149 150Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is151enabled but inactive due to its watermarks rule.152 153wmarks_high154-----------155 156Free memory rate (per thousand) for the high watermark.157 158If free memory of the system in bytes per thousand bytes is higher than this,159DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks160the watermarks.161 162wmarks_mid163----------164 165Free memory rate (per thousand) for the middle watermark.166 167If free memory of the system in bytes per thousand bytes is between this and168the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and169the reclaiming.170 171wmarks_low172----------173 174Free memory rate (per thousand) for the low watermark.175 176If free memory of the system in bytes per thousand bytes is lower than this,177DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the178watermarks. In the case, the system falls back to the LRU-list based page179granularity reclamation logic.180 181sample_interval182---------------183 184Sampling interval for the monitoring in microseconds.185 186The sampling interval of DAMON for the cold memory monitoring. Please refer to187the DAMON documentation (:doc:`usage`) for more detail.188 189aggr_interval190-------------191 192Aggregation interval for the monitoring in microseconds.193 194The aggregation interval of DAMON for the cold memory monitoring. Please195refer to the DAMON documentation (:doc:`usage`) for more detail.196 197min_nr_regions198--------------199 200Minimum number of monitoring regions.201 202The minimal number of monitoring regions of DAMON for the cold memory203monitoring. This can be used to set lower-bound of the monitoring quality.204But, setting this too high could result in increased monitoring overhead.205Please refer to the DAMON documentation (:doc:`usage`) for more detail.206 207max_nr_regions208--------------209 210Maximum number of monitoring regions.211 212The maximum number of monitoring regions of DAMON for the cold memory213monitoring. This can be used to set upper-bound of the monitoring overhead.214However, setting this too low could result in bad monitoring quality. Please215refer to the DAMON documentation (:doc:`usage`) for more detail.216 217monitor_region_start218--------------------219 220Start of target memory region in physical address.221 222The start physical address of memory region that DAMON_RECLAIM will do work223against. That is, DAMON_RECLAIM will find cold memory regions in this region224and reclaims. By default, biggest System RAM is used as the region.225 226monitor_region_end227------------------228 229End of target memory region in physical address.230 231The end physical address of memory region that DAMON_RECLAIM will do work232against. That is, DAMON_RECLAIM will find cold memory regions in this region233and reclaims. By default, biggest System RAM is used as the region.234 235skip_anon236---------237 238Skip anonymous pages reclamation.239 240If this parameter is set as ``Y``, DAMON_RECLAIM does not reclaim anonymous241pages. By default, ``N``.242 243 244kdamond_pid245-----------246 247PID of the DAMON thread.248 249If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else,250-1.251 252nr_reclaim_tried_regions253------------------------254 255Number of memory regions that tried to be reclaimed by DAMON_RECLAIM.256 257bytes_reclaim_tried_regions258---------------------------259 260Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM.261 262nr_reclaimed_regions263--------------------264 265Number of memory regions that successfully be reclaimed by DAMON_RECLAIM.266 267bytes_reclaimed_regions268-----------------------269 270Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM.271 272nr_quota_exceeds273----------------274 275Number of times that the time/space quota limits have exceeded.276 277Example278=======279 280Below runtime example commands make DAMON_RECLAIM to find memory regions that281not accessed for 30 seconds or more and pages out. The reclamation is limited282to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too283much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do284nothing if the system's free memory rate is more than 50%, but start the real285works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and286therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to287do nothing again, so that we can fall back to the LRU-list based page288granularity reclamation. ::289 290 # cd /sys/module/damon_reclaim/parameters291 # echo 30000000 > min_age292 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz293 # echo 1000 > quota_reset_interval_ms294 # echo 500 > wmarks_high295 # echo 400 > wmarks_mid296 # echo 200 > wmarks_low297 # echo Y > enabled298 299.. [1] https://research.google/pubs/pub48551/300.. [2] https://lwn.net/Articles/787611/301.. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html302