119 lines · plain
1==================2Idle Page Tracking3==================4 5Motivation6==========7 8The idle page tracking feature allows to track which memory pages are being9accessed by a workload and which are idle. This information can be useful for10estimating the workload's working set size, which, in turn, can be taken into11account when configuring the workload parameters, setting memory cgroup limits,12or deciding where to place the workload within a compute cluster.13 14It is enabled by CONFIG_IDLE_PAGE_TRACKING=y.15 16.. _user_api:17 18User API19========20 21The idle page tracking API is located at ``/sys/kernel/mm/page_idle``.22Currently, it consists of the only read-write file,23``/sys/kernel/mm/page_idle/bitmap``.24 25The file implements a bitmap where each bit corresponds to a memory page. The26bitmap is represented by an array of 8-byte integers, and the page at PFN #i is27mapped to bit #i%64 of array element #i/64, byte order is native. When a bit is28set, the corresponding page is idle.29 30A page is considered idle if it has not been accessed since it was marked idle31(for more details on what "accessed" actually means see the :ref:`Implementation32Details <impl_details>` section).33To mark a page idle one has to set the bit corresponding to34the page by writing to the file. A value written to the file is OR-ed with the35current bitmap value.36 37Only accesses to user memory pages are tracked. These are pages mapped to a38process address space, page cache and buffer pages, swap cache pages. For other39page types (e.g. SLAB pages) an attempt to mark a page idle is silently ignored,40and hence such pages are never reported idle.41 42For huge pages the idle flag is set only on the head page, so one has to read43``/proc/kpageflags`` in order to correctly count idle huge pages.44 45Reading from or writing to ``/sys/kernel/mm/page_idle/bitmap`` will return46-EINVAL if you are not starting the read/write on an 8-byte boundary, or47if the size of the read/write is not a multiple of 8 bytes. Writing to48this file beyond max PFN will return -ENXIO.49 50That said, in order to estimate the amount of pages that are not used by a51workload one should:52 53 1. Mark all the workload's pages as idle by setting corresponding bits in54 ``/sys/kernel/mm/page_idle/bitmap``. The pages can be found by reading55 ``/proc/pid/pagemap`` if the workload is represented by a process, or by56 filtering out alien pages using ``/proc/kpagecgroup`` in case the workload57 is placed in a memory cgroup.58 59 2. Wait until the workload accesses its working set.60 61 3. Read ``/sys/kernel/mm/page_idle/bitmap`` and count the number of bits set.62 If one wants to ignore certain types of pages, e.g. mlocked pages since they63 are not reclaimable, he or she can filter them out using64 ``/proc/kpageflags``.65 66The page-types tool in the tools/mm directory can be used to assist in this.67If the tool is run initially with the appropriate option, it will mark all the68queried pages as idle. Subsequent runs of the tool can then show which pages have69their idle flag cleared in the interim.70 71See Documentation/admin-guide/mm/pagemap.rst for more information about72``/proc/pid/pagemap``, ``/proc/kpageflags``, and ``/proc/kpagecgroup``.73 74.. _impl_details:75 76Implementation Details77======================78 79The kernel internally keeps track of accesses to user memory pages in order to80reclaim unreferenced pages first on memory shortage conditions. A page is81considered referenced if it has been recently accessed via a process address82space, in which case one or more PTEs it is mapped to will have the Accessed bit83set, or marked accessed explicitly by the kernel (see mark_page_accessed()). The84latter happens when:85 86 - a userspace process reads or writes a page using a system call (e.g. read(2)87 or write(2))88 89 - a page that is used for storing filesystem buffers is read or written,90 because a process needs filesystem metadata stored in it (e.g. lists a91 directory tree)92 93 - a page is accessed by a device driver using get_user_pages()94 95When a dirty page is written to swap or disk as a result of memory reclaim or96exceeding the dirty memory limit, it is not marked referenced.97 98The idle memory tracking feature adds a new page flag, the Idle flag. This flag99is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the100:ref:`User API <user_api>`101section), and cleared automatically whenever a page is referenced as defined102above.103 104When a page is marked idle, the Accessed bit must be cleared in all PTEs it is105mapped to, otherwise we will not be able to detect accesses to the page coming106from a process address space. To avoid interference with the reclaimer, which,107as noted above, uses the Accessed bit to promote actively referenced pages, one108more page flag is introduced, the Young flag. When the PTE Accessed bit is109cleared as a result of setting or updating a page's Idle flag, the Young flag110is set on the page. The reclaimer treats the Young flag as an extra PTE111Accessed bit and therefore will consider such a page as referenced.112 113Since the idle memory tracking feature is based on the memory reclaimer logic,114it only works with pages that are on an LRU list, other pages are silently115ignored. That means it will ignore a user memory page if it is isolated, but116since there are usually not many of them, it should not affect the overall117result noticeably. In order not to stall scanning of the idle page bitmap,118locked pages may be skipped too.119