120 lines · plain
1============================2Subsystem Trace Points: kmem3============================4 5The kmem tracing system captures events related to object and page allocation6within the kernel. Broadly speaking there are five major subheadings.7 8 - Slab allocation of small objects of unknown type (kmalloc)9 - Slab allocation of small objects of known type10 - Page allocation11 - Per-CPU Allocator Activity12 - External Fragmentation13 14This document describes what each of the tracepoints is and why they15might be useful.16 171. Slab allocation of small objects of unknown type18===================================================19::20 21 kmalloc call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s22 kmalloc_node call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s node=%d23 kfree call_site=%lx ptr=%p24 25Heavy activity for these events may indicate that a specific cache is26justified, particularly if kmalloc slab pages are getting significantly27internal fragmented as a result of the allocation pattern. By correlating28kmalloc with kfree, it may be possible to identify memory leaks and where29the allocation sites were.30 31 322. Slab allocation of small objects of known type33=================================================34::35 36 kmem_cache_alloc call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s37 kmem_cache_alloc_node call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s node=%d38 kmem_cache_free call_site=%lx ptr=%p39 40These events are similar in usage to the kmalloc-related events except that41it is likely easier to pin the event down to a specific cache. At the time42of writing, no information is available on what slab is being allocated from,43but the call_site can usually be used to extrapolate that information.44 453. Page allocation46==================47::48 49 mm_page_alloc page=%p pfn=%lu order=%d migratetype=%d gfp_flags=%s50 mm_page_alloc_zone_locked page=%p pfn=%lu order=%u migratetype=%d cpu=%d percpu_refill=%d51 mm_page_free page=%p pfn=%lu order=%d52 mm_page_free_batched page=%p pfn=%lu order=%d cold=%d53 54These four events deal with page allocation and freeing. mm_page_alloc is55a simple indicator of page allocator activity. Pages may be allocated from56the per-CPU allocator (high performance) or the buddy allocator.57 58If pages are allocated directly from the buddy allocator, the59mm_page_alloc_zone_locked event is triggered. This event is important as high60amounts of activity imply high activity on the zone->lock. Taking this lock61impairs performance by disabling interrupts, dirtying cache lines between62CPUs and serialising many CPUs.63 64When a page is freed directly by the caller, the only mm_page_free event65is triggered. Significant amounts of activity here could indicate that the66callers should be batching their activities.67 68When pages are freed in batch, the also mm_page_free_batched is triggered.69Broadly speaking, pages are taken off the LRU lock in bulk and70freed in batch with a page list. Significant amounts of activity here could71indicate that the system is under memory pressure and can also indicate72contention on the lruvec->lru_lock.73 744. Per-CPU Allocator Activity75=============================76::77 78 mm_page_alloc_zone_locked page=%p pfn=%lu order=%u migratetype=%d cpu=%d percpu_refill=%d79 mm_page_pcpu_drain page=%p pfn=%lu order=%d cpu=%d migratetype=%d80 81In front of the page allocator is a per-cpu page allocator. It exists only82for order-0 pages, reduces contention on the zone->lock and reduces the83amount of writing on struct page.84 85When a per-CPU list is empty or pages of the wrong type are allocated,86the zone->lock will be taken once and the per-CPU list refilled. The event87triggered is mm_page_alloc_zone_locked for each page allocated with the88event indicating whether it is for a percpu_refill or not.89 90When the per-CPU list is too full, a number of pages are freed, each one91which triggers a mm_page_pcpu_drain event.92 93The individual nature of the events is so that pages can be tracked94between allocation and freeing. A number of drain or refill pages that occur95consecutively imply the zone->lock being taken once. Large amounts of per-CPU96refills and drains could imply an imbalance between CPUs where too much work97is being concentrated in one place. It could also indicate that the per-CPU98lists should be a larger size. Finally, large amounts of refills on one CPU99and drains on another could be a factor in causing large amounts of cache100line bounces due to writes between CPUs and worth investigating if pages101can be allocated and freed on the same CPU through some algorithm change.102 1035. External Fragmentation104=========================105::106 107 mm_page_alloc_extfrag page=%p pfn=%lu alloc_order=%d fallback_order=%d pageblock_order=%d alloc_migratetype=%d fallback_migratetype=%d fragmenting=%d change_ownership=%d108 109External fragmentation affects whether a high-order allocation will be110successful or not. For some types of hardware, this is important although111it is avoided where possible. If the system is using huge pages and needs112to be able to resize the pool over the lifetime of the system, this value113is important.114 115Large numbers of this event implies that memory is fragmenting and116high-order allocations will start failing at some time in the future. One117means of reducing the occurrence of this event is to increase the size of118min_free_kbytes in increments of 3*pageblock_size*nr_online_nodes where119pageblock_size is usually the size of the default hugepage size.120