471 lines · plain
1 2Performance Counters for Linux3------------------------------4 5Performance counters are special hardware registers available on most modern6CPUs. These registers count the number of certain types of hw events: such7as instructions executed, cachemisses suffered, or branches mis-predicted -8without slowing down the kernel or applications. These registers can also9trigger interrupts when a threshold number of events have passed - and can10thus be used to profile the code that runs on that CPU.11 12The Linux Performance Counter subsystem provides an abstraction of these13hardware capabilities. It provides per task and per CPU counters, counter14groups, and it provides event capabilities on top of those. It15provides "virtual" 64-bit counters, regardless of the width of the16underlying hardware counters.17 18Performance counters are accessed via special file descriptors.19There's one file descriptor per virtual counter used.20 21The special file descriptor is opened via the sys_perf_event_open()22system call:23 24 int sys_perf_event_open(struct perf_event_attr *hw_event_uptr,25 pid_t pid, int cpu, int group_fd,26 unsigned long flags);27 28The syscall returns the new fd. The fd can be used via the normal29VFS system calls: read() can be used to read the counter, fcntl()30can be used to set the blocking mode, etc.31 32Multiple counters can be kept open at a time, and the counters33can be poll()ed.34 35When creating a new counter fd, 'perf_event_attr' is:36 37struct perf_event_attr {38 /*39 * The MSB of the config word signifies if the rest contains cpu40 * specific (raw) counter configuration data, if unset, the next41 * 7 bits are an event type and the rest of the bits are the event42 * identifier.43 */44 __u64 config;45 46 __u64 irq_period;47 __u32 record_type;48 __u32 read_format;49 50 __u64 disabled : 1, /* off by default */51 inherit : 1, /* children inherit it */52 pinned : 1, /* must always be on PMU */53 exclusive : 1, /* only group on PMU */54 exclude_user : 1, /* don't count user */55 exclude_kernel : 1, /* ditto kernel */56 exclude_hv : 1, /* ditto hypervisor */57 exclude_idle : 1, /* don't count when idle */58 mmap : 1, /* include mmap data */59 munmap : 1, /* include munmap data */60 comm : 1, /* include comm data */61 62 __reserved_1 : 52;63 64 __u32 extra_config_len;65 __u32 wakeup_events; /* wakeup every n events */66 67 __u64 __reserved_2;68 __u64 __reserved_3;69};70 71The 'config' field specifies what the counter should count. It72is divided into 3 bit-fields:73 74raw_type: 1 bit (most significant bit) 0x8000_0000_0000_000075type: 7 bits (next most significant) 0x7f00_0000_0000_000076event_id: 56 bits (least significant) 0x00ff_ffff_ffff_ffff77 78If 'raw_type' is 1, then the counter will count a hardware event79specified by the remaining 63 bits of event_config. The encoding is80machine-specific.81 82If 'raw_type' is 0, then the 'type' field says what kind of counter83this is, with the following encoding:84 85enum perf_type_id {86 PERF_TYPE_HARDWARE = 0,87 PERF_TYPE_SOFTWARE = 1,88 PERF_TYPE_TRACEPOINT = 2,89};90 91A counter of PERF_TYPE_HARDWARE will count the hardware event92specified by 'event_id':93 94/*95 * Generalized performance counter event types, used by the hw_event.event_id96 * parameter of the sys_perf_event_open() syscall:97 */98enum perf_hw_id {99 /*100 * Common hardware events, generalized by the kernel:101 */102 PERF_COUNT_HW_CPU_CYCLES = 0,103 PERF_COUNT_HW_INSTRUCTIONS = 1,104 PERF_COUNT_HW_CACHE_REFERENCES = 2,105 PERF_COUNT_HW_CACHE_MISSES = 3,106 PERF_COUNT_HW_BRANCH_INSTRUCTIONS = 4,107 PERF_COUNT_HW_BRANCH_MISSES = 5,108 PERF_COUNT_HW_BUS_CYCLES = 6,109 PERF_COUNT_HW_STALLED_CYCLES_FRONTEND = 7,110 PERF_COUNT_HW_STALLED_CYCLES_BACKEND = 8,111 PERF_COUNT_HW_REF_CPU_CYCLES = 9,112};113 114These are standardized types of events that work relatively uniformly115on all CPUs that implement Performance Counters support under Linux,116although there may be variations (e.g., different CPUs might count117cache references and misses at different levels of the cache hierarchy).118If a CPU is not able to count the selected event, then the system call119will return -EINVAL.120 121More hw_event_types are supported as well, but they are CPU-specific122and accessed as raw events. For example, to count "External bus123cycles while bus lock signal asserted" events on Intel Core CPUs, pass124in a 0x4064 event_id value and set hw_event.raw_type to 1.125 126A counter of type PERF_TYPE_SOFTWARE will count one of the available127software events, selected by 'event_id':128 129/*130 * Special "software" counters provided by the kernel, even if the hardware131 * does not support performance counters. These counters measure various132 * physical and sw events of the kernel (and allow the profiling of them as133 * well):134 */135enum perf_sw_ids {136 PERF_COUNT_SW_CPU_CLOCK = 0,137 PERF_COUNT_SW_TASK_CLOCK = 1,138 PERF_COUNT_SW_PAGE_FAULTS = 2,139 PERF_COUNT_SW_CONTEXT_SWITCHES = 3,140 PERF_COUNT_SW_CPU_MIGRATIONS = 4,141 PERF_COUNT_SW_PAGE_FAULTS_MIN = 5,142 PERF_COUNT_SW_PAGE_FAULTS_MAJ = 6,143 PERF_COUNT_SW_ALIGNMENT_FAULTS = 7,144 PERF_COUNT_SW_EMULATION_FAULTS = 8,145};146 147Counters of the type PERF_TYPE_TRACEPOINT are available when the ftrace event148tracer is available, and event_id values can be obtained from149/debug/tracing/events/*/*/id150 151 152Counters come in two flavours: counting counters and sampling153counters. A "counting" counter is one that is used for counting the154number of events that occur, and is characterised by having155irq_period = 0.156 157 158A read() on a counter returns the current value of the counter and possible159additional values as specified by 'read_format', each value is a u64 (8 bytes)160in size.161 162/*163 * Bits that can be set in hw_event.read_format to request that164 * reads on the counter should return the indicated quantities,165 * in increasing order of bit value, after the counter value.166 */167enum perf_event_read_format {168 PERF_FORMAT_TOTAL_TIME_ENABLED = 1,169 PERF_FORMAT_TOTAL_TIME_RUNNING = 2,170};171 172Using these additional values one can establish the overcommit ratio for a173particular counter allowing one to take the round-robin scheduling effect174into account.175 176 177A "sampling" counter is one that is set up to generate an interrupt178every N events, where N is given by 'irq_period'. A sampling counter179has irq_period > 0. The record_type controls what data is recorded on each180interrupt:181 182/*183 * Bits that can be set in hw_event.record_type to request information184 * in the overflow packets.185 */186enum perf_event_record_format {187 PERF_RECORD_IP = 1U << 0,188 PERF_RECORD_TID = 1U << 1,189 PERF_RECORD_TIME = 1U << 2,190 PERF_RECORD_ADDR = 1U << 3,191 PERF_RECORD_GROUP = 1U << 4,192 PERF_RECORD_CALLCHAIN = 1U << 5,193};194 195Such (and other) events will be recorded in a ring-buffer, which is196available to user-space using mmap() (see below).197 198The 'disabled' bit specifies whether the counter starts out disabled199or enabled. If it is initially disabled, it can be enabled by ioctl200or prctl (see below).201 202The 'inherit' bit, if set, specifies that this counter should count203events on descendant tasks as well as the task specified. This only204applies to new descendents, not to any existing descendents at the205time the counter is created (nor to any new descendents of existing206descendents).207 208The 'pinned' bit, if set, specifies that the counter should always be209on the CPU if at all possible. It only applies to hardware counters210and only to group leaders. If a pinned counter cannot be put onto the211CPU (e.g. because there are not enough hardware counters or because of212a conflict with some other event), then the counter goes into an213'error' state, where reads return end-of-file (i.e. read() returns 0)214until the counter is subsequently enabled or disabled.215 216The 'exclusive' bit, if set, specifies that when this counter's group217is on the CPU, it should be the only group using the CPU's counters.218In future, this will allow sophisticated monitoring programs to supply219extra configuration information via 'extra_config_len' to exploit220advanced features of the CPU's Performance Monitor Unit (PMU) that are221not otherwise accessible and that might disrupt other hardware222counters.223 224The 'exclude_user', 'exclude_kernel' and 'exclude_hv' bits provide a225way to request that counting of events be restricted to times when the226CPU is in user, kernel and/or hypervisor mode.227 228Furthermore the 'exclude_host' and 'exclude_guest' bits provide a way229to request counting of events restricted to guest and host contexts when230using Linux as the hypervisor.231 232The 'mmap' and 'munmap' bits allow recording of PROT_EXEC mmap/munmap233operations, these can be used to relate userspace IP addresses to actual234code, even after the mapping (or even the whole process) is gone,235these events are recorded in the ring-buffer (see below).236 237The 'comm' bit allows tracking of process comm data on process creation.238This too is recorded in the ring-buffer (see below).239 240The 'pid' parameter to the sys_perf_event_open() system call allows the241counter to be specific to a task:242 243 pid == 0: if the pid parameter is zero, the counter is attached to the244 current task.245 246 pid > 0: the counter is attached to a specific task (if the current task247 has sufficient privilege to do so)248 249 pid < 0: all tasks are counted (per cpu counters)250 251The 'cpu' parameter allows a counter to be made specific to a CPU:252 253 cpu >= 0: the counter is restricted to a specific CPU254 cpu == -1: the counter counts on all CPUs255 256(Note: the combination of 'pid == -1' and 'cpu == -1' is not valid.)257 258A 'pid > 0' and 'cpu == -1' counter is a per task counter that counts259events of that task and 'follows' that task to whatever CPU the task260gets schedule to. Per task counters can be created by any user, for261their own tasks.262 263A 'pid == -1' and 'cpu == x' counter is a per CPU counter that counts264all events on CPU-x. Per CPU counters need CAP_PERFMON or CAP_SYS_ADMIN265privilege.266 267The 'flags' parameter is currently unused and must be zero.268 269The 'group_fd' parameter allows counter "groups" to be set up. A270counter group has one counter which is the group "leader". The leader271is created first, with group_fd = -1 in the sys_perf_event_open call272that creates it. The rest of the group members are created273subsequently, with group_fd giving the fd of the group leader.274(A single counter on its own is created with group_fd = -1 and is275considered to be a group with only 1 member.)276 277A counter group is scheduled onto the CPU as a unit, that is, it will278only be put onto the CPU if all of the counters in the group can be279put onto the CPU. This means that the values of the member counters280can be meaningfully compared, added, divided (to get ratios), etc.,281with each other, since they have counted events for the same set of282executed instructions.283 284 285Like stated, asynchronous events, like counter overflow or PROT_EXEC mmap286tracking are logged into a ring-buffer. This ring-buffer is created and287accessed through mmap().288 289The mmap size should be 1+2^n pages, where the first page is a meta-data page290(struct perf_event_mmap_page) that contains various bits of information such291as where the ring-buffer head is.292 293/*294 * Structure of the page that can be mapped via mmap295 */296struct perf_event_mmap_page {297 __u32 version; /* version number of this structure */298 __u32 compat_version; /* lowest version this is compat with */299 300 /*301 * Bits needed to read the hw counters in user-space.302 *303 * u32 seq;304 * s64 count;305 *306 * do {307 * seq = pc->lock;308 *309 * barrier()310 * if (pc->index) {311 * count = pmc_read(pc->index - 1);312 * count += pc->offset;313 * } else314 * goto regular_read;315 *316 * barrier();317 * } while (pc->lock != seq);318 *319 * NOTE: for obvious reason this only works on self-monitoring320 * processes.321 */322 __u32 lock; /* seqlock for synchronization */323 __u32 index; /* hardware counter identifier */324 __s64 offset; /* add to hardware counter value */325 326 /*327 * Control data for the mmap() data buffer.328 *329 * User-space reading this value should issue an rmb(), on SMP capable330 * platforms, after reading this value -- see perf_event_wakeup().331 */332 __u32 data_head; /* head in the data section */333};334 335NOTE: the hw-counter userspace bits are arch specific and are currently only336 implemented on powerpc.337 338The following 2^n pages are the ring-buffer which contains events of the form:339 340#define PERF_RECORD_MISC_KERNEL (1 << 0)341#define PERF_RECORD_MISC_USER (1 << 1)342#define PERF_RECORD_MISC_OVERFLOW (1 << 2)343 344struct perf_event_header {345 __u32 type;346 __u16 misc;347 __u16 size;348};349 350enum perf_event_type {351 352 /*353 * The MMAP events record the PROT_EXEC mappings so that we can354 * correlate userspace IPs to code. They have the following structure:355 *356 * struct {357 * struct perf_event_header header;358 *359 * u32 pid, tid;360 * u64 addr;361 * u64 len;362 * u64 pgoff;363 * char filename[];364 * };365 */366 PERF_RECORD_MMAP = 1,367 PERF_RECORD_MUNMAP = 2,368 369 /*370 * struct {371 * struct perf_event_header header;372 *373 * u32 pid, tid;374 * char comm[];375 * };376 */377 PERF_RECORD_COMM = 3,378 379 /*380 * When header.misc & PERF_RECORD_MISC_OVERFLOW the event_type field381 * will be PERF_RECORD_*382 *383 * struct {384 * struct perf_event_header header;385 *386 * { u64 ip; } && PERF_RECORD_IP387 * { u32 pid, tid; } && PERF_RECORD_TID388 * { u64 time; } && PERF_RECORD_TIME389 * { u64 addr; } && PERF_RECORD_ADDR390 *391 * { u64 nr;392 * { u64 event, val; } cnt[nr]; } && PERF_RECORD_GROUP393 *394 * { u16 nr,395 * hv,396 * kernel,397 * user;398 * u64 ips[nr]; } && PERF_RECORD_CALLCHAIN399 * };400 */401};402 403NOTE: PERF_RECORD_CALLCHAIN is arch specific and currently only implemented404 on x86.405 406Notification of new events is possible through poll()/select()/epoll() and407fcntl() managing signals.408 409Normally a notification is generated for every page filled, however one can410additionally set perf_event_attr.wakeup_events to generate one every411so many counter overflow events.412 413Future work will include a splice() interface to the ring-buffer.414 415 416Counters can be enabled and disabled in two ways: via ioctl and via417prctl. When a counter is disabled, it doesn't count or generate418events but does continue to exist and maintain its count value.419 420An individual counter can be enabled with421 422 ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);423 424or disabled with425 426 ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);427 428For a counter group, pass PERF_IOC_FLAG_GROUP as the third argument.429Enabling or disabling the leader of a group enables or disables the430whole group; that is, while the group leader is disabled, none of the431counters in the group will count. Enabling or disabling a member of a432group other than the leader only affects that counter - disabling an433non-leader stops that counter from counting but doesn't affect any434other counter.435 436Additionally, non-inherited overflow counters can use437 438 ioctl(fd, PERF_EVENT_IOC_REFRESH, nr);439 440to enable a counter for 'nr' events, after which it gets disabled again.441 442A process can enable or disable all the counter groups that are443attached to it, using prctl:444 445 prctl(PR_TASK_PERF_EVENTS_ENABLE);446 447 prctl(PR_TASK_PERF_EVENTS_DISABLE);448 449This applies to all counters on the current process, whether created450by this process or by another, and doesn't affect any counters that451this process has created on other processes. It only enables or452disables the group leaders, not any other members in the groups.453 454 455Arch requirements456-----------------457 458If your architecture does not have hardware performance metrics, you can459still use the generic software counters based on hrtimers for sampling.460 461So to start with, in order to add HAVE_PERF_EVENTS to your Kconfig, you462will need at least this:463 - asm/perf_event.h - a basic stub will suffice at first464 - support for atomic64 types (and associated helper functions)465 466If your architecture does have hardware capabilities, you can override the467weak stub hw_perf_event_init() to register hardware counters.468 469Architectures that have d-cache aliassing issues, such as Sparc and ARM,470should select PERF_USE_VMALLOC in order to avoid these for perf mmap().471