831 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3================4Perf ring buffer5================6 7.. CONTENTS8 9 1. Introduction10 11 2. Ring buffer implementation12 2.1 Basic algorithm13 2.2 Ring buffer for different tracing modes14 2.2.1 Default mode15 2.2.2 Per-thread mode16 2.2.3 Per-CPU mode17 2.2.4 System wide mode18 2.3 Accessing buffer19 2.3.1 Producer-consumer model20 2.3.2 Properties of the ring buffers21 2.3.3 Writing samples into buffer22 2.3.4 Reading samples from buffer23 2.3.5 Memory synchronization24 25 3. The mechanism of AUX ring buffer26 3.1 The relationship between AUX and regular ring buffers27 3.2 AUX events28 3.3 Snapshot mode29 30 311. Introduction32===============33 34The ring buffer is a fundamental mechanism for data transfer. perf uses35ring buffers to transfer event data from kernel to user space, another36kind of ring buffer which is so called auxiliary (AUX) ring buffer also37plays an important role for hardware tracing with Intel PT, Arm38CoreSight, etc.39 40The ring buffer implementation is critical but it's also a very41challenging work. On the one hand, the kernel and perf tool in the user42space use the ring buffer to exchange data and stores data into data43file, thus the ring buffer needs to transfer data with high throughput;44on the other hand, the ring buffer management should avoid significant45overload to distract profiling results.46 47This documentation dives into the details for perf ring buffer with two48parts: firstly it explains the perf ring buffer implementation, then the49second part discusses the AUX ring buffer mechanism.50 512. Ring buffer implementation52=============================53 542.1 Basic algorithm55-------------------56 57That said, a typical ring buffer is managed by a head pointer and a tail58pointer; the head pointer is manipulated by a writer and the tail59pointer is updated by a reader respectively.60 61::62 63 +---------------------------+64 | | |***|***|***| | |65 +---------------------------+66 `-> Tail `-> Head67 68 * : the data is filled by the writer.69 70 Figure 1. Ring buffer71 72Perf uses the same way to manage its ring buffer. In the implementation73there are two key data structures held together in a set of consecutive74pages, the control structure and then the ring buffer itself. The page75with the control structure in is known as the "user page". Being held76in continuous virtual addresses simplifies locating the ring buffer77address, it is in the pages after the page with the user page.78 79The control structure is named as ``perf_event_mmap_page``, it contains a80head pointer ``data_head`` and a tail pointer ``data_tail``. When the81kernel starts to fill records into the ring buffer, it updates the head82pointer to reserve the memory so later it can safely store events into83the buffer. On the other side, when the user page is a writable mapping,84the perf tool has the permission to update the tail pointer after consuming85data from the ring buffer. Yet another case is for the user page's86read-only mapping, which is to be addressed in the section87:ref:`writing_samples_into_buffer`.88 89::90 91 user page ring buffer92 +---------+---------+ +---------------------------------------+93 |data_head|data_tail|...| | |***|***|***|***|***| | | |94 +---------+---------+ +---------------------------------------+95 ` `----------------^ ^96 `----------------------------------------------|97 98 * : the data is filled by the writer.99 100 Figure 2. Perf ring buffer101 102When using the ``perf record`` tool, we can specify the ring buffer size103with option ``-m`` or ``--mmap-pages=``, the given size will be rounded up104to a power of two that is a multiple of a page size. Though the kernel105allocates at once for all memory pages, it's deferred to map the pages106to VMA area until the perf tool accesses the buffer from the user space.107In other words, at the first time accesses the buffer's page from user108space in the perf tool, a data abort exception for page fault is taken109and the kernel uses this occasion to map the page into process VMA110(see ``perf_mmap_fault()``), thus the perf tool can continue to access111the page after returning from the exception.112 1132.2 Ring buffer for different tracing modes114-------------------------------------------115 116The perf profiles programs with different modes: default mode, per thread117mode, per cpu mode, and system wide mode. This section describes these118modes and how the ring buffer meets requirements for them. At last we119will review the race conditions caused by these modes.120 1212.2.1 Default mode122^^^^^^^^^^^^^^^^^^123 124Usually we execute ``perf record`` command followed by a profiling program125name, like below command::126 127 perf record test_program128 129This command doesn't specify any options for CPU and thread modes, the130perf tool applies the default mode on the perf event. It maps all the131CPUs in the system and the profiled program's PID on the perf event, and132it enables inheritance mode on the event so that child tasks inherits133the events. As a result, the perf event is attributed as::134 135 evsel::cpus::map[] = { 0 .. _SC_NPROCESSORS_ONLN-1 }136 evsel::threads::map[] = { pid }137 evsel::attr::inherit = 1138 139These attributions finally will be reflected on the deployment of ring140buffers. As shown below, the perf tool allocates individual ring buffer141for each CPU, but it only enables events for the profiled program rather142than for all threads in the system. The *T1* thread represents the143thread context of the 'test_program', whereas *T2* and *T3* are irrelevant144threads in the system. The perf samples are exclusively collected for145the *T1* thread and stored in the ring buffer associated with the CPU on146which the *T1* thread is running.147 148::149 150 T1 T2 T1151 +----+ +-----------+ +----+152 CPU0 |xxxx| |xxxxxxxxxxx| |xxxx|153 +----+--------------+-----------+----------+----+-------->154 | |155 v v156 +-----------------------------------------------------+157 | Ring buffer 0 |158 +-----------------------------------------------------+159 160 T1161 +-----+162 CPU1 |xxxxx|163 -----+-----+--------------------------------------------->164 |165 v166 +-----------------------------------------------------+167 | Ring buffer 1 |168 +-----------------------------------------------------+169 170 T1 T3171 +----+ +-------+172 CPU2 |xxxx| |xxxxxxx|173 --------------------------+----+--------+-------+-------->174 |175 v176 +-----------------------------------------------------+177 | Ring buffer 2 |178 +-----------------------------------------------------+179 180 T1181 +--------------+182 CPU3 |xxxxxxxxxxxxxx|183 -----------+--------------+------------------------------>184 |185 v186 +-----------------------------------------------------+187 | Ring buffer 3 |188 +-----------------------------------------------------+189 190 T1: Thread 1; T2: Thread 2; T3: Thread 3191 x: Thread is in running state192 193 Figure 3. Ring buffer for default mode194 1952.2.2 Per-thread mode196^^^^^^^^^^^^^^^^^^^^^197 198By specifying option ``--per-thread`` in perf command, e.g.199 200::201 202 perf record --per-thread test_program203 204The perf event doesn't map to any CPUs and is only bound to the205profiled process, thus, the perf event's attributions are::206 207 evsel::cpus::map[0] = { -1 }208 evsel::threads::map[] = { pid }209 evsel::attr::inherit = 0210 211In this mode, a single ring buffer is allocated for the profiled thread;212if the thread is scheduled on a CPU, the events on that CPU will be213enabled; and if the thread is scheduled out from the CPU, the events on214the CPU will be disabled. When the thread is migrated from one CPU to215another, the events are to be disabled on the previous CPU and enabled216on the next CPU correspondingly.217 218::219 220 T1 T2 T1221 +----+ +-----------+ +----+222 CPU0 |xxxx| |xxxxxxxxxxx| |xxxx|223 +----+--------------+-----------+----------+----+-------->224 | |225 | T1 |226 | +-----+ |227 CPU1 | |xxxxx| |228 --|--+-----+----------------------------------|---------->229 | | |230 | | T1 T3 |231 | | +----+ +---+ |232 CPU2 | | |xxxx| |xxx| |233 --|-----|-----------------+----+--------+---+-|---------->234 | | | |235 | | T1 | |236 | | +--------------+ | |237 CPU3 | | |xxxxxxxxxxxxxx| | |238 --|-----|--+--------------+-|-----------------|---------->239 | | | | |240 v v v v v241 +-----------------------------------------------------+242 | Ring buffer |243 +-----------------------------------------------------+244 245 T1: Thread 1246 x: Thread is in running state247 248 Figure 4. Ring buffer for per-thread mode249 250When perf runs in per-thread mode, a ring buffer is allocated for the251profiled thread *T1*. The ring buffer is dedicated for thread *T1*, if the252thread *T1* is running, the perf events will be recorded into the ring253buffer; when the thread is sleeping, all associated events will be254disabled, thus no trace data will be recorded into the ring buffer.255 2562.2.3 Per-CPU mode257^^^^^^^^^^^^^^^^^^258 259The option ``-C`` is used to collect samples on the list of CPUs, for260example the below perf command receives option ``-C 0,2``::261 262 perf record -C 0,2 test_program263 264It maps the perf event to CPUs 0 and 2, and the event is not associated to any265PID. Thus the perf event attributions are set as::266 267 evsel::cpus::map[0] = { 0, 2 }268 evsel::threads::map[] = { -1 }269 evsel::attr::inherit = 0270 271This results in the session of ``perf record`` will sample all threads on CPU0272and CPU2, and be terminated until test_program exits. Even there have tasks273running on CPU1 and CPU3, since the ring buffer is absent for them, any274activities on these two CPUs will be ignored. A usage case is to combine the275options for per-thread mode and per-CPU mode, e.g. the options ``–C 0,2`` and276``––per–thread`` are specified together, the samples are recorded only when277the profiled thread is scheduled on any of the listed CPUs.278 279::280 281 T1 T2 T1282 +----+ +-----------+ +----+283 CPU0 |xxxx| |xxxxxxxxxxx| |xxxx|284 +----+--------------+-----------+----------+----+-------->285 | | |286 v v v287 +-----------------------------------------------------+288 | Ring buffer 0 |289 +-----------------------------------------------------+290 291 T1292 +-----+293 CPU1 |xxxxx|294 -----+-----+--------------------------------------------->295 296 T1 T3297 +----+ +-------+298 CPU2 |xxxx| |xxxxxxx|299 --------------------------+----+--------+-------+-------->300 | |301 v v302 +-----------------------------------------------------+303 | Ring buffer 1 |304 +-----------------------------------------------------+305 306 T1307 +--------------+308 CPU3 |xxxxxxxxxxxxxx|309 -----------+--------------+------------------------------>310 311 T1: Thread 1; T2: Thread 2; T3: Thread 3312 x: Thread is in running state313 314 Figure 5. Ring buffer for per-CPU mode315 3162.2.4 System wide mode317^^^^^^^^^^^^^^^^^^^^^^318 319By using option ``–a`` or ``––all–cpus``, perf collects samples on all CPUs320for all tasks, we call it as the system wide mode, the command is::321 322 perf record -a test_program323 324Similar to the per-CPU mode, the perf event doesn't bind to any PID, and325it maps to all CPUs in the system::326 327 evsel::cpus::map[] = { 0 .. _SC_NPROCESSORS_ONLN-1 }328 evsel::threads::map[] = { -1 }329 evsel::attr::inherit = 0330 331In the system wide mode, every CPU has its own ring buffer, all threads332are monitored during the running state and the samples are recorded into333the ring buffer belonging to the CPU which the events occurred on.334 335::336 337 T1 T2 T1338 +----+ +-----------+ +----+339 CPU0 |xxxx| |xxxxxxxxxxx| |xxxx|340 +----+--------------+-----------+----------+----+-------->341 | | |342 v v v343 +-----------------------------------------------------+344 | Ring buffer 0 |345 +-----------------------------------------------------+346 347 T1348 +-----+349 CPU1 |xxxxx|350 -----+-----+--------------------------------------------->351 |352 v353 +-----------------------------------------------------+354 | Ring buffer 1 |355 +-----------------------------------------------------+356 357 T1 T3358 +----+ +-------+359 CPU2 |xxxx| |xxxxxxx|360 --------------------------+----+--------+-------+-------->361 | |362 v v363 +-----------------------------------------------------+364 | Ring buffer 2 |365 +-----------------------------------------------------+366 367 T1368 +--------------+369 CPU3 |xxxxxxxxxxxxxx|370 -----------+--------------+------------------------------>371 |372 v373 +-----------------------------------------------------+374 | Ring buffer 3 |375 +-----------------------------------------------------+376 377 T1: Thread 1; T2: Thread 2; T3: Thread 3378 x: Thread is in running state379 380 Figure 6. Ring buffer for system wide mode381 3822.3 Accessing buffer383--------------------384 385Based on the understanding of how the ring buffer is allocated in386various modes, this section explains access the ring buffer.387 3882.3.1 Producer-consumer model389^^^^^^^^^^^^^^^^^^^^^^^^^^^^^390 391In the Linux kernel, the PMU events can produce samples which are stored392into the ring buffer; the perf command in user space consumes the393samples by reading out data from the ring buffer and finally saves the394data into the file for post analysis. It’s a typical producer-consumer395model for using the ring buffer.396 397The perf process polls on the PMU events and sleeps when no events are398incoming. To prevent frequent exchanges between the kernel and user399space, the kernel event core layer introduces a watermark, which is400stored in the ``perf_buffer::watermark``. When a sample is recorded into401the ring buffer, and if the used buffer exceeds the watermark, the402kernel wakes up the perf process to read samples from the ring buffer.403 404::405 406 Perf407 / | Read samples408 Polling / `--------------| Ring buffer409 v v ;---------------------v410 +----------------+ +---------+---------+ +-------------------+411 |Event wait queue| |data_head|data_tail| |***|***| | |***|412 +----------------+ +---------+---------+ +-------------------+413 ^ ^ `------------------------^414 | Wake up tasks | Store samples415 +-----------------------------+416 | Kernel event core layer |417 +-----------------------------+418 419 * : the data is filled by the writer.420 421 Figure 7. Writing and reading the ring buffer422 423When the kernel event core layer notifies the user space, because424multiple events might share the same ring buffer for recording samples,425the core layer iterates every event associated with the ring buffer and426wakes up tasks waiting on the event. This is fulfilled by the kernel427function ``ring_buffer_wakeup()``.428 429After the perf process is woken up, it starts to check the ring buffers430one by one, if it finds any ring buffer containing samples it will read431out the samples for statistics or saving into the data file. Given the432perf process is able to run on any CPU, this leads to the ring buffer433potentially being accessed from multiple CPUs simultaneously, which434causes race conditions. The race condition handling is described in the435section :ref:`memory_synchronization`.436 4372.3.2 Properties of the ring buffers438^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^439 440Linux kernel supports two write directions for the ring buffer: forward and441backward. The forward writing saves samples from the beginning of the ring442buffer, the backward writing stores data from the end of the ring buffer with443the reversed direction. The perf tool determines the writing direction.444 445Additionally, the tool can map buffers in either read-write mode or read-only446mode to the user space.447 448The ring buffer in the read-write mode is mapped with the property449``PROT_READ | PROT_WRITE``. With the write permission, the perf tool450updates the ``data_tail`` to indicate the data start position. Combining451with the head pointer ``data_head``, which works as the end position of452the current data, the perf tool can easily know where read out the data453from.454 455Alternatively, in the read-only mode, only the kernel keeps to update456the ``data_head`` while the user space cannot access the ``data_tail`` due457to the mapping property ``PROT_READ``.458 459As a result, the matrix below illustrates the various combinations of460direction and mapping characteristics. The perf tool employs two of these461combinations to support buffer types: the non-overwrite buffer and the462overwritable buffer.463 464.. list-table::465 :widths: 1 1 1466 :header-rows: 1467 468 * - Mapping mode469 - Forward470 - Backward471 * - read-write472 - Non-overwrite ring buffer473 - Not used474 * - read-only475 - Not used476 - Overwritable ring buffer477 478The non-overwrite ring buffer uses the read-write mapping with forward479writing. It starts to save data from the beginning of the ring buffer480and wrap around when overflow, which is used with the read-write mode in481the normal ring buffer. When the consumer doesn't keep up with the482producer, it would lose some data, the kernel keeps how many records it483lost and generates the ``PERF_RECORD_LOST`` records in the next time484when it finds a space in the ring buffer.485 486The overwritable ring buffer uses the backward writing with the487read-only mode. It saves the data from the end of the ring buffer and488the ``data_head`` keeps the position of current data, the perf always489knows where it starts to read and until the end of the ring buffer, thus490it don't need the ``data_tail``. In this mode, it will not generate the491``PERF_RECORD_LOST`` records.492 493.. _writing_samples_into_buffer:494 4952.3.3 Writing samples into buffer496^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^497 498When a sample is taken and saved into the ring buffer, the kernel499prepares sample fields based on the sample type; then it prepares the500info for writing ring buffer which is stored in the structure501``perf_output_handle``. In the end, the kernel outputs the sample into502the ring buffer and updates the head pointer in the user page so the503perf tool can see the latest value.504 505The structure ``perf_output_handle`` serves as a temporary context for506tracking the information related to the buffer. The advantages of it is507that it enables concurrent writing to the buffer by different events.508For example, a software event and a hardware PMU event both are enabled509for profiling, two instances of ``perf_output_handle`` serve as separate510contexts for the software event and the hardware event respectively.511This allows each event to reserve its own memory space for populating512the record data.513 5142.3.4 Reading samples from buffer515^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^516 517In the user space, the perf tool utilizes the ``perf_event_mmap_page``518structure to handle the head and tail of the buffer. It also uses519``perf_mmap`` structure to keep track of a context for the ring buffer, this520context includes information about the buffer's starting and ending521addresses. Additionally, the mask value can be utilized to compute the522circular buffer pointer even for an overflow.523 524Similar to the kernel, the perf tool in the user space first reads out525the recorded data from the ring buffer, and then updates the buffer's526tail pointer ``perf_event_mmap_page::data_tail``.527 528.. _memory_synchronization:529 5302.3.5 Memory synchronization531^^^^^^^^^^^^^^^^^^^^^^^^^^^^532 533The modern CPUs with relaxed memory model cannot promise the memory534ordering, this means it’s possible to access the ring buffer and the535``perf_event_mmap_page`` structure out of order. To assure the specific536sequence for memory accessing perf ring buffer, memory barriers are537used to assure the data dependency. The rationale for the memory538synchronization is as below::539 540 Kernel User space541 542 if (LOAD ->data_tail) { LOAD ->data_head543 (A) smp_rmb() (C)544 STORE $data LOAD $data545 smp_wmb() (B) smp_mb() (D)546 STORE ->data_head STORE ->data_tail547 }548 549The comments in tools/include/linux/ring_buffer.h gives nice description550for why and how to use memory barriers, here we will just provide an551alternative explanation:552 553(A) is a control dependency so that CPU assures order between checking554pointer ``perf_event_mmap_page::data_tail`` and filling sample into ring555buffer;556 557(D) pairs with (A). (D) separates the ring buffer data reading from558writing the pointer ``data_tail``, perf tool first consumes samples and then559tells the kernel that the data chunk has been released. Since a reading560operation is followed by a writing operation, thus (D) is a full memory561barrier.562 563(B) is a writing barrier in the middle of two writing operations, which564makes sure that recording a sample must be prior to updating the head565pointer.566 567(C) pairs with (B). (C) is a read memory barrier to ensure the head568pointer is fetched before reading samples.569 570To implement the above algorithm, the ``perf_output_put_handle()`` function571in the kernel and two helpers ``ring_buffer_read_head()`` and572``ring_buffer_write_tail()`` in the user space are introduced, they rely573on memory barriers as described above to ensure the data dependency.574 575Some architectures support one-way permeable barrier with load-acquire576and store-release operations, these barriers are more relaxed with less577performance penalty, so (C) and (D) can be optimized to use barriers578``smp_load_acquire()`` and ``smp_store_release()`` respectively.579 580If an architecture doesn’t support load-acquire and store-release in its581memory model, it will roll back to the old fashion of memory barrier582operations. In this case, ``smp_load_acquire()`` encapsulates583``READ_ONCE()`` + ``smp_mb()``, since ``smp_mb()`` is costly,584``ring_buffer_read_head()`` doesn't invoke ``smp_load_acquire()`` and it uses585the barriers ``READ_ONCE()`` + ``smp_rmb()`` instead.586 5873. The mechanism of AUX ring buffer588===================================589 590In this chapter, we will explain the implementation of the AUX ring591buffer. In the first part it will discuss the connection between the592AUX ring buffer and the regular ring buffer, then the second part will593examine how the AUX ring buffer co-works with the regular ring buffer,594as well as the additional features introduced by the AUX ring buffer for595the sampling mechanism.596 5973.1 The relationship between AUX and regular ring buffers598---------------------------------------------------------599 600Generally, the AUX ring buffer is an auxiliary for the regular ring601buffer. The regular ring buffer is primarily used to store the event602samples and every event format complies with the definition in the603union ``perf_event``; the AUX ring buffer is for recording the hardware604trace data and the trace data format is hardware IP dependent.605 606The general use and advantage of the AUX ring buffer is that it is607written directly by hardware rather than by the kernel. For example,608regular profile samples that write to the regular ring buffer cause an609interrupt. Tracing execution requires a high number of samples and610using interrupts would be overwhelming for the regular ring buffer611mechanism. Having an AUX buffer allows for a region of memory more612decoupled from the kernel and written to directly by hardware tracing.613 614The AUX ring buffer reuses the same algorithm with the regular ring615buffer for the buffer management. The control structure616``perf_event_mmap_page`` extends the new fields ``aux_head`` and ``aux_tail``617for the head and tail pointers of the AUX ring buffer.618 619During the initialisation phase, besides the mmap()-ed regular ring620buffer, the perf tool invokes a second syscall in the621``auxtrace_mmap__mmap()`` function for the mmap of the AUX buffer with622non-zero file offset; ``rb_alloc_aux()`` in the kernel allocates pages623correspondingly, these pages will be deferred to map into VMA when624handling the page fault, which is the same lazy mechanism with the625regular ring buffer.626 627AUX events and AUX trace data are two different things. Let's see an628example::629 630 perf record -a -e cycles -e cs_etm/@tmc_etr0/ -- sleep 2631 632The above command enables two events: one is the event *cycles* from PMU633and another is the AUX event *cs_etm* from Arm CoreSight, both are saved634into the regular ring buffer while the CoreSight's AUX trace data is635stored in the AUX ring buffer.636 637As a result, we can see the regular ring buffer and the AUX ring buffer638are allocated in pairs. The perf in default mode allocates the regular639ring buffer and the AUX ring buffer per CPU-wise, which is the same as640the system wide mode, however, the default mode records samples only for641the profiled program, whereas the latter mode profiles for all programs642in the system. For per-thread mode, the perf tool allocates only one643regular ring buffer and one AUX ring buffer for the whole session. For644the per-CPU mode, the perf allocates two kinds of ring buffers for645selected CPUs specified by the option ``-C``.646 647The below figure demonstrates the buffers' layout in the system wide648mode; if there are any activities on one CPU, the AUX event samples and649the hardware trace data will be recorded into the dedicated buffers for650the CPU.651 652::653 654 T1 T2 T1655 +----+ +-----------+ +----+656 CPU0 |xxxx| |xxxxxxxxxxx| |xxxx|657 +----+--------------+-----------+----------+----+-------->658 | | |659 v v v660 +-----------------------------------------------------+661 | Ring buffer 0 |662 +-----------------------------------------------------+663 | | |664 v v v665 +-----------------------------------------------------+666 | AUX Ring buffer 0 |667 +-----------------------------------------------------+668 669 T1670 +-----+671 CPU1 |xxxxx|672 -----+-----+--------------------------------------------->673 |674 v675 +-----------------------------------------------------+676 | Ring buffer 1 |677 +-----------------------------------------------------+678 |679 v680 +-----------------------------------------------------+681 | AUX Ring buffer 1 |682 +-----------------------------------------------------+683 684 T1 T3685 +----+ +-------+686 CPU2 |xxxx| |xxxxxxx|687 --------------------------+----+--------+-------+-------->688 | |689 v v690 +-----------------------------------------------------+691 | Ring buffer 2 |692 +-----------------------------------------------------+693 | |694 v v695 +-----------------------------------------------------+696 | AUX Ring buffer 2 |697 +-----------------------------------------------------+698 699 T1700 +--------------+701 CPU3 |xxxxxxxxxxxxxx|702 -----------+--------------+------------------------------>703 |704 v705 +-----------------------------------------------------+706 | Ring buffer 3 |707 +-----------------------------------------------------+708 |709 v710 +-----------------------------------------------------+711 | AUX Ring buffer 3 |712 +-----------------------------------------------------+713 714 T1: Thread 1; T2: Thread 2; T3: Thread 3715 x: Thread is in running state716 717 Figure 8. AUX ring buffer for system wide mode718 7193.2 AUX events720--------------721 722Similar to ``perf_output_begin()`` and ``perf_output_end()``'s working for the723regular ring buffer, ``perf_aux_output_begin()`` and ``perf_aux_output_end()``724serve for the AUX ring buffer for processing the hardware trace data.725 726Once the hardware trace data is stored into the AUX ring buffer, the PMU727driver will stop hardware tracing by calling the ``pmu::stop()`` callback.728Similar to the regular ring buffer, the AUX ring buffer needs to apply729the memory synchronization mechanism as discussed in the section730:ref:`memory_synchronization`. Since the AUX ring buffer is managed by the731PMU driver, the barrier (B), which is a writing barrier to ensure the trace732data is externally visible prior to updating the head pointer, is asked733to be implemented in the PMU driver.734 735Then ``pmu::stop()`` can safely call the ``perf_aux_output_end()`` function to736finish two things:737 738- It fills an event ``PERF_RECORD_AUX`` into the regular ring buffer, this739 event delivers the information of the start address and data size for a740 chunk of hardware trace data has been stored into the AUX ring buffer;741 742- Since the hardware trace driver has stored new trace data into the AUX743 ring buffer, the argument *size* indicates how many bytes have been744 consumed by the hardware tracing, thus ``perf_aux_output_end()`` updates the745 header pointer ``perf_buffer::aux_head`` to reflect the latest buffer usage.746 747At the end, the PMU driver will restart hardware tracing. During this748temporary suspending period, it will lose hardware trace data, which749will introduce a discontinuity during decoding phase.750 751The event ``PERF_RECORD_AUX`` presents an AUX event which is handled in the752kernel, but it lacks the information for saving the AUX trace data in753the perf file. When the perf tool copies the trace data from AUX ring754buffer to the perf data file, it synthesizes a ``PERF_RECORD_AUXTRACE``755event which is not a kernel ABI, it's defined by the perf tool to describe756which portion of data in the AUX ring buffer is saved. Afterwards, the perf757tool reads out the AUX trace data from the perf file based on the758``PERF_RECORD_AUXTRACE`` events, and the ``PERF_RECORD_AUX`` event is used to759decode a chunk of data by correlating with time order.760 7613.3 Snapshot mode762-----------------763 764Perf supports snapshot mode for AUX ring buffer, in this mode, users765only record AUX trace data at a specific time point which users are766interested in. E.g. below gives an example of how to take snapshots767with 1 second interval with Arm CoreSight::768 769 perf record -e cs_etm/@tmc_etr0/u -S -a program &770 PERFPID=$!771 while true; do772 kill -USR2 $PERFPID773 sleep 1774 done775 776The main flow for snapshot mode is:777 778- Before a snapshot is taken, the AUX ring buffer acts in free run mode.779 During free run mode the perf doesn't record any of the AUX events and780 trace data;781 782- Once the perf tool receives the *USR2* signal, it triggers the callback783 function ``auxtrace_record::snapshot_start()`` to deactivate hardware784 tracing. The kernel driver then populates the AUX ring buffer with the785 hardware trace data, and the event ``PERF_RECORD_AUX`` is stored in the786 regular ring buffer;787 788- Then perf tool takes a snapshot, ``record__read_auxtrace_snapshot()``789 reads out the hardware trace data from the AUX ring buffer and saves it790 into perf data file;791 792- After the snapshot is finished, ``auxtrace_record::snapshot_finish()``793 restarts the PMU event for AUX tracing.794 795The perf only accesses the head pointer ``perf_event_mmap_page::aux_head``796in snapshot mode and doesn’t touch tail pointer ``aux_tail``, this is797because the AUX ring buffer can overflow in free run mode, the tail798pointer is useless in this case. Alternatively, the callback799``auxtrace_record::find_snapshot()`` is introduced for making the decision800of whether the AUX ring buffer has been wrapped around or not, at the801end it fixes up the AUX buffer's head which are used to calculate the802trace data size.803 804As we know, the buffers' deployment can be per-thread mode, per-CPU805mode, or system wide mode, and the snapshot can be applied to any of806these modes. Below is an example of taking snapshot with system wide807mode.808 809::810 811 Snapshot is taken812 |813 v814 +------------------------+815 | AUX Ring buffer 0 | <- aux_head816 +------------------------+817 v818 +--------------------------------+819 | AUX Ring buffer 1 | <- aux_head820 +--------------------------------+821 v822 +--------------------------------------------+823 | AUX Ring buffer 2 | <- aux_head824 +--------------------------------------------+825 v826 +---------------------------------------+827 | AUX Ring buffer 3 | <- aux_head828 +---------------------------------------+829 830 Figure 9. Snapshot with system wide mode831