288 lines · plain
1.. SPDX-License-Identifier: GPL-2.02.. include:: <isonum.txt>3 4==============================================5``intel_idle`` CPU Idle Time Management Driver6==============================================7 8:Copyright: |copy| 2020 Intel Corporation9 10:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>11 12 13General Information14===================15 16``intel_idle`` is a part of the17:doc:`CPU idle time management subsystem <cpuidle>` in the Linux kernel18(``CPUIdle``). It is the default CPU idle time management driver for the19Nehalem and later generations of Intel processors, but the level of support for20a particular processor model in it depends on whether or not it recognizes that21processor model and may also depend on information coming from the platform22firmware. [To understand ``intel_idle`` it is necessary to know how ``CPUIdle``23works in general, so this is the time to get familiar with24Documentation/admin-guide/pm/cpuidle.rst if you have not done that yet.]25 26``intel_idle`` uses the ``MWAIT`` instruction to inform the processor that the27logical CPU executing it is idle and so it may be possible to put some of the28processor's functional blocks into low-power states. That instruction takes two29arguments (passed in the ``EAX`` and ``ECX`` registers of the target CPU), the30first of which, referred to as a *hint*, can be used by the processor to31determine what can be done (for details refer to Intel Software Developer’s32Manual [1]_). Accordingly, ``intel_idle`` refuses to work with processors in33which the support for the ``MWAIT`` instruction has been disabled (for example,34via the platform firmware configuration menu) or which do not support that35instruction at all.36 37``intel_idle`` is not modular, so it cannot be unloaded, which means that the38only way to pass early-configuration-time parameters to it is via the kernel39command line.40 41 42.. _intel-idle-enumeration-of-states:43 44Enumeration of Idle States45==========================46 47Each ``MWAIT`` hint value is interpreted by the processor as a license to48reconfigure itself in a certain way in order to save energy. The processor49configurations (with reduced power draw) resulting from that are referred to50as C-states (in the ACPI terminology) or idle states. The list of meaningful51``MWAIT`` hint values and idle states (i.e. low-power configurations of the52processor) corresponding to them depends on the processor model and it may also53depend on the configuration of the platform.54 55In order to create a list of available idle states required by the ``CPUIdle``56subsystem (see :ref:`idle-states-representation` in57Documentation/admin-guide/pm/cpuidle.rst),58``intel_idle`` can use two sources of information: static tables of idle states59for different processor models included in the driver itself and the ACPI tables60of the system. The former are always used if the processor model at hand is61recognized by ``intel_idle`` and the latter are used if that is required for62the given processor model (which is the case for all server processor models63recognized by ``intel_idle``) or if the processor model is not recognized.64[There is a module parameter that can be used to make the driver use the ACPI65tables with any processor model recognized by it; see66`below <intel-idle-parameters_>`_.]67 68If the ACPI tables are going to be used for building the list of available idle69states, ``intel_idle`` first looks for a ``_CST`` object under one of the ACPI70objects corresponding to the CPUs in the system (refer to the ACPI specification71[2]_ for the description of ``_CST`` and its output package). Because the72``CPUIdle`` subsystem expects that the list of idle states supplied by the73driver will be suitable for all of the CPUs handled by it and ``intel_idle`` is74registered as the ``CPUIdle`` driver for all of the CPUs in the system, the75driver looks for the first ``_CST`` object returning at least one valid idle76state description and such that all of the idle states included in its return77package are of the FFH (Functional Fixed Hardware) type, which means that the78``MWAIT`` instruction is expected to be used to tell the processor that it can79enter one of them. The return package of that ``_CST`` is then assumed to be80applicable to all of the other CPUs in the system and the idle state81descriptions extracted from it are stored in a preliminary list of idle states82coming from the ACPI tables. [This step is skipped if ``intel_idle`` is83configured to ignore the ACPI tables; see `below <intel-idle-parameters_>`_.]84 85Next, the first (index 0) entry in the list of available idle states is86initialized to represent a "polling idle state" (a pseudo-idle state in which87the target CPU continuously fetches and executes instructions), and the88subsequent (real) idle state entries are populated as follows.89 90If the processor model at hand is recognized by ``intel_idle``, there is a91(static) table of idle state descriptions for it in the driver. In that case,92the "internal" table is the primary source of information on idle states and the93information from it is copied to the final list of available idle states. If94using the ACPI tables for the enumeration of idle states is not required95(depending on the processor model), all of the listed idle state are enabled by96default (so all of them will be taken into consideration by ``CPUIdle``97governors during CPU idle state selection). Otherwise, some of the listed idle98states may not be enabled by default if there are no matching entries in the99preliminary list of idle states coming from the ACPI tables. In that case user100space still can enable them later (on a per-CPU basis) with the help of101the ``disable`` idle state attribute in ``sysfs`` (see102:ref:`idle-states-representation` in103Documentation/admin-guide/pm/cpuidle.rst). This basically means that104the idle states "known" to the driver may not be enabled by default if they have105not been exposed by the platform firmware (through the ACPI tables).106 107If the given processor model is not recognized by ``intel_idle``, but it108supports ``MWAIT``, the preliminary list of idle states coming from the ACPI109tables is used for building the final list that will be supplied to the110``CPUIdle`` core during driver registration. For each idle state in that list,111the description, ``MWAIT`` hint and exit latency are copied to the corresponding112entry in the final list of idle states. The name of the idle state represented113by it (to be returned by the ``name`` idle state attribute in ``sysfs``) is114"CX_ACPI", where X is the index of that idle state in the final list (note that115the minimum value of X is 1, because 0 is reserved for the "polling" state), and116its target residency is based on the exit latency value. Specifically, for117C1-type idle states the exit latency value is also used as the target residency118(for compatibility with the majority of the "internal" tables of idle states for119various processor models recognized by ``intel_idle``) and for the other idle120state types (C2 and C3) the target residency value is 3 times the exit latency121(again, that is because it reflects the target residency to exit latency ratio122in the majority of cases for the processor models recognized by ``intel_idle``).123All of the idle states in the final list are enabled by default in this case.124 125 126.. _intel-idle-initialization:127 128Initialization129==============130 131The initialization of ``intel_idle`` starts with checking if the kernel command132line options forbid the use of the ``MWAIT`` instruction. If that is the case,133an error code is returned right away.134 135The next step is to check whether or not the processor model is known to the136driver, which determines the idle states enumeration method (see137`above <intel-idle-enumeration-of-states_>`_), and whether or not the processor138supports ``MWAIT`` (the initialization fails if that is not the case). Then,139the ``MWAIT`` support in the processor is enumerated through ``CPUID`` and the140driver initialization fails if the level of support is not as expected (for141example, if the total number of ``MWAIT`` substates returned is 0).142 143Next, if the driver is not configured to ignore the ACPI tables (see144`below <intel-idle-parameters_>`_), the idle states information provided by the145platform firmware is extracted from them.146 147Then, ``CPUIdle`` device objects are allocated for all CPUs and the list of148available idle states is created as explained149`above <intel-idle-enumeration-of-states_>`_.150 151Finally, ``intel_idle`` is registered with the help of cpuidle_register_driver()152as the ``CPUIdle`` driver for all CPUs in the system and a CPU online callback153for configuring individual CPUs is registered via cpuhp_setup_state(), which154(among other things) causes the callback routine to be invoked for all of the155CPUs present in the system at that time (each CPU executes its own instance of156the callback routine). That routine registers a ``CPUIdle`` device for the CPU157running it (which enables the ``CPUIdle`` subsystem to operate that CPU) and158optionally performs some CPU-specific initialization actions that may be159required for the given processor model.160 161 162.. _intel-idle-parameters:163 164Kernel Command Line Options and Module Parameters165=================================================166 167The *x86* architecture support code recognizes three kernel command line168options related to CPU idle time management: ``idle=poll``, ``idle=halt``,169and ``idle=nomwait``. If any of them is present in the kernel command line, the170``MWAIT`` instruction is not allowed to be used, so the initialization of171``intel_idle`` will fail.172 173Apart from that there are five module parameters recognized by ``intel_idle``174itself that can be set via the kernel command line (they cannot be updated via175sysfs, so that is the only way to change their values).176 177The ``max_cstate`` parameter value is the maximum idle state index in the list178of idle states supplied to the ``CPUIdle`` core during the registration of the179driver. It is also the maximum number of regular (non-polling) idle states that180can be used by ``intel_idle``, so the enumeration of idle states is terminated181after finding that number of usable idle states (the other idle states that182potentially might have been used if ``max_cstate`` had been greater are not183taken into consideration at all). Setting ``max_cstate`` can prevent184``intel_idle`` from exposing idle states that are regarded as "too deep" for185some reason to the ``CPUIdle`` core, but it does so by making them effectively186invisible until the system is shut down and started again which may not always187be desirable. In practice, it is only really necessary to do that if the idle188states in question cannot be enabled during system startup, because in the189working state of the system the CPU power management quality of service (PM190QoS) feature can be used to prevent ``CPUIdle`` from touching those idle states191even if they have been enumerated (see :ref:`cpu-pm-qos` in192Documentation/admin-guide/pm/cpuidle.rst).193Setting ``max_cstate`` to 0 causes the ``intel_idle`` initialization to fail.194 195The ``no_acpi`` and ``use_acpi`` module parameters (recognized by ``intel_idle``196if the kernel has been configured with ACPI support) can be set to make the197driver ignore the system's ACPI tables entirely or use them for all of the198recognized processor models, respectively (they both are unset by default and199``use_acpi`` has no effect if ``no_acpi`` is set).200 201The value of the ``states_off`` module parameter (0 by default) represents a202list of idle states to be disabled by default in the form of a bitmask.203 204Namely, the positions of the bits that are set in the ``states_off`` value are205the indices of idle states to be disabled by default (as reflected by the names206of the corresponding idle state directories in ``sysfs``, :file:`state0`,207:file:`state1` ... :file:`state<i>` ..., where ``<i>`` is the index of the given208idle state; see :ref:`idle-states-representation` in209Documentation/admin-guide/pm/cpuidle.rst).210 211For example, if ``states_off`` is equal to 3, the driver will disable idle212states 0 and 1 by default, and if it is equal to 8, idle state 3 will be213disabled by default and so on (bit positions beyond the maximum idle state index214are ignored).215 216The idle states disabled this way can be enabled (on a per-CPU basis) from user217space via ``sysfs``.218 219The ``ibrs_off`` module parameter is a boolean flag (defaults to220false). If set, it is used to control if IBRS (Indirect Branch Restricted221Speculation) should be turned off when the CPU enters an idle state.222This flag does not affect CPUs that use Enhanced IBRS which can remain223on with little performance impact.224 225For some CPUs, IBRS will be selected as mitigation for Spectre v2 and Retbleed226security vulnerabilities by default. Leaving the IBRS mode on while idling may227have a performance impact on its sibling CPU. The IBRS mode will be turned off228by default when the CPU enters into a deep idle state, but not in some229shallower ones. Setting the ``ibrs_off`` module parameter will force the IBRS230mode to off when the CPU is in any one of the available idle states. This may231help performance of a sibling CPU at the expense of a slightly higher wakeup232latency for the idle CPU.233 234 235.. _intel-idle-core-and-package-idle-states:236 237Core and Package Levels of Idle States238======================================239 240Typically, in a processor supporting the ``MWAIT`` instruction there are (at241least) two levels of idle states (or C-states). One level, referred to as242"core C-states", covers individual cores in the processor, whereas the other243level, referred to as "package C-states", covers the entire processor package244and it may also involve other components of the system (GPUs, memory245controllers, I/O hubs etc.).246 247Some of the ``MWAIT`` hint values allow the processor to use core C-states only248(most importantly, that is the case for the ``MWAIT`` hint value corresponding249to the ``C1`` idle state), but the majority of them give it a license to put250the target core (i.e. the core containing the logical CPU executing ``MWAIT``251with the given hint value) into a specific core C-state and then (if possible)252to enter a specific package C-state at the deeper level. For example, the253``MWAIT`` hint value representing the ``C3`` idle state allows the processor to254put the target core into the low-power state referred to as "core ``C3``" (or255``CC3``), which happens if all of the logical CPUs (SMT siblings) in that core256have executed ``MWAIT`` with the ``C3`` hint value (or with a hint value257representing a deeper idle state), and in addition to that (in the majority of258cases) it gives the processor a license to put the entire package (possibly259including some non-CPU components such as a GPU or a memory controller) into the260low-power state referred to as "package ``C3``" (or ``PC3``), which happens if261all of the cores have gone into the ``CC3`` state and (possibly) some additional262conditions are satisfied (for instance, if the GPU is covered by ``PC3``, it may263be required to be in a certain GPU-specific low-power state for ``PC3`` to be264reachable).265 266As a rule, there is no simple way to make the processor use core C-states only267if the conditions for entering the corresponding package C-states are met, so268the logical CPU executing ``MWAIT`` with a hint value that is not core-level269only (like for ``C1``) must always assume that this may cause the processor to270enter a package C-state. [That is why the exit latency and target residency271values corresponding to the majority of ``MWAIT`` hint values in the "internal"272tables of idle states in ``intel_idle`` reflect the properties of package273C-states.] If using package C-states is not desirable at all, either274:ref:`PM QoS <cpu-pm-qos>` or the ``max_cstate`` module parameter of275``intel_idle`` described `above <intel-idle-parameters_>`_ must be used to276restrict the range of permissible idle states to the ones with core-level only277``MWAIT`` hint values (like ``C1``).278 279 280References281==========282 283.. [1] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 2B*,284 https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-vol-2b-manual.html285 286.. [2] *Advanced Configuration and Power Interface (ACPI) Specification*,287 https://uefi.org/specifications288