420 lines · plain
1.. SPDX-License-Identifier: GPL-2.02 3=======================4Energy Model of devices5=======================6 71. Overview8-----------9 10The Energy Model (EM) framework serves as an interface between drivers knowing11the power consumed by devices at various performance levels, and the kernel12subsystems willing to use that information to make energy-aware decisions.13 14The source of the information about the power consumed by devices can vary greatly15from one platform to another. These power costs can be estimated using16devicetree data in some cases. In others, the firmware will know better.17Alternatively, userspace might be best positioned. And so on. In order to avoid18each and every client subsystem to re-implement support for each and every19possible source of information on its own, the EM framework intervenes as an20abstraction layer which standardizes the format of power cost tables in the21kernel, hence enabling to avoid redundant work.22 23The power values might be expressed in micro-Watts or in an 'abstract scale'.24Multiple subsystems might use the EM and it is up to the system integrator to25check that the requirements for the power value scale types are met. An example26can be found in the Energy-Aware Scheduler documentation27Documentation/scheduler/sched-energy.rst. For some subsystems like thermal or28powercap power values expressed in an 'abstract scale' might cause issues.29These subsystems are more interested in estimation of power used in the past,30thus the real micro-Watts might be needed. An example of these requirements can31be found in the Intelligent Power Allocation in32Documentation/driver-api/thermal/power_allocator.rst.33Kernel subsystems might implement automatic detection to check whether EM34registered devices have inconsistent scale (based on EM internal flag).35Important thing to keep in mind is that when the power values are expressed in36an 'abstract scale' deriving real energy in micro-Joules would not be possible.37 38The figure below depicts an example of drivers (Arm-specific here, but the39approach is applicable to any architecture) providing power costs to the EM40framework, and interested clients reading the data from it::41 42 +---------------+ +-----------------+ +---------------+43 | Thermal (IPA) | | Scheduler (EAS) | | Other |44 +---------------+ +-----------------+ +---------------+45 | | em_cpu_energy() |46 | | em_cpu_get() |47 +---------+ | +---------+48 | | |49 v v v50 +---------------------+51 | Energy Model |52 | Framework |53 +---------------------+54 ^ ^ ^55 | | | em_dev_register_perf_domain()56 +----------+ | +---------+57 | | |58 +---------------+ +---------------+ +--------------+59 | cpufreq-dt | | arm_scmi | | Other |60 +---------------+ +---------------+ +--------------+61 ^ ^ ^62 | | |63 +--------------+ +---------------+ +--------------+64 | Device Tree | | Firmware | | ? |65 +--------------+ +---------------+ +--------------+66 67In case of CPU devices the EM framework manages power cost tables per68'performance domain' in the system. A performance domain is a group of CPUs69whose performance is scaled together. Performance domains generally have a701-to-1 mapping with CPUFreq policies. All CPUs in a performance domain are71required to have the same micro-architecture. CPUs in different performance72domains can have different micro-architectures.73 74To better reflect power variation due to static power (leakage) the EM75supports runtime modifications of the power values. The mechanism relies on76RCU to free the modifiable EM perf_state table memory. Its user, the task77scheduler, also uses RCU to access this memory. The EM framework provides78API for allocating/freeing the new memory for the modifiable EM table.79The old memory is freed automatically using RCU callback mechanism when there80are no owners anymore for the given EM runtime table instance. This is tracked81using kref mechanism. The device driver which provided the new EM at runtime,82should call EM API to free it safely when it's no longer needed. The EM83framework will handle the clean-up when it's possible.84 85The kernel code which want to modify the EM values is protected from concurrent86access using a mutex. Therefore, the device driver code must run in sleeping87context when it tries to modify the EM.88 89With the runtime modifiable EM we switch from a 'single and during the entire90runtime static EM' (system property) design to a 'single EM which can be91changed during runtime according e.g. to the workload' (system and workload92property) design.93 94It is possible also to modify the CPU performance values for each EM's95performance state. Thus, the full power and performance profile (which96is an exponential curve) can be changed according e.g. to the workload97or system property.98 99 1002. Core APIs101------------102 1032.1 Config options104^^^^^^^^^^^^^^^^^^105 106CONFIG_ENERGY_MODEL must be enabled to use the EM framework.107 108 1092.2 Registration of performance domains110^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^111 112Registration of 'advanced' EM113~~~~~~~~~~~~~~~~~~~~~~~~~~~~~114 115The 'advanced' EM gets its name due to the fact that the driver is allowed116to provide more precised power model. It's not limited to some implemented math117formula in the framework (like it is in 'simple' EM case). It can better reflect118the real power measurements performed for each performance state. Thus, this119registration method should be preferred in case considering EM static power120(leakage) is important.121 122Drivers are expected to register performance domains into the EM framework by123calling the following API::124 125 int em_dev_register_perf_domain(struct device *dev, unsigned int nr_states,126 struct em_data_callback *cb, cpumask_t *cpus, bool microwatts);127 128Drivers must provide a callback function returning <frequency, power> tuples129for each performance state. The callback function provided by the driver is free130to fetch data from any relevant location (DT, firmware, ...), and by any mean131deemed necessary. Only for CPU devices, drivers must specify the CPUs of the132performance domains using cpumask. For other devices than CPUs the last133argument must be set to NULL.134The last argument 'microwatts' is important to set with correct value. Kernel135subsystems which use EM might rely on this flag to check if all EM devices use136the same scale. If there are different scales, these subsystems might decide137to return warning/error, stop working or panic.138See Section 3. for an example of driver implementing this139callback, or Section 2.4 for further documentation on this API140 141Registration of EM using DT142~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~143 144The EM can also be registered using OPP framework and information in DT145"operating-points-v2". Each OPP entry in DT can be extended with a property146"opp-microwatt" containing micro-Watts power value. This OPP DT property147allows a platform to register EM power values which are reflecting total power148(static + dynamic). These power values might be coming directly from149experiments and measurements.150 151Registration of 'artificial' EM152~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~153 154There is an option to provide a custom callback for drivers missing detailed155knowledge about power value for each performance state. The callback156.get_cost() is optional and provides the 'cost' values used by the EAS.157This is useful for platforms that only provide information on relative158efficiency between CPU types, where one could use the information to159create an abstract power model. But even an abstract power model can160sometimes be hard to fit in, given the input power value size restrictions.161The .get_cost() allows to provide the 'cost' values which reflect the162efficiency of the CPUs. This would allow to provide EAS information which163has different relation than what would be forced by the EM internal164formulas calculating 'cost' values. To register an EM for such platform, the165driver must set the flag 'microwatts' to 0, provide .get_power() callback166and provide .get_cost() callback. The EM framework would handle such platform167properly during registration. A flag EM_PERF_DOMAIN_ARTIFICIAL is set for such168platform. Special care should be taken by other frameworks which are using EM169to test and treat this flag properly.170 171Registration of 'simple' EM172~~~~~~~~~~~~~~~~~~~~~~~~~~~173 174The 'simple' EM is registered using the framework helper function175cpufreq_register_em_with_opp(). It implements a power model which is tight to176math formula::177 178 Power = C * V^2 * f179 180The EM which is registered using this method might not reflect correctly the181physics of a real device, e.g. when static power (leakage) is important.182 183 1842.3 Accessing performance domains185^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^186 187There are two API functions which provide the access to the energy model:188em_cpu_get() which takes CPU id as an argument and em_pd_get() with device189pointer as an argument. It depends on the subsystem which interface it is190going to use, but in case of CPU devices both functions return the same191performance domain.192 193Subsystems interested in the energy model of a CPU can retrieve it using the194em_cpu_get() API. The energy model tables are allocated once upon creation of195the performance domains, and kept in memory untouched.196 197The energy consumed by a performance domain can be estimated using the198em_cpu_energy() API. The estimation is performed assuming that the schedutil199CPUfreq governor is in use in case of CPU device. Currently this calculation is200not provided for other type of devices.201 202More details about the above APIs can be found in ``<linux/energy_model.h>``203or in Section 2.5204 205 2062.4 Runtime modifications207^^^^^^^^^^^^^^^^^^^^^^^^^208 209Drivers willing to update the EM at runtime should use the following dedicated210function to allocate a new instance of the modified EM. The API is listed211below::212 213 struct em_perf_table __rcu *em_table_alloc(struct em_perf_domain *pd);214 215This allows to allocate a structure which contains the new EM table with216also RCU and kref needed by the EM framework. The 'struct em_perf_table'217contains array 'struct em_perf_state state[]' which is a list of performance218states in ascending order. That list must be populated by the device driver219which wants to update the EM. The list of frequencies can be taken from220existing EM (created during boot). The content in the 'struct em_perf_state'221must be populated by the driver as well.222 223This is the API which does the EM update, using RCU pointers swap::224 225 int em_dev_update_perf_domain(struct device *dev,226 struct em_perf_table __rcu *new_table);227 228Drivers must provide a pointer to the allocated and initialized new EM229'struct em_perf_table'. That new EM will be safely used inside the EM framework230and will be visible to other sub-systems in the kernel (thermal, powercap).231The main design goal for this API is to be fast and avoid extra calculations232or memory allocations at runtime. When pre-computed EMs are available in the233device driver, than it should be possible to simply re-use them with low234performance overhead.235 236In order to free the EM, provided earlier by the driver (e.g. when the module237is unloaded), there is a need to call the API::238 239 void em_table_free(struct em_perf_table __rcu *table);240 241It will allow the EM framework to safely remove the memory, when there is242no other sub-system using it, e.g. EAS.243 244To use the power values in other sub-systems (like thermal, powercap) there is245a need to call API which protects the reader and provide consistency of the EM246table data::247 248 struct em_perf_state *em_perf_state_from_pd(struct em_perf_domain *pd);249 250It returns the 'struct em_perf_state' pointer which is an array of performance251states in ascending order.252This function must be called in the RCU read lock section (after the253rcu_read_lock()). When the EM table is not needed anymore there is a need to254call rcu_real_unlock(). In this way the EM safely uses the RCU read section255and protects the users. It also allows the EM framework to manage the memory256and free it. More details how to use it can be found in Section 3.2 in the257example driver.258 259There is dedicated API for device drivers to calculate em_perf_state::cost260values::261 262 int em_dev_compute_costs(struct device *dev, struct em_perf_state *table,263 int nr_states);264 265These 'cost' values from EM are used in EAS. The new EM table should be passed266together with the number of entries and device pointer. When the computation267of the cost values is done properly the return value from the function is 0.268The function takes care for right setting of inefficiency for each performance269state as well. It updates em_perf_state::flags accordingly.270Then such prepared new EM can be passed to the em_dev_update_perf_domain()271function, which will allow to use it.272 273More details about the above APIs can be found in ``<linux/energy_model.h>``274or in Section 3.2 with an example code showing simple implementation of the275updating mechanism in a device driver.276 277 2782.5 Description details of this API279^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^280.. kernel-doc:: include/linux/energy_model.h281 :internal:282 283.. kernel-doc:: kernel/power/energy_model.c284 :export:285 286 2873. Examples288-----------289 2903.1 Example driver with EM registration291^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^292 293The CPUFreq framework supports dedicated callback for registering294the EM for a given CPU(s) 'policy' object: cpufreq_driver::register_em().295That callback has to be implemented properly for a given driver,296because the framework would call it at the right time during setup.297This section provides a simple example of a CPUFreq driver registering a298performance domain in the Energy Model framework using the (fake) 'foo'299protocol. The driver implements an est_power() function to be provided to the300EM framework::301 302 -> drivers/cpufreq/foo_cpufreq.c303 304 01 static int est_power(struct device *dev, unsigned long *mW,305 02 unsigned long *KHz)306 03 {307 04 long freq, power;308 05309 06 /* Use the 'foo' protocol to ceil the frequency */310 07 freq = foo_get_freq_ceil(dev, *KHz);311 08 if (freq < 0);312 09 return freq;313 10314 11 /* Estimate the power cost for the dev at the relevant freq. */315 12 power = foo_estimate_power(dev, freq);316 13 if (power < 0);317 14 return power;318 15319 16 /* Return the values to the EM framework */320 17 *mW = power;321 18 *KHz = freq;322 19323 20 return 0;324 21 }325 22326 23 static void foo_cpufreq_register_em(struct cpufreq_policy *policy)327 24 {328 25 struct em_data_callback em_cb = EM_DATA_CB(est_power);329 26 struct device *cpu_dev;330 27 int nr_opp;331 28332 29 cpu_dev = get_cpu_device(cpumask_first(policy->cpus));333 30334 31 /* Find the number of OPPs for this policy */335 32 nr_opp = foo_get_nr_opp(policy);336 33337 34 /* And register the new performance domain */338 35 em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb, policy->cpus,339 36 true);340 37 }341 38342 39 static struct cpufreq_driver foo_cpufreq_driver = {343 40 .register_em = foo_cpufreq_register_em,344 41 };345 346 3473.2 Example driver with EM modification348^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^349 350This section provides a simple example of a thermal driver modifying the EM.351The driver implements a foo_thermal_em_update() function. The driver is woken352up periodically to check the temperature and modify the EM data::353 354 -> drivers/soc/example/example_em_mod.c355 356 01 static void foo_get_new_em(struct foo_context *ctx)357 02 {358 03 struct em_perf_table __rcu *em_table;359 04 struct em_perf_state *table, *new_table;360 05 struct device *dev = ctx->dev;361 06 struct em_perf_domain *pd;362 07 unsigned long freq;363 08 int i, ret;364 09365 10 pd = em_pd_get(dev);366 11 if (!pd)367 12 return;368 13369 14 em_table = em_table_alloc(pd);370 15 if (!em_table)371 16 return;372 17373 18 new_table = em_table->state;374 19375 20 rcu_read_lock();376 21 table = em_perf_state_from_pd(pd);377 22 for (i = 0; i < pd->nr_perf_states; i++) {378 23 freq = table[i].frequency;379 24 foo_get_power_perf_values(dev, freq, &new_table[i]);380 25 }381 26 rcu_read_unlock();382 27383 28 /* Calculate 'cost' values for EAS */384 29 ret = em_dev_compute_costs(dev, table, pd->nr_perf_states);385 30 if (ret) {386 31 dev_warn(dev, "EM: compute costs failed %d\n", ret);387 32 em_free_table(em_table);388 33 return;389 34 }390 35391 36 ret = em_dev_update_perf_domain(dev, em_table);392 37 if (ret) {393 38 dev_warn(dev, "EM: update failed %d\n", ret);394 39 em_free_table(em_table);395 40 return;396 41 }397 42398 43 /*399 44 * Since it's one-time-update drop the usage counter.400 45 * The EM framework will later free the table when needed.401 46 */402 47 em_table_free(em_table);403 48 }404 49405 50 /*406 51 * Function called periodically to check the temperature and407 52 * update the EM if needed408 53 */409 54 static void foo_thermal_em_update(struct foo_context *ctx)410 55 {411 56 struct device *dev = ctx->dev;412 57 int cpu;413 58414 59 ctx->temperature = foo_get_temp(dev, ctx);415 60 if (ctx->temperature < FOO_EM_UPDATE_TEMP_THRESHOLD)416 61 return;417 62418 63 foo_get_new_em(ctx);419 64 }420