When __cpufreq_driver_target() in speedchange_task failed for some reason, the
policy->cur could be lower than the target_freq. The governor misses to change
the target_freq if the target_freq is equal to the next_freq at the next sample
time.
Added a check to prevent the CPU to stay at the speed that is lower than the
target_freq for long duration.
Change-Id: Ibfdcd193b8280390b8f8374a63218aa31267f310
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
The change is to compile on kernels where cpufreq stats are compiled as
a module (CONFIG_CPU_FREQ_STAT=m), because total_cpus is not exported for
module use.
Reported-By: Emilio López <elopez93@gmail.com>
Signed-off-by: Ruchi Kandoi <kandoiruchi@google.com>
Change-Id: I4f3c74f0fac5e8d9449655b26bf3b407b0fe4290
From: Cylen Yao <cylen.yao@mediatek.com>
common_tunables should be static.
Change-Id: I502ee3062bece5082fea7861eff2f6237e25cede
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Cpufreq time_in_state data for all CPUs is made persistent across
hotplug and exposed to userspace via sysfs file
/sys/devices/system/cpu/cpufreq/all_time_in_state
Change-Id: I97cb5de24b6de16189bf8b5df9592d0a6e6ddf32
Signed-off-by: Ruchi Kandoi <kandoiruchi@google.com>
hispeed_freq from a lower frequency.
When the load was below go_hispeed_load, there is a possibility that
choose_freq() would return a frequency which would be higher than the
hispeed_freq. According to the policy we should first jump to the
hispeed_freq, stay there for above_hispeed_delay and then be allowed to
raise higher than that.
Added a check to prevent the frequency to be directly raised to
something higher than the hispeed_freq.
Change-Id: Icda5d848dd9beadcc18835082ddf269732c75bd0
Signed-off-by: Ruchi Kandoi <kandoiruchi@google.com>
The cpufreq_interactive_timer gets cancelled and rescheduled
whenever the cpufreq_policy is changed. When the cpufreq policy is
changed at a rate faster than the sampling_rate of the interactive
governor, then the governor misses to change the target frequency
for long duration. The patch removes the need of cancelling the
timers when policy->min is changed.
Signed-off-by: Badhri Jagan Sridharan <Badhri@google.com>
Change-Id: Ibd98d151e1c73b8bd969484583ff98ee9f1135ef
Clear ->cur_policy when stopping a governor, or the ->cur_policy
pointer may be stale on systems with have_governor_per_policy when a
new policy is allocated due to CPU hotplug offline/online.
[rjw: Changelog]
Suggested-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Jacob Shin <jacob.shin@amd.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
get_governor_parent_kobj() can be used by any governor, generic
cpufreq governors or platform specific ones and so must be present in
cpufreq.c instead of cpufreq_governor.c.
This patch moves it to cpufreq.c. This also adds
EXPORT_SYMBOL_GPL(get_governor_parent_kobj) so that modules can use
this function too.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
This patch adds: EXPORT_SYMBOL_GPL(have_governor_per_policy), so that
this routine can be used by modules too.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
sysfs ops for target_loads and above_hispeed_delay can be called before
initializing tunables at CPUFREQ_GOV_POLICY_INIT. Create sysfs entries after
initialization.
Change-Id: I50356198d7629731c0d32a3066d61fe8354e0001
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
The gcc warns like:
cpufreq_interactive.c:745:6: warning: operation on 'ret' may be undefined [-Wsequence-point]
It was introduced by commit cf0fad49d1.
Since sprintf(...) just return 1 (one character) in this case, ret should not changed.
Just discarding the result of sprintf(...) leads to the result that
the committer of cf0fad49d1 wants.
Change-Id: Ifed1cef6d6a31c3ed23dad03a567b3b9eddf3a57
Signed-off-by: Chih-Wei Huang <cwhuang@android-x86.org>
Make sure that timers cpu_timer and cpu_slack_timer
deactivated before addition of new.
Change-Id: If31c4049606871df6f00efdc24b1d713c86a6f69
Signed-off-by: Shridhar Rasal <srasal@nvidia.com>
Signed-off-by: Bharat Nihalani <bnihalani@nvidia.com>
If we have a multi-package system, where we have multiple instances of struct
policy (per package), currently we can't have multiple instances of same
governor. i.e. We can't have multiple instances of Interactive governor for
multiple packages.
This is a bottleneck for multicluster system, where we want different packages
to use Interactive governor, but with different tunables.
This patch uses the infrastructure provided by earlier patches pushed in
Mainline in v3.10-rc1/rc2 and implements per policy instances of Interactive
governor.
Change-Id: I70436d4a5a45c6cb6edf37f3e46d0b9fbc930982
[toddpoynor@google.com: merge with later code, minor changes]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
This moves definition of cpufreq_gov_interactive towards the bottom of file, so
that we don't have to add prototype of cpufreq_governor_interactive() in the
beginning of file.
Change-Id: I04bd1004954eb36502c5cd7e35d3d7274cddaf95
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Cpufreq no longer calls governor callback for offlined cpus. i.e. All
policy->cpus are guaranteed to be online. Hence we don't need explicit check to
see if cpu is online or not.
Change-Id: I9ad85ea4addd5b4a40952e59ed730dd15e328690
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Remove a trailing whitespace from target_loads and above_hispeed_delay. Problem
happens when user-space program tried to restore parameters that saved before
changing parameters. In this case was returned error(EINVAL).
Change-Id: I5a74e3824602cd6f2b74651adda5ec1b627e61e9
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
When the policy max freq is raised, and before the timer is
rescheduled in idle callback, the cpu freq may stuck at a
lower freq.
The target_freq shall be updated too, else on a high load
situation, the new_freq is always equal to target_freq and
which will cause freq stuck at a lower freq too.
Reschedule the timer on gov limits callback.
Change-Id: I6c187001ab43e859731429b64f75a74eebc37a24
Signed-off-by: Lianwei Wang <a22439@motorola.com>
The cpufreq TRANSTION notifier callback does not check the
governor_enabled state on affected CPUS, which will case
kernel panic in update_load because the policy object maybe
NULL or invalid when governor_enabled is false.
Change-Id: Ie0f1718124f61e2f9b5da57abc6981ada5b83908
Signed-off-by: Lianwei Wang <a22439@motorola.com>
Check for idle time delta less than elapsed time delta, avoid
underflow computing active time.
Change-Id: I3e4c6ef1ad794eec49ed379c0c50fa727fd6ad28
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
Reschedule load sampling timer after timestamp of sample start taken,
hold spinlock across entire sequence to avoid preemption. Avoid the
WARN for zero time delta in the load sampling timer function.
Change-Id: Idc10a756f09141decb6df92669521a1ebf0dbc10
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Add checks for error return from cpufreq_frequency_table_target, and be
less noisy on the existing call with an error check. CPU hotplug and
system shutdown may cause this call to return -EINVAL.
Bug: 8613560
Change-Id: Id78d8829920462c0db1c7e14e717d91740d6cb44
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Time to wait should be based on the intended target speed, not the
actual speed (which may be held high by another CPU).
Change-Id: Ifc5bb55d06adddb9a02af90af05398a78f282272
Reported-by: Arve Hjønnevåg <arve@android.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Previously the idle time returned from get_cpu_idle_time_us included the
iowait time. So the iowait time was always calculated as idle time.
But now the idle time returned from get_cpu_idle_time_us does not include
the iowait time anymore because of below commit which cause the iowait time
always calculated as busy time:
6beea0c nohz: Fix update_ts_time_stat idle accounting
Add the io_is_busy interface, as does the ondemand governor, and let the user
configure the iowait time as busy or idle through the io_is_busy sysfs
interface.
By default, io_is_busy is disabled.
[toddpoynor@google.com: minor updates]
Change-Id: If7d70ff864c43bc9c8d7fd7cfc66f930d339f9b4
Signed-off-by: Lianwei Wang <lian-wei.wang@motorola.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Accept a string of delays and speeds at which to apply the delay before
raising each step above hispeed. For example, "80000 1300000:200000
1500000:40000" means that the delay at or above 1GHz, until 1.3GHz is 80 msecs,
the delay until 1.5GHz is 200 msecs and the delay at or above 1.5GHz is 40
msecs when hispeed_freq is 1GHz.
[toddpoynor@google.com: add documentation]
Change-Id: Ifeebede8b1acbdd0a53e5c6916bccbf764dc854f
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
There is race condition when both two cpu do CPUFREQ_GOV_STOP and one cpu
do CPUFREQ_GOV_START soon. The sysfs_remove_group is not done yet on one
cpu, but sysfs_create_group is called on another cpu, which cause governor
start failed and then kernel panic in timer callback because the policy and
cpu mask are all kfree in cpufreq driver.
Replace atomic with mutex to lock the whole START/STOP sequence.
Change-Id: I3762b3d44315ae021b8275aca84f5ea9147cc540
Signed-off-by: Lianwei Wang <a22439@motorola.com>
Need to use irqsave/restore spinlock calls to avoid a deadlock in calls
from the timer.
Change-Id: I15b6b590045ba1447e34ca7b5ff342723e53a605
Signed-off-by: Todd Poynor <toddpoynor@google.com>
If multiple governors are in use then avoid processing frequency transition
notifications for CPUs on which the interactive governor is not enabled.
Change-Id: Ibd75255b921d887501a64774a8c4f62302f2d4e4
Reported-by: Francisco Franco <francisco.franco@cloudcar.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Update default go_hispeed_load from 85% to 99%. Recent changes to the
governor now use a default target_load of 90%. go_hispeed_load should
not be lower than the target load for hispeed_freq, which could lead
to oscillating speed decisions. Other recent changes reduce the need
to dampen speed jumps on load spikes, while input event boosts from
userspace are the preferred method for anticipating load spikes with
UI impacts.
General update to the documentation to reflect recent changes.
Change-Id: I1b92f3091f42c04b10503cd1169a943b5dfd6faf
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Starting the governor, or restarting on a hotplugged-in CPU, can race
with the timer start in idle, triggering a BUG on timer already pending.
Start the timer before setting the enable flag, and use enable_sem to
protect the sequence (and ensure correct order of the update to the
enable flag). Delete any existing timer for safety.
Change-Id: Ife77cf9fe099e8fd8543224cbf148c6722c2ffb0
Reported-by: Francisco Franco <francisco.franco@cloudcar.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
When stopping the governor, del_timer_sync() can race against an
invocation of the idle notifier callback, which has the potential
to reactivate the timer.
To fix this issue, a read-write semaphore is used. Multiple readers are
allowed as long as pcpu->governor_enabled is true. However it can be
moved to false only after taking a write semaphore which would wait for
any on-going asynchronous activities to complete and prevent any more of
those activities to be initiated.
[toddpoynor@google.com: cosmetic and commit text changes]
Change-Id: Ib51165a735d73dcf964a06754c48bdc1913e13d0
Signed-off-by: Nicolas Pitre <nicolas.pitre@linaro.org>
35a84de cpufreq: interactive: apply above_hispeed_delay to each step above hispeed
caused the speed choice logic to osciallate between boosting and not boosting.
Add back code to ensure speed does not drop below boost frequency while
boosting.
Change-Id: Id420068480fcc7f5c4989ff523e2a8d22e2f4db2
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Always use deferrable timer for load sampling.
Set a non-deferrable timer to an additional slack time to allow prior to
waking up from idle to drop speed when not at minimum speed. Slack value
-1 avoids wakeups to drop speed. Default is 80ms.
Remove the governidle module param and its timer management in idle. For
platforms on which holding speed above mimum in idle costs power, use the
new timer slack to select how long to wait before waking up to drop speed.
Change-Id: I270b3980667e2c70a68e5bff534124b4411dbad5
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Sysfs attribute boostpulse_duration specifies the duration of boosting CPU
speed in response to bootpulse events. Duration is specified in usecs,
default 80ms.
Change-Id: Ifd41625574891a44f1787a4e85d1e7b4f2afb52b
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Add notifier for speed transitions. Keep a count of CPU active
microseconds times current frequency, converted to a percentage relative
to the current frequency when load is evaluated.
Change-Id: I5c27adb11081c50490219784ca57cc46e97fc28c
Signed-off-by: Todd Poynor <toddpoynor@google.com>
The longer-term load since last speed change isn't terribly useful,
may delay recognition of dropping load, and would need forthcoming
changes to adjust load for changing CPU speeds. Drop it.
Change-Id: Ic3cbb0542cc3484617031787e03ed9bdd632dec1
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Accept a string of target loads and speeds at which to apply the
target loads, per the documentation update in this patch. For example,
"85 1000000:90 1700000:99" targets CPU load 85% below speed 1GHz, 90%
at or above 1GHz, until 1.7GHz and above, at which load 99% is targeted.
Attempt to avoid oscillations by evaluating the current speed
weighted by current load against each new choice of speed, choosing a
higher speed if the current load requires a higher speed.
Change-Id: Ie3300206047c84eca5a26b0b63ea512e5207550e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Apply above_hispeed_delay whenever increasing speed to a new speed above
hispeed (not just the first step above hispeed).
Change-Id: Ibb7add7db47f2a4306a9458c4e1ebabb60698636
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Add a target_load attribute that specifies how aggressively the governor is
to adjust speed to meet the observed load. New target speed is calculated
as the current actual speed (may be higher than target speed on SMP) times
the CPU load (as a fraction) divided by target load (fraction).
cpufreq_frequency_table_target() call use CPUFREQ_RELATION_L to set
the next higher speed rather than next lower speed.
Change-Id: If432451da82f5fed12e15c9421d7d27792376150
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Tracing adds actual speed since this is expected to be key to the
choice of target speed.
Change-Id: Iec936102d0010c4e9dfa143c38a9fd0d551189c3
Signed-off-by: Todd Poynor <toddpoynor@google.com>
The deferrable timer list isn't checked on all idle exits, such as when
hi-res timers expire or ISRs schedule workers. If the idle loop is
exited and it's past time to run the governor load polling timer,
run it immediately. This ensures we handle load spikes caused by actvity
that does not run the normal timer list.
Rename the field that timestamps the "time_in_idle" value to be more
accurate.
Change-Id: Ied590ecbefc83c9a9ec5eb9e31903557f6fa1614
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Avoid wakeups only to handle the governor timer when the system is otherwise
idle.
For platforms where the power cost of remaining in idle at higher CPU
speed may outweigh the cost of a governor wakeup from idle to lower the speed,
set parameter cpufreq_interactive.governidle=1.
Change-Id: Id6c43eb35caecf9b0574fcdd5b769711bc7e6de6
Signed-off-by: LianWei WANG <a22439@motorola.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Helps avoid waking up other CPUs to react to activity on the local CPU.
Change-Id: Ife272aaa7916894a437705d44521b1a1693fbe8e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
When load is below go_hispeed_load, apply the percentage of CPU load to
a max frequency of hispeed_freq instead of the max speed. This avoids
jumping too quickly to hispeed_freq when it is a relatively low
percentage of max speed. This also allows go_hispeed_load to be set to
a high percentage relative to hispeed_freq (as a percentage of max speed,
again useful when hispeed_freq is a low fraction of max speed), to cap
larger loads at hispeed_freq. For example, a load of 60% will typically
move to 60% of hispeed_freq, not 60% of max speed. This causes the
governor to apply two different speed caps, depending on whether load is
below or above go_hispeed_load.
Also fix the type of hispeed_freq, which was u64, to match other
speed data types (and avoid overhead and allow division).
Change-Id: Ie2d0668be161c074aaad77db2037505431457b3a
Signed-off-by: Todd Poynor <toddpoynor@google.com>
First bump speed up to hispeed_freq whenever the current speed is below
hispeed_freq, instead of only when the current speed is the minimum speed.
The previous code made it too difficult to use hispeed_freq as a common
intermediate speed on systems that frequently run at speeds between
minimum and hispeed_freq.
Change-Id: I04ec30bafabf5741e267ff289209b8c2d846824b
Signed-off-by: Todd Poynor <toddpoynor@google.com>
