summaryrefslogtreecommitdiff
path: root/kernel
diff options
context:
space:
mode:
authorHui Su <sh_def@163.com>2026-09-28 21:47:54 +0900
committerPeter Zijlstra <peterz@infradead.org>2026-10-16 09:49:00 +0200
commit2f93bcb3f9aaf6ceee1aec0a97d4fe386d48e781 (patch)
tree25e6c36f8cc90a0e885b93a8b262995db2017e32 /kernel
parent118a2cf4ced60777a2408001e2b4838fed1cea55 (diff)
sched/fair: Avoid overflow in place_entity()
place_entity() inflates an entity's virtual lag with lag * (load + weight) / load Commit 4823725d9d1d ("sched/fair: Increase weight bits for avg_vruntime") replaced the scaled-down weights used in this calculation with avg_vruntime_weight(), removing the arithmetic headroom provided by scale_load_down(). The multiplication can overflow s64 before the division even when the final quotient is representable. A task's hierarchical effective weight can be as small as 2 after avg_vruntime_weight() scaling under group scheduling, while entity_lag() bounds |vlag| by calc_delta_fair(cfs_rq_max_slice(cfs_rq) + TICK_NSEC, se) For example, with HZ=1000, NICE_0_LOAD=1048576, and the minimum scaled hierarchical weight scale_load(2)=2048, this bound is 891289600. With 114 maximum-weight entities, the runqueue load is 10361604096. The intermediate product is: 891289600 * (10361604096 + 2) = 9235189971864780800 > S64_MAX The signed overflow corrupts the entity's placement. Rewrite lag * (load + weight) / load as lag + lag * weight / load The two expressions are equivalent, but the latter avoids multiplying the virtual lag by the total runqueue weight. entity_lag() bounds vlag against the entity's effective weight, so the remaining lag * weight product stays bounded. rescale_entity() preserves this scale across h_load changes and already relies on the same product when rescaling vlag. Keep the original zero-load warning semantics by multiplying lag by weight after replacing the divisor with one. Fixes: 4823725d9d1d ("sched/fair: Increase weight bits for avg_vruntime") Signed-off-by: Hui Su <sh_def@163.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260928124754.1692077-1-sh_def@163.com
Diffstat (limited to 'kernel')
-rw-r--r--kernel/sched/fair.c16
1 files changed, 12 insertions, 4 deletions
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 57360f5cdde4..0f2364479f9d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6463,12 +6463,20 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
load = cfs_rq->sum_weight;
if (curr && curr->on_rq)
load += avg_vruntime_weight(cfs_rq, curr->h_load.weight);
-
- weight = avg_vruntime_weight(cfs_rq, se->h_load.weight);
- lag *= load + weight;
if (WARN_ON_ONCE(!load))
load = 1;
- lag = div64_long(lag, load);
+
+ weight = avg_vruntime_weight(cfs_rq, se->h_load.weight);
+
+ /*
+ * Rewrite, because the product (W + w_i)*vl'_i might
+ * overflow s64:
+ *
+ * vl_i = (W + w_i)*vl'_i / W
+ * = (W*vl'_i + w_i*vl'_i) / W
+ * = vl'_i + w_i*vl'_i/W
+ */
+ lag += div64_long(lag * weight, load);
/*
* A heavy entity (relative to the tree) will pull the