From b9843b0399e807824020ccf7a47d06c19fbfa2d2 Mon Sep 17 00:00:00 2001 From: Tejun Heo Date: Mon, 26 Oct 2026 14:17:43 -1000 Subject: sched_ext: Expire prev's slice in dispatch_one() under bypass The next patch keeps a bypassed rescuee on the local DSQ and needs dispatch_one() to keep it running too. dispatch_one()'s keep test refuses any prev whose scheduler is bypassing, so another local task could run ahead of the rescuee, even an escalated SCX_TASK_PROTECTED one. The test exists because a cpu-form scheduler can write p->scx.slice directly from BPF, so the slice can't be trusted under bypass. The refusal doesn't hold anyway. Since 025b1bd41965 ("sched_ext: Implement hierarchical bypass mode"), the host's ops.dispatch() runs while prev's scheduler bypasses, and the post-dispatch test in scx_dispatch_sched() keeps a queued prev with slice left without checking bypass. A task dispatched in between waits behind prev until the next bypass tick zeroes the slice, at most a tick. Expire the slice before the keep test instead when prev's scheduler is bypassing, as the bypass tick would. Neither test keeps prev on a zeroed slice, so prev rotates. A kernel slice survives the expiry and keeps prev. Only a sub-scheduler's task can hold one, and sub-schedulers are cid-form, which rejects direct slice writes, so it can be trusted. Drop the stale pointer to scx_disable_workfn(), update scx_bypass()'s kerneldoc, and add a TODO noting that the bypass slice handling can be cleaned up once the cpu form is deprecated and the kfuncs refuse slice raises from a bypassing scheduler. Signed-off-by: Tejun Heo Reviewed-by: Andrea Righi --- kernel/sched/ext/ext.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-) (limited to 'kernel') diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index 27b027833f06..f1c833f1b78d 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -3376,18 +3376,33 @@ static enum scx_dsp_verdict dispatch_one(struct rq *rq, struct task_struct *prev if (prev->sched_class == &ext_sched_class) { update_curr_scx(rq); + /* + * Under bypass, @prev's slice can't be trusted as a cpu-form + * scheduler can write it directly from BPF. Skipping the keep + * below is not enough as the post-dispatch test in + * scx_dispatch_sched() would keep @prev on the same slice. + * + * Expire it as the bypass tick would, so that neither test + * keeps @prev on it and @prev rotates through the bypass + * enqueue. A kernel slice is left alone and the keep below + * keeps @prev on it, see %SCX_TASK_KERNEL_SLICE. + * + * TODO: Once the cpu form is deprecated and the kfuncs refuse + * slice raises from a bypassing scheduler, slices can be + * trusted under bypass. This expiry, the bypass tick's zeroing + * and the related slice handling can then be cleaned up. + */ + if (scx_bypassing(scx_task_sched(prev), cpu)) + expire_task_slice(prev); + /* * If @prev is runnable & has slice left, it has priority and * fetching more just increases latency for the fetched tasks. * Tell pick_task_scx() to keep running @prev. If the BPF * scheduler wants to handle this explicitly, it should * implement ->cpu_release(). - * - * See scx_disable_workfn() for the explanation on the bypassing - * test. */ - if ((prev->scx.flags & SCX_TASK_QUEUED) && prev->scx.slice && - !scx_bypassing(scx_task_sched(prev), cpu)) { + if ((prev->scx.flags & SCX_TASK_QUEUED) && prev->scx.slice) { verdict = SCX_DSP_PREV; goto has_tasks; } @@ -6917,9 +6932,10 @@ static void unbypass_renotify_idle(struct rq *rq, struct scx_sched *pos, * - ops.sub_ecaps_updated() and ops.sub_child_ecaps_updated() are still called. * The bypassed tasks are on the bypass DSQs, out of their reach. * - * - dispatch_one() does not report %SCX_DSP_PREV on non-zero slice as slice - * can't be trusted. Whenever a tick triggers, the running task is rotated to - * the tail of the queue. + * - dispatch_one() expires prev's slice before its keep test as the slice can't + * be trusted. Each tick does the same and rotates the running task to the + * tail of the queue. A kernel slice is left alone, see + * %SCX_TASK_KERNEL_SLICE. * * - pick_next_task() suppresses zero slice warning. * -- cgit v1.2.3