sched_ext: Expire prev's slice in dispatch_one() under bypass

The next patch keeps a bypassed rescuee on the local DSQ and needs
dispatch_one() to keep it running too. dispatch_one()'s keep test refuses
any prev whose scheduler is bypassing, so another local task could run ahead
of the rescuee, even an escalated SCX_TASK_PROTECTED one. The test exists
because a cpu-form scheduler can write p->scx.slice directly from BPF, so
the slice can't be trusted under bypass.

The refusal doesn't hold anyway. Since 025b1bd41965 ("sched_ext: Implement
hierarchical bypass mode"), the host's ops.dispatch() runs while prev's
scheduler bypasses, and the post-dispatch test in scx_dispatch_sched() keeps
a queued prev with slice left without checking bypass. A task dispatched in
between waits behind prev until the next bypass tick zeroes the slice, at
most a tick.

Expire the slice before the keep test instead when prev's scheduler is
bypassing, as the bypass tick would. Neither test keeps prev on a zeroed
slice, so prev rotates. A kernel slice survives the expiry and keeps prev.
Only a sub-scheduler's task can hold one, and sub-schedulers are cid-form,
which rejects direct slice writes, so it can be trusted.

Drop the stale pointer to scx_disable_workfn(), update scx_bypass()'s
kerneldoc, and add a TODO noting that the bypass slice handling can be
cleaned up once the cpu form is deprecated and the kfuncs refuse slice
raises from a bypassing scheduler.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>

1 files changed, 24 insertions, 8 deletions

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 27b027833f06..f1c833f1b78d 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3377,17 +3377,32 @@ static enum scx_dsp_verdict dispatch_one(struct rq *rq, struct task_struct *prev
 		update_curr_scx(rq);
 
 		/*
+		 * Under bypass, @prev's slice can't be trusted as a cpu-form
+		 * scheduler can write it directly from BPF. Skipping the keep
+		 * below is not enough as the post-dispatch test in
+		 * scx_dispatch_sched() would keep @prev on the same slice.
+		 *
+		 * Expire it as the bypass tick would, so that neither test
+		 * keeps @prev on it and @prev rotates through the bypass
+		 * enqueue. A kernel slice is left alone and the keep below
+		 * keeps @prev on it, see %SCX_TASK_KERNEL_SLICE.
+		 *
+		 * TODO: Once the cpu form is deprecated and the kfuncs refuse
+		 * slice raises from a bypassing scheduler, slices can be
+		 * trusted under bypass. This expiry, the bypass tick's zeroing
+		 * and the related slice handling can then be cleaned up.
+		 */
+		if (scx_bypassing(scx_task_sched(prev), cpu))
+			expire_task_slice(prev);
+
+		/*
 		 * If @prev is runnable & has slice left, it has priority and
 		 * fetching more just increases latency for the fetched tasks.
 		 * Tell pick_task_scx() to keep running @prev. If the BPF
 		 * scheduler wants to handle this explicitly, it should
 		 * implement ->cpu_release().
-		 *
-		 * See scx_disable_workfn() for the explanation on the bypassing
-		 * test.
 		 */
-		if ((prev->scx.flags & SCX_TASK_QUEUED) && prev->scx.slice &&
-		    !scx_bypassing(scx_task_sched(prev), cpu)) {
+		if ((prev->scx.flags & SCX_TASK_QUEUED) && prev->scx.slice) {
 			verdict = SCX_DSP_PREV;
 			goto has_tasks;
 		}
@@ -6917,9 +6932,10 @@ static void unbypass_renotify_idle(struct rq *rq, struct scx_sched *pos,
  * - ops.sub_ecaps_updated() and ops.sub_child_ecaps_updated() are still called.
  *   The bypassed tasks are on the bypass DSQs, out of their reach.
  *
- * - dispatch_one() does not report %SCX_DSP_PREV on non-zero slice as slice
- *   can't be trusted. Whenever a tick triggers, the running task is rotated to
- *   the tail of the queue.
+ * - dispatch_one() expires prev's slice before its keep test as the slice can't
+ *   be trusted. Each tick does the same and rotates the running task to the
+ *   tail of the queue. A kernel slice is left alone, see
+ *   %SCX_TASK_KERNEL_SLICE.
  *
  * - pick_next_task() suppresses zero slice warning.
  *