io_uring/cancel: rearm local work wake up before sleep

io_uring_try_cancel_requests() sets ->cq_wait_nr to 1 so that the next
io_req_local_work_add() wakes the cancelling task. The add that takes the
count to zero is the only one that wakes, later ones see a count <= 0 and
return. io_uring_cancel_generic() however only becomes
TASK_INTERRUPTIBLE much later, so the one wake up can be spent on a
running task. That is fine as long as the entry of that add is still
queued, as the io_local_work_pending() check then skips the sleep. It is
not if the add was delayed after queueing its entry:

CPU0 (submitter in exec or exit)      CPU1 (io_req_local_work_add)
                                      mpscq_push(req A)
io_uring_try_cancel_requests()
  io_run_local_work()  // runs req A
io_uring_try_cancel_requests()
  atomic_set(&ctx->cq_wait_nr, 1)
                                      atomic_read(&ctx->cq_wait_nr) == 1
                                      count taken, wake_up_state()
                                        // task is running, no-op
prepare_to_wait(TASK_INTERRUPTIBLE)
io_local_work_pending() == false
schedule()
                                      mpscq_push(req B)
                                      cq_wait_nr <= 0, return

Only the sleeping task can run req B, so it is stuck in execve() until a
signal happens to wake it. An exiting task doesn't even get that.

Arm the count again once the task state is set and before the work list
is checked, for the rings whose local work this task is the one to run.

Fixes: d46ab2c98aba ("io_uring: switch local task_work to a mpscq")
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Link: https://patch.msgid.link/20261015-b4-io-uring-fixes-v1-8-c1556f5d7bf5@toxicpanda.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

0 files changed, 0 insertions, 0 deletions