io_uring/io-wq: don't go to sleep on pending work

io_wq_enqueue() wakes a worker off the free list. If there is none it
creates one, unless ->nr_running says that a worker is running. That
relies on whoever takes ->nr_running to zero to check for pending work,
which io_wq_dec_running() no longer does for a worker without
->cur_work, on the grounds that an idle worker is either woken right
away or someone else has created a worker. Neither has to be the case:

worker A (idle)          worker B (idle)          io_wq_enqueue()
                         woken for work 1
woken (e.g. timeout)     takes work 1
work list is empty
                                                  queues work 2
                                                  B is first on the free
                                                  list: wake_up_process(B)
                         leaves the free list
                         work 1 blocks
                           nr_running 2 -> 1
schedule_timeout()
  nr_running 1 -> 0
  ->cur_work == NULL

Work 2 isn't looked at until work 1 is done or A's idle timeout fires,
five seconds later. ->cur_work is also NULL while a worker blocks on
->uring_lock in io_wq_put_work(), with the same result if it was the
only one running.

Let the idle worker leave ->nr_running by itself before it takes its last
look at the work list. Both that and queueing work are done under
acct->lock, so either the worker sees the work, or the enqueue side sees
that it can't count on this worker. This doesn't bring back the request
for a new worker which the check was meant to avoid, and as idle workers
don't get to io_wq_dec_running() any more, the check can go.

Fixes: 8343cae362e1 ("io_uring/io-wq: ignore non-busy worker going to sleep")
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Link: https://patch.msgid.link/20261015-b4-io-uring-fixes-v1-10-c1556f5d7bf5@toxicpanda.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

0 files changed, 0 insertions, 0 deletions