diff options
| author | Jens Axboe <axboe@kernel.dk> | 2026-10-19 13:30:51 -0600 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2026-10-20 08:35:27 -0600 |
| commit | ecb6817b769fd7d71af2c9f446188649020b2e53 (patch) | |
| tree | f1ae8f57c64ace4d50682d9f555b8d5dfd0ebd7b /include | |
| parent | 02caa5922184e99da04b37fd3336066625ea8e2e (diff) | |
io_uring: end the drain when the drain request completes
Since requests are drained based on allocated requests, the drain stays
active after the drain request has posted its CQE, until it is freed and
a new request finds the defer list empty. A drain request run by io-wq
is freed only after the worker drops its reference, so a request
submitted after the drain completed can still be deferred behind it,
and get issued much later than the application expects.
A drain request dispatched from the defer list runs alone. Track it,
and end the drain when its CQE is flushed and nothing else is deferred.
Have io-wq complete drain requests through task_work, so that this
happens under ->uring_lock before the CQE is visible. Linked drain
requests keep waiting until the link is freed.
Reported-by: Wei Gao <wegao@suse.com>
Link: https://lore.kernel.org/io-uring/20261018070154.16944-1-wegao@suse.com/
Fixes: 8fb7aee05591 ("io_uring: drain based on allocates reqs")
Cc: stable@vger.kernel.org
Tested-by: Wei Gao <wegao@suse.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'include')
| -rw-r--r-- | include/linux/io_uring_types.h | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index 7c5d99a8f75d..2e5a786f2a2a 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -524,6 +524,8 @@ struct io_ring_ctx { atomic_t poll_wq_rcu_pending; struct list_head defer_list; unsigned nr_drained; + /* drain request dispatched from ->defer_list, until it completes */ + struct io_kiocb *drain_req; /* protected by ->completion_lock */ unsigned nr_req_allocated; |