diff options
| author | Jakub Kicinski <kuba@kernel.org> | 2024-03-29 15:03:14 -0700 |
|---|---|---|
| committer | Jakub Kicinski <kuba@kernel.org> | 2024-03-29 15:03:14 -0700 |
| commit | 9494dc0b0860e5ca28a5cd5586cca3e171609fee (patch) | |
| tree | b8f7952bdc3936448e9ee4da907745b2f9c1a7f8 /include | |
| parent | 46dc11bee2d5272f200ac419f9e7e238c246b802 (diff) | |
| parent | 1abe267f173eae7ae76cf56232292e9641eb652f (diff) | |
Merge branch 'udp-small-changes-on-receive-path'
Eric Dumazet says:
====================
udp: small changes on receive path
This series is based on an observation I made in UDP receive path.
The sock_def_readable() costs are pretty high, especially when
epoll is used to generate EPOLLIN events.
First patch annotates races on sk->sk_rcvbuf reads.
Second patch replaces an atomic_add_return()
with a less expensive atomic_add()
Third patch avoids calling sock_def_readable() when possible.
Fourth patch adds sk_wake_async_rcu() to get better inlining
and code generation.
====================
Link: https://lore.kernel.org/r/20240328144032.1864988-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Diffstat (limited to 'include')
| -rw-r--r-- | include/net/sock.h | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/include/net/sock.h b/include/net/sock.h index f57bfd8a2ad2..2253eefe2848 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2513,6 +2513,12 @@ static inline void sk_wake_async(const struct sock *sk, int how, int band) } } +static inline void sk_wake_async_rcu(const struct sock *sk, int how, int band) +{ + if (unlikely(sock_flag(sk, SOCK_FASYNC))) + sock_wake_async(rcu_dereference(sk->sk_wq), how, band); +} + /* Since sk_{r,w}mem_alloc sums skb->truesize, even a small frame might * need sizeof(sk_buff) + MTU + padding, unless net driver perform copybreak. * Note: for send buffers, TCP works better if we can build two skbs at |