| Age | Commit message (Collapse) | Author |
|---|
|
verify_success() attaches each cpumask test program to task_newtask,
forks a single child, and expects err to be 0 afterwards. Most programs
return right away for other tasks than the test one.
test_insert_leave and test_insert_remove_release don't, and insert a
cpumask into the single entry of __cpumask_map for every task created
in the system while they are attached. test_insert_leave leaves its
cpumask in the map, so a second task created, for instance by a test
running in parallel, fails to insert one and sets err to 3. Two tasks
created at the same time can do the same to test_insert_remove_release.
Return early for other tasks than the test one in both programs, as the
other programs do.
Fixes: 7b6abcfa15cd ("selftests/bpf: Add selftest suite for cpumask kfuncs")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Link: https://lore.kernel.org/bpf/20261009-selftests-bpf-cpumask-insert-filter-v1-1-3249218594d7@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Extend verifier_global_ptr_args.c with calls that pass bare and
RCU-protected task pointers to a global __arg_trusted parameter. Both
calls must be rejected because neither pointer is referenced or trusted.
Add paired test-module kfuncs that return and accept a nullable trusted
task pointer. Passing the return value directly must succeed, covering
PTR_TRUSTED | PTR_MAYBE_NULL handling in the common argument checker.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20261007-a3-arg-trusted-v4-v4-2-253ef3a1a69a@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_proactive_reclaim() performs one bounded reclaim pass per call and,
unlike a write to memory.reclaim, does not retry until the goal is
reached.
Charge 32 MiB of page cache to a cgroup, ask for all of it in one call,
and check that the result is positive but well below the request: one
pass is capped at MEMCG_CHARGE_BATCH pages, far short of 32 MiB on both
4K and 64K page kernels.
The test deliberately covers only the kfunc itself. A full example of
the intended asynchronous use, where a BPF program watches one cgroup's
workingset refaults and reclaims another one from bpf_wq callbacks, is
maintained out of tree at [1], released under GPLv2.
Add CONFIG_MEMCG to the config fragment, without which mm/bpf_memcontrol.c
is not built at all.
[1] https://github.com/teawater/memcg-async-reclaim
Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
Reviewed-by: Barry Song <baohua@kernel.org>
Link: https://lore.kernel.org/bpf/20261009062637.176005-1-hui.zhu@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
All possible combinations of (BPF_EXIST, BPF_NOEXIST) and
(BPF_F_ALL_CPUS, BPF_F_CPU) flags are covered for all percpu map types.
As BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE does not support BPF_NOEXIST and
also in order to reduce the amount of code duplicates, I added its
new test scenarios to the existing cpu_flag_percpu_cgroup_storage test.
But for other percpu map types, I added new tests to be able to
exercise all flag combinations and the edge cases such as percpu LRU
unnecessary deletion issue.
Signed-off-by: Masoud Aghasi <maghasi@disroot.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://patch.msgid.link/20261008120734.1565972-4-maghasi@disroot.org
|
|
child_func() terminates with exit(), but spawn_child_flag() also starts
it via clone(CLONE_VM), which shares the parent's address space. That
exit() then runs atexit handlers and static destructors against globals
the parent is still using.
LLVM commit `3c3ed59c6a2a ("[llvm] Add format check for
MCSubtargetFeatures")` [1], included in the LLVM 23.1 releases, added
Target::isValidFeatureListFormat() with a function-local static holding
the compiled regex:
static const llvm::Regex pattern("^([+-][^,]+)(,[+-][^,]+)*,?$");
return pattern.match(Features);
The child exit destroys that static while its guard variable still marks
it as initialized, so the regex is never rebuilt in the parent. From that
point on, get_jited_program_text() gets NULL from
LLVMCreateDisasmCPUFeatures(), so every jited program text check in the
tests running after filter_clone_vm fails, e.g. "verifier_jit_convergence",
"verifier_private_stack", "verifier_tailcall_jit" and "verifier_jit_inline".
Use _exit(), which runs neither atexit handlers nor static destructors, as
uprobe_syscall.c already does for its own CLONE_VM child.
[1] https://github.com/llvm/llvm-project/pull/180943
Fixes: d2520bdb1932f ("selftests/bpf: Add uprobe multi pid filter test for clone-ed processes")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261018005348.2195130-5-kafai.wan@linux.dev
|
|
Commit f414d20879ae ("bpf: Save DFS postorder numbers for program
instructions") changed bpf_compute_postorder() to use a real DFS, which
changes the postorder of this test's instructions and thus the insn edge
the arg track join at insn 10 is reported on:
arg JOIN insn 8 -> 10 r1: _ + fp0-8 => fp0-8|fp0+0
arg JOIN insn 8 -> 10 sa0: _ + fp0-8 => fp0-8|fp0+0
Update the r1 and sa0 expectations to the new output.
Fixes: f414d20879ae ("bpf: Save DFS postorder numbers for program instructions")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261018005348.2195130-4-kafai.wan@linux.dev
|
|
Commit 147400e9c27c ("bpf: Compute parameters liveness in
bpf_compute_live_registers()") stopped assuming that a call to a static
subprogram reads r1..r5 and now uses the callee's entry live-in. R1-R5 are
not live at these call insns, so __clean_func_state() clears them from the
caller state and the backtracked parent state prints as:
mark_precise: frame0: parent state regs= stack=: R10=fp0
Update both expectations accordingly.
Fixes: 147400e9c27c ("bpf: Compute parameters liveness in bpf_compute_live_registers()")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261018005348.2195130-3-kafai.wan@linux.dev
|
|
bpf_check_attach_target() rejects a freplace whose target is a static
function, using the subprog "unreliable" flag as a proxy for "static".
That flag is raised whenever a subprogram's BTF prototype disagrees with
the registers at its call site.
`147400e9c27c ("bpf: Compute parameters liveness in
bpf_compute_live_registers()")` made that work for static callees: a
static function with an unused argument no longer matches its BTF
prototype at the call site and is marked unreliable.
Since LLVM commit `29b36a16f111 ("[BPF] Emit ABI-accurate BTF
prototypes for DW_CC_nocall")` in 23.1 [1] that premise is gone:
BTFDebug may now drop arguments removed by the optimizer, so for
__noinline static int subprog(struct __sk_buff *ctx)
{
return bpf_ktime_get_ns();
}
compiled with clang 23 the BTF describes subprog() as
[75] FUNC_PROTO '(anon)' ret_type_id=2 vlen=0
[76] FUNC 'subprog' type_id=75 linkage=static
There is then no argument left to compare, the subprog stays reliable,
and the check in bpf_check_attach_target() is skipped. The freplace is
then rejected by the later linkage check instead, reporting "subprog()
is not a global function", and the selftest "exceptions" fails.
Determine staticness from the BTF linkage instead, and keep the
unreliable check for global functions whose BTF does not match their
call site.
There is no functional change: freplacing a static function was rejected
before and is still rejected with the same -EINVAL. Selftest "exceptions"
is updated to match.
[1] https://github.com/llvm/llvm-project/pull/198426
Fixes: 147400e9c27c ("bpf: Compute parameters liveness in bpf_compute_live_registers()")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261018005348.2195130-2-kafai.wan@linux.dev
|
|
The inlined expansion of bpf_get_func_arg() bounds-checks only the low 32
bits of the index, so a helper prototype that keeps the upper bits of the
index is needed to reach it.
Add check_wide_index() and call it from the fexit and tp_btf tests. It
drives the helper with indices that have upper bits set and with indices
whose low 32 bits are out of range, and it checks that the destination is
left untouched when the helper reports -EINVAL. Without the previous patch
the wide index is shifted as 64-bit and the load uses an address far
outside the context.
Signed-off-by: Woojin Ji <random6.xyz@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://patch.msgid.link/20261018-get-func-arg-v2-2-048848b3eddf@gmail.com
|
|
nolibc does not support position independent executables (PIE).
However a toolchain might generate these by default depending on various
circumstances. Recently clang started to do so for hexagon. See LLVM
commit ab43cb8520c0 ("[Hexagon] Pass -pie to linker when PIE is the
toolchain default (#189723)").
As clang can also be configured to create PIE for other architectures by
default disable it globally. While we are at it also pass the argument
for GCC to be on the safe side.
If PIE is disabled already, clang will generate
-Wunused-command-line-argument warnings. Inhibit those.
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Reviewed-by: Willy Tarreau <w@1wt.eu>
Link: https://patch.msgid.link/20261018-nolibc-unused-arguments-v1-1-6cba71555fe8@weissschuh.net
|
|
An extension cannot be sleepable, and it is verified as a program that
runs under the RCU read lock. But it runs in the context of the function
that it replaces. For a sleepable program, or a function that is called
from a bpf_wq or bpf_task_work callback, that is rcu_read_lock_trace().
The verifier lets such an extension use maps and pointers that rely on
RCU. Its maps are not counted as maps with sleepable users either, so
that what it got from them is freed behind a plain RCU grace period:
WARNING: net/core/sock_map.c:903 at __sock_hash_lookup_elem+0xe8/0xf0
BUG: KASAN: slab-use-after-free in bpf_prog_b3bad285227d855b_ext_storage+0x245/0x297
Write of size 8 at addr ffff888108717a50 by task fr/955
The verifier knows the contexts that a global function is called in.
Refuse to replace one that is called without the RCU read lock, and the
main function of a sleepable program. A function that sleepable code only
calls under bpf_rcu_read_lock() can still be replaced.
The summarization test has replaced functions of sleepable programs by
ones that do not sleep. Expect that to fail.
Fixes: 1e6c62a88215 ("bpf: Introduce sleepable BPF programs")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261017230950.802052-5-alexei.starovoitov@gmail.com
|
|
zone subtest prints FAIL, but did not set 'ret=1', so kselftest
framework reports success.
While at it, also fail when "nft -f" returns an error.
Noticed thanks to a (broken) local patch that should have caused this
selftest (resp. the kselftest framework) to report a failure.
Fixes: 33b8aad21ac1 ("selftests: netfilter: add a vrf+conntrack testcase")
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Fernando Fernandez Mancera <fmancera@suse.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
Add test_hook_drop_scope. It queues 20 UDP packets from an inet output
chain to a queue program that holds all verdicts, deletes a base chain
while they wait, and reads the queue length from
/proc/net/netfilter/nfnetlink_queue right after the deletion. It then
releases the queue program and checks with a counter in the input
chain and the packet total of the queue program that the surviving
packets are delivered.
Deleting a base chain at another hook point (ip6 prerouting) must keep
the 20 queued packets. Deleting a base chain at the queueing hook point
(inet output, another priority) drops them, because the index of a
queued packet into the hook array of its hook point is stale once that
array changed. The second case documents the remaining behaviour so
that a change to it is visible.
To hold the packets without depending on timing, add the -H option to
the nf_queue helper: it blocks SIGUSR1, reads the first packet and
waits in sigwait() until it receives SIGUSR1, then sends the verdicts
as usual. The test sends the signal after the ruleset change, so the
queue length it reads is exact on slow machines too.
Without the previous patch the first case fails: 0 of 20 packets remain
queued after the deletion and none is delivered.
The test and the helper option were written with an LLM coding
assistant from a task description; the author reviewed the result,
and validated the test in virtme-ng on kernels with and without the
previous patch.
Assisted-by: LLM
Signed-off-by: Antonio Ojea <aojea@google.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
Enable the seccomp BPF selftests on Alpha with a fallback seccomp syscall
number and architecture-specific syscall register accessors.
Use the NT_PRSTATUS register layout rather than struct pt_regs. The
register set contains r0-r30 followed by pc and unique. Read and update
the syscall number in r1, and set both r0 and the r19/a3 error flag when
synthesizing syscall results.
At syscall entry, r19 still contains the fourth syscall argument, so
callers setting a synthetic return value must also set the syscall
number to -1 to skip execution.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
Link: https://lore.kernel.org/r/20261010134214.974565-1-linmag7@gmail.com
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
|
|
Add a test that reads the two u64 fields bytes_received and bytes_acked
at offset + 4 with a u32 load, to make sure such a narrower access is
rejected with "invalid tcp_sock access".
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261017143819.209510-2-jiayuan.chen@linux.dev
|
|
Socket storage can't be iterated from user space. Create a
BPF_MAP_TYPE_SK_STORAGE map, attach storage to a socket, pin it, and
check that "bpftool map dump" fails with nothing but an error: a JSON
error object with -j, no elements on stdout in plain mode, and the
reason on stderr.
Socket storage always has BTF, so its plain dump goes through the BTF
writer and never prints an element count. Check that path with a queue
map created without BTF that holds one element: its plain dump must
fail without a "Found" line, which bpftool printed as "Found 0
elements" before, and give the reason on stderr.
Signed-off-by: Maxim Skokov <skokovmaksimevg@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Quentin Monnet <qmo@kernel.org>
Link: https://patch.msgid.link/20261016194825.230433-4-skokovmaksimevg@gmail.com
|
|
Send bulk TCP through GRE tunnels carried over UDP by FOU and by GUE,
each over IPv4 and IPv6, with GRO on at the receiving veth and
segmentation off at the sender so the receiver gets wire-sized packets
to merge. Check that the receiver counts no UDP errors.
Verified under virtme-ng: without the previous patch, all four cases
fail, with the encapsulation socket counting every merged packet in
UdpInErrors or Udp6InErrors. With it, they pass in a few seconds.
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Link: https://patch.msgid.link/20261011-bradfitz-gre-fou-gro-b4-v1-2-df713aad1531@tailscale.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Now that bareudp allows BIG TCP too, run the same checks over bareudp as
for vxlan and geneve.
bareudp only works in external mode, so attach the tunnel metadata to
all packets sent through the device with a tc tunnel_key action on
egress. The device uses ethertype ipv4 with multiproto, so that both
IPv4 and IPv6 traffic can go through it. Its MTU does not account for
the encapsulation, so lower it to leave room for the IPv6 and UDP
headers, otherwise the encapsulated packets would be fragmented.
Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20261011102023.1603595-3-yanhaishuang@cmss.chinamobile.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
The bareudp tests only send pings, which are not GSO packets, so the
segmentation path of bareudp is not covered.
Copy a file over TCP from NS0 to NS3 with socat after the IPv4 and IPv6
ping tests, with UDP tunnel segmentation offload disabled on the
underlay veths, so that the GSO packets encapsulated by bareudp are
segmented in software. Raise the MTU of the underlay veths, so that full
sized packets fit with the encapsulation, as there is no path MTU
discovery through bareudp.
The MPLS overlay gets no stream test: bareudp does not set
mpls_features, so MPLS GSO packets are segmented by mpls_gso before
they reach bareudp, and are dropped if mpls_gso is not available.
The ping and stream tests share the result reporting, so move it into
report_result(). Without socat or ethtool, the stream tests are skipped.
Without commit c57b2e60d4bf ("bareudp: set the inner protocol to the
protocol of the packet"), the IPv4 and IPv6 streams over UDPv6, and the
IPv6 stream over UDPv4 in multiproto mode, time out.
Suggested-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20261011104300.1606351-1-yanhaishuang@cmss.chinamobile.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Manual merge fixup in ns_common_types.h:ns_common to work around a conflict
with the nsfs RCU grace period fix.
* lsm/dev:
lsm: remove redundant NULL check in security_init()
selftests/landlock: Check full ioctl commands in audit records
lsm: Preserve full ioctl commands in audit records
doc: fix LSM ordering description for /sys/kernel/security/lsm
doc: describe CONFIG_LSM and lsm= as the selection mechanism
selftests/bpf: verify mount idmaps reach inode hooks
lsm: expose mount idmaps to inode hooks
rust: lsm: replace `core::mem::zeroed` with `pin_init::zeroed`
cred: prevent slab cache merging for cred_jar
lsm: don't call security_backing_file_free() multiple times
lsm: delete duplicate assignment in lsm_prepare()
lsm: update the BUILD_BUG_ON() in audit_log_lsm_data()
lsm: add LSM_AUDIT_DATA_NS for namespace audit records
lsm: add LSM blob and hooks for namespaces
ns: Free anonymous mount namespaces via ns_common_free()
cred: clarify that task_struct::cred is only for the current task
|
|
Add a test to test_pids.c that checks a fork rejected by pids.max
increments pids.events and generates an IN_MODIFY inotify event on that
file. The test cgroup is the one over the limit, so the notification is
expected both on a default mount and with pids_localevents.
This covers the fix in commit 1765a153d985 ("cgroup/pids: Restore
pids.events notifications in local mode"), where a rejected fork changed
pids.events without waking its pollers when pids_localevents was enabled.
Tested on x86_64 under QEMU without KVM, on kernels built from the
cgroup for-7.4 branch (eb69e955de7b) and from the same tree with that
fix reverted. With the fix, the new test passes on a default cgroup2
mount and on a pids_localevents mount. With the fix reverted, it passes
on a default mount and fails on a pids_localevents mount. No kernel
warnings were logged.
Link: https://lore.kernel.org/r/20261013064812.875787-1-sahaj123.sc@gmail.com
Signed-off-by: Sahaj Chaudhari <sahaj123.sc@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Add two functions to the BPF target program of the int128_tgt_prog test.
callee_split() takes a struct holding an __int128 after five u64s, so
on x86-64 it is passed split between r9 and the stack. fentry and fexit
programs check that the trampoline takes it from there and hands it back
there.
callee_unreliable() takes six u64-sized arguments, the first a plain
pointer, which makes its BTF unreliable while the sixth is still passed
on the stack. The new tgt_prog_unreliable subtest checks that an fentry
program cannot be attached to it.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016183125.1136468-1-yonghong.song@linux.dev
|
|
Attach fentry and fexit programs to two functions of another BPF
program, each taking a struct holding an __int128 at an odd argument
slot:
- callee_odd(): the struct after a single u64, where arm64 would
leave a register empty.
- callee_stack(): the struct after seven u64s, where x86-64 would
leave a stack slot empty and arm64 a register.
BPF code calls them with the arguments packed in the BPF convention, so
that is where the trampoline has to take them from. The fentry programs
take the arguments with BPF_PROG2(), which reads each from its slots in
ctx[], and the fexit programs check the return value, weighted by slot,
which is only right if the function got its arguments back where its
caller put them. Both take a struct by value, which GCC passes by
reference instead, so they are only built with clang, and callee_stack()
only with a clang that supports stack arguments.
Unlike a kernel function taking an __int128, which s390x passes by
reference, nothing in the test depends on how the arch passes a 16-byte
value, so it runs on s390x as well as on x86_64 and arm64. BPF_PROG2()
reads the struct from its two context slots the way BPF code passed it,
so its halves come out right on big-endian s390x too. The s390 JIT does
not take stack arguments, so callee_stack() is only built for x86 and
arm64, and only callee_odd() is traced on s390x.
Describing the functions to the trampoline by the types of their
arguments, rather than as one u64 per slot, the x86-64 trampoline fails
the test with:
test_int128_tgt_prog:FAIL:stack:h.lo unexpected stack:h.lo: actual 9 != expected 8
test_int128_tgt_prog:FAIL:stack:h.hi unexpected stack:h.hi: actual 10 != expected 9
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016183115.1135762-1-yonghong.song@linux.dev
|
|
Add two bpf_testmod functions whose 16-byte aligned argument, a struct
holding an __int128, is passed on the stack after a hole:
- bpf_testmod_test_int128_stack(): seven u64s before the struct, so it
leaves a register empty on arm64 and a stack slot empty on x86-64.
- bpf_testmod_test_int128_backfill(): a 16-byte struct of two longs
moves to the x86-64 stack and gives R9 back to the next argument; the
struct then leaves a stack slot empty on both x86-64 and arm64.
The __int128 is wrapped in a struct because btf_ctx_access() refuses a
program reading an integer argument wider than eight bytes, while it
lets the program read each slot of a struct argument.
The fentry programs read each argument slot from ctx[] and check that
the trampoline packs them in the BPF convention, and the fexit programs
check the return value, weighted by argument slot, which is only right
if the original function got its arguments back where its convention
wants them.
Without the preceding x86 fix the test fails with:
test_int128_args:FAIL:s:h.lo unexpected s:h.lo: actual 2 != expected 8
...
test_int128_args:FAIL:s ret unexpected s ret: actual 285 != expected 385
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016183110.1135597-1-yonghong.song@linux.dev
|
|
This follows up the series "bpf: Support by-value struct and __int128
arguments" [1]. The __int128 kfunc tests it added pass the __int128 at
an even argument slot, after two u64s, or past the argument registers,
where it lands on an even arm64 stack slot. So neither the AAPCS64 rule
that rounds the register number up to an even one for an argument
aligned to 16 bytes, nor the same rule for a stack slot, is exercised.
Each leaves a hole the arm64 JIT has to open by moving the __int128 and
every argument after it up a slot.
Add three kfuncs. Like the earlier ones, each weighs its arguments by
argument slot, so that two eightbytes landing in each other's slots are
caught as well as one landing in a hole:
- bpf_kfunc_call_test_i128_arg_odd(): the __int128 at an odd register
slot.
- bpf_kfunc_call_test_i128_arg_odd_many(): the same, with arguments
after it that arm64 shifts onto its stack, and a second __int128 at
an odd arm64 stack slot.
- bpf_kfunc_call_test_i128_arg_stack_hole(): a struct holding an
__int128 at an odd stack slot on both x86-64 and arm64, after x86-64
gives R9 back to an argument following a 16-byte struct.
x86-64 has no register alignment rule and places the first two like the
BPF convention, so they check that its JIT leaves them where they are.
The last two need more than the five BPF argument registers and are only
built with a compiler that supports stack arguments.
On arm64 the first two kfuncs get BTF only with the pahole patch
"dwarf_loader: Skip the argument register the arm64 ABI leaves as a hole"
[2], in pahole master after v1.32. An older pahole finds the argument
after the __int128 in an unexpected register, x4 rather than x3, and
leaves the function out of BTF, so the tests fail to load there.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20260912195156.980886-1-yonghong.song@linux.dev/ [1]
Link: https://lore.kernel.org/dwarves/20260917164043.1577508-1-yonghong.song@linux.dev/ [2]
Link: https://patch.msgid.link/20261016183054.1134162-1-yonghong.song@linux.dev
|
|
Test the previous patch with a target program made of two call chains:
run() -> call_stack() -> callee_stack(a, b, c, d, e, f, g, h)
run() -> subprog_tail() -> bpf_tail_call()
callee_stack() takes eight u64 arguments, so g and h are passed on the
stack. Because run() can reach a tail call, a trampoline on any function
of the program keeps the tail call counter pointer. callee_stack()
itself does not lead to the tail call, which is why the verifier still
lets it take stack arguments.
Attach fentry and fexit programs to callee_stack(). The fentry program
checks that it sees g and h, and the test checks the value that
callee_stack() returns when the fexit trampoline calls it.
Without the previous patch, the function takes the counter pointer for
its first stack argument:
test_stack_arg_tailcall:FAIL:stack ret unexpected stack ret: actual -359789677662885869 != expected 204
test_stack_arg_tailcall:FAIL:stack fexit ret unexpected stack fexit ret: actual -359789677662885869 != expected 204
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016183049.1133231-1-yonghong.song@linux.dev
|
|
The arena and/or/xor tests never exercised the fetching form of the
instruction. Commit 2897b1e2a2f4 ("selftests/bpf: Fix arena_atomics
failure due to llvm change") deliberately switched them to
__c11_atomic_fetch_*() with memory_order_relaxed, which clang lowers to a
non-fetching locked instruction when the result is unused, because
x86-64 could not JIT the fetching one against an arena.
It can now, so check the returned old value with __sync_fetch_and_*().
Keep a __c11_atomic_fetch_*() of each width per operation with the result
discarded: that spelling is what selects the non-fetching insn, which
would otherwise lose its only coverage, on every architecture rather than
just x86. Compiling the prog with clang 22 before and after confirms the
split: 6 non-fetching and no fetching arena and/or/xor before, 6 and 22
after.
The new globals go in their operation's block, and the or block keeps its
u32s first, so .addr_space.1 stays free of the padding holes that clang
before 18 cannot fill when it is built without address space casts.
Add a fetch_r0 test for the two register assignments the x86 JIT has to
special-case, where the operand and where the arena pointer is R0 and
BPF_REG_AX is substituted for it. Clang picks its own registers and will
not reliably produce either, so spell the instructions out. The values
have bits set above 32 so that a truncating move on that path shows up.
Add both forms to the uaf test as well, since the substitution only
reaches the exception table when such an access faults, and check that
the fault cleared the fetched register and left the address alone. They
fault on the page after the freed one: the freed page is the first of the
arena, whose 32-bit address is 0, which would make the address check
empty. The address is read back afterwards, which gives 0 if the JIT's
exception table entry recovered the fault, but not if the arena had
mapped its zeroed scratch page and let the access succeed. The powerpc64
JIT does not clear the register of a faulting atomic yet, so those checks
are skipped there.
Both spell addr_space_cast textually, which needs LLVM 19, so uaf gains
the __BPF_FEATURE_ADDR_SPACE_CAST guard that every other textual use in
the file already has. It was plain C before and built on older clang.
Drop x86 from the exclusion list of the uaf test, which covers the fault
path. Its page is gone before the first access, so it faults on the load;
the CMPXCHG entry is only reached when the page goes away between the two
accesses, which a test cannot arrange.
Finally cover fetching AND/OR/XOR through a number in
verifier_arena_scalar, on x86-64 and arm64. Both widths are run with the
operand in a register and in R0, which is where the x86-64 JIT keeps the
operand on the stack because BPF_REG_AX holds the address. The 32-bit
operands have bits set above 32 so that a missing zero-extension shows.
The page is also freed first, after storing a nonzero value in it so that
an access which did not fault would show, and the test checks that the
fetched register comes back 0 and R0 is left alone, and that reading the
address back gives 0. R0 is compared with a value derived from the page
address: the verifier would decide a comparison with a constant at load
time and drop it.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://patch.msgid.link/20261016171319.1623447-3-puranjay@kernel.org
|
|
Close gaps in the JIT instruction tests found by earlier Sashiko review,
by comparing the programs with what reaches the JIT, and by a sweep
through RFC9669 to construct tests for corner cases. Non-exhaustive list:
- The staggered jumps loaded their operands once, so the verifier decided
the later guards of ja and gotol and the later jumps of jeq_k and
jlt32_k, and a landing short of the target passed with r0 at zero.
Reload the operands per region and let r0 carry the region number
from the jump to its target.
- The matching 32-bit cmpxchg compared an r0 the mismatching one had zero
extended. Reload r0 with the full value first and check that the source
register keeps its upper half, as test_bpf did.
- The 32-bit atomics wrote and read back only their word, so a JIT touching
all 64 bits passed when the other half was zero. Fill and read back the
whole slot.
- The sub-word loads of the zero offset, offsets and probe_mem tests landed
in a register holding nothing in particular. Set it to all ones first.
- Let probe_mem load every size, sign extending too, and run ld_imm64 into
every register, not only r1.
- Drop the load-acquire probe: a JIT without the instruction fails like for
any other one instead of skipping.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t jit_insns
[...]
#191/1 jit_insns/arena access with zero offset:OK
#191/2 jit_insns/cast to a user address:OK
#191/3 jit_insns/alu_imm:OK
#191/4 jit_insns/alu_reg:OK
#191/5 jit_insns/alu_imm_regs:OK
#191/6 jit_insns/div_const:OK
#191/7 jit_insns/alu_reg_pairs:OK
#191/8 jit_insns/alu_edges:OK
#191/9 jit_insns/shifts:OK
#191/10 jit_insns/shift_counts:OK
#191/11 jit_insns/atomics:OK
#191/12 jit_insns/atomic_reg_pairs:OK
#191/13 jit_insns/xadd_chain:OK
#191/14 jit_insns/ld_imm64:OK
#191/15 jit_insns/jmp_imm:OK
#191/16 jit_insns/jmp_reg:OK
#191/17 jit_insns/jmp_reg_pairs:OK
#191/18 jit_insns/jmp_edges:OK
#191/19 jit_insns/staggered_jumps:OK
#191/20 jit_insns/max_jmp:OK
#191/21 jit_insns/convergence:OK
#191/22 jit_insns/calls:OK
#191/23 jit_insns/helper_calls:OK
#191/24 jit_insns/ldst_zero_off:OK
#191/25 jit_insns/ldst_offsets:OK
#191/26 jit_insns/acq_rel:OK
#191/27 jit_insns/movsx:OK
#191/28 jit_insns/jset_imm:OK
#191/29 jit_insns/cmp_imm:OK
#191/30 jit_insns/alu_consts:OK
#191/31 jit_insns/bswap:OK
#191/32 jit_insns/probe_mem:OK
#191 jit_insns:OK
#192/1 jit_insns_blinded/alu_imm:OK
#192/2 jit_insns_blinded/alu_imm_regs:OK
#192/3 jit_insns_blinded/alu_edges:OK
#192/4 jit_insns_blinded/div_const:OK
#192/5 jit_insns_blinded/shifts:OK
#192/6 jit_insns_blinded/xadd_chain:OK
#192/7 jit_insns_blinded/ld_imm64:OK
#192/8 jit_insns_blinded/jmp_imm:OK
#192/9 jit_insns_blinded/jmp_edges:OK
#192/10 jit_insns_blinded/ldst_zero_off:OK
#192/11 jit_insns_blinded/ldst_offsets:OK
#192/12 jit_insns_blinded/jset_imm:OK
#192/13 jit_insns_blinded/cmp_imm:OK
#192/14 jit_insns_blinded/alu_consts:OK
#192 jit_insns_blinded:OK
Summary: 2/46 PASSED, 0 SKIPPED, 0/0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016101822.632891-3-daniel@iogearbox.net
|
|
The bpf_jit_convergence je <-> jmp test reproduces the cycle fixed by
commit c8831bdbfbab ("bpf, x64: Fix a jit convergence issue") with 125
bytes of x86-64 code between the je to l0 and l0 when the last goto l9 is
two bytes and 128 when it is five, so that the je fits into an imm8 only
while the jmp does not and the two flip each other's size from pass to
pass.
The recent JIT changes shortened that code by five bytes: the three loads
and stores with a zero offset lost their displacement byte with commit
44df4d88aa63 ("bpf, x86: Omit the displacement byte for zero offsets"),
and r2 += 0x1 became an inc and r3 &= 0x1 an and without REX.W with commit
d59f47e031ae ("bpf, x86: Use shorter and faster forms for some ALU
immediates"). At 120 and 123 bytes the je fits either way, nothing flips,
and the test passes without exercising the fix.
Add a five byte mov to the region to get the distance back. The je is
forced into its long form again as the fix intends:
# ./vmtest.sh -- ./test_progs -v -t verifier_jit_convergence
[...]
96: 0f 84 7d 00 00 00 je L14
117: eb 83 jmp L6
119: bf 03 00 00 00 L14: movl $0x3, %edi
[...]
#668/1 verifier_jit_convergence/bpf_jit_convergence je <-> jmp:OK
#668 verifier_jit_convergence:OK
Summary: 1/1 PASSED, 0 SKIPPED, 0/0 FAILED
Reported-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261016101822.632891-2-daniel@iogearbox.net
|
|
Extend test_ksock_wq() added in 'c93cbdb13f99 ("selftests/bpf: Add ksock
test for async callback guard")' to test that the bpf_ksock_connect()
and bpf_ksock_send() kfuncs are now allowed and work when called from
workqueues.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Tested-by: Alan Maguire (Oracle) <almagui@kernel.org>
Link: https://patch.msgid.link/20261016133318.33999-2-mahe.tardy@gmail.com
|
|
configfs_rmdir() drops the last reference to an item while its dentry is
still hashed, so a symlink(2) resolving that target takes a reference on
freed memory. syzbot reported it [1].
Race mkdir/rmdir of a target against symlink/unlink of a link to it,
watching /sys/kernel/warn_count.
Fails until the fix [2] lands.
Link: https://syzkaller.appspot.com/bug?extid=6b16e3d085833cbf3e25 [1]
Link: https://lore.kernel.org/all/20260730093435.195441-1-vasilisalmpanis@gmail.com/ [2]
Link: https://patch.msgid.link/20260826-config_selftest-v1-4-e364b07e91ac@debian.org
Signed-off-by: Breno Leitao <leitao@debian.org>
|
|
samples/configfs had nothing to link from, so symlink(2) went untested
beyond the EPERM a subsystem without ->allow_link() returns.
Cover creating and removing a link, the relative body configfs stores, the
count ->allow_link() and ->drop_link() keep, the busy errors both ends
return, and what get_target() rejects.
Link: https://patch.msgid.link/20260826-config_selftest-v1-3-e364b07e91ac@debian.org
Signed-off-by: Breno Leitao <leitao@debian.org>
|
|
configfs has no selftest coverage. Its userspace ABI is entirely
mkdir/rmdir/read/write on attribute files, and the error each of those
returns is part of the contract.
Add tests driving the subsystems samples/configfs registers, each in its
own mount namespace on a private configfs mount.
Link: https://patch.msgid.link/20260826-config_selftest-v1-1-e364b07e91ac@debian.org
Signed-off-by: Breno Leitao <leitao@debian.org>
|
|
There are spelling mistakes in the descr literal strings. Fix them.
Fixes: 095ddb501b39 ("selftests/bpf: Add sockopt case to verify prog_type")
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261012212334.183116-1-colin.i.king@gmail.com
|
|
missed_kprobe attaches test1 to bpf_fentry_test1 and test2 to
bpf_kfunc_common_test, which test1 calls. test_missed_perf_kprobe()
expects the kprobe of test2 to miss exactly one run, the one test1
triggers, since kprobes don't nest.
The test is not serial, and the kprobe on bpf_fentry_test1 is global.
Any other test that does bpf_prog_test_run() on a fentry/fexit program
calls bpf_fentry_test1() and thereby fires test1, which calls
bpf_kfunc_common_test() from kprobe context. The kprobe of test2 then
counts an extra miss, and the test reads 2 instead of 1.
Expect at least one miss, as missed_kprobe_recursion does since commit
08a7078feacf ("selftests/bpf: Improve missed_kprobe_recursion test
robustness").
Fixes: 01e4ae474e39 ("selftests/bpf: Add test for missed counts of perf event link kprobe")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261009-selftests-bpf-missed-perf-kprobe-v1-1-c9d0f75f6f99@gmail.com
|
|
into HEAD
KVM PPC selftests support for 7.4
Rework KVM selftest's handling of memory regions all so that PPC can force its
page table allocations to be naturally aligned, and then add initial selftests
support for PPC.
|
|
KVM selftests changes and a few KVM x86 fixes for 7.4
- Fix emulation of VMPTRST and VMPTRLD when eVMCS is in-use to match real
Hyper-V's behavior.
- Saturate L2's TSC frequency if stacking L1 and L2 scaling multipliers
exceeds what hardware supports, as attempting to run L2 with an illegal
multiplier will either cause VM-Entry to fail (VMX) or trigger a #GP when
writing MSR_AMD64_TSC_RATIO (SVM). The #GP is thankfully benign in the
grand scheme as even "unsafe" WRMSRs have exception fixup.
- Rework KVM selftest's handling of memory regions so that PPC can force its
page table allocations to be naturally aligned.
- Clean up selftest's VMX instruction helpers, and drop an absurd number of
completely useless return values, e.g. functions that unconditionally
return %true and then have every caller assert success.
- Add selftests support for Zhaoxin CPUs.
- Overhaul the TSC scaling test to add a regression test for the saturation
behavior, and to increase its coverage in general.
- Fix selftests usage various NUMA syscalls to account for an undocumented,
but known since 2007, off-by-one kernel bug. Add helpers to try and avoid
re-introducing the same flaws in the future.
- Automatically run xAPIC IPI migration test (between NUMA nodes) when
possible, as the entire point of the test is verify KVM's handling of page
migration of the APIC backing page.
- Misc cleanups.
|
|
tc_netkit_scrub attaches tc8 to the primary netkit device, sends an ICMP
echo with a mark and a priority through it, and checks the mark and the
priority that tc8 saw against what the scrub mode lets through.
tc8 records the mark and priority of every packet it sees, and the
device also carries what the system sends on its own once the device
is up, like IPv6 neighbor discovery or the mDNS of avahi-daemon. When
such a packet goes out after the echo, it overwrites what tc8 recorded,
and the test finds a mark and a priority of 0. Record only IPv4 ICMP
packets in tc8.
tc_netkit_scrub run 100 times on a bpf-next kernel with KASAN enabled,
x86_64 VM with 32 vCPUs and avahi-daemon running, before and after this
patch:
before after
outcome 2 of 100 failed (mark and priority 0) 0 of 100 failed
Fixes: 716fa7dadf11 ("selftests/bpf: Extend netkit tests to validate skb meta data")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261016-selftests-bpf-tc-netkit-scrub-v2-1-b484a7ce2a63@gmail.com
|
|
KVM x86 misc changes for 7.4
- Mask off the address space ID when checking for illegal usage of
kvm_set_internal_memslot() to prevent false negatives on the sanity check.
- Explicitly zero reserved flags when delivering a SynIC message to avoid
leaking a byte of stack information to the guest.
- Rework TDX's handling of the canonical MMU root level to not rely on
userspace to fill CPUID correctly, as doing so lets userspace trivially
trigger a KVM_BUG_ON().
- Reject KVM_SET_NESTED_STATE if userspace "restores" L1 with EFER.LMA=1 &&
EFER.LME=0, which confuses KVM into trying to configure an 64-bit MMU
without PAE.
- Ignore EFER.LMA (clear it from KVM's cached save state) if EFER.LME=0 when
preparing L2 state, to fix another way to confuse KVM's MMU (and because
real hardware also ignores EFER.LMA).
- Improve KVM's sanity checks for a misconfigured MMU to (hopefully) detect
all of the ways KVM can botch EFER.LMA vs. CR4.PAE.
- Remove the "full" out-param from kvm_x86_ops.flush_tlb_gva() as the param
is only used to avoid what effectively amounts to a single memory write.
- Fix a mostly-benign bug where KVM can write through a guest's read-only
mapping due to not checking the error code when expediting emulated MMIO
page faults.
- Honor the guest's view of the "EFER.LMSLE must be zero" defeature so that
userspace can provide migration compatibility between Rome and Milan+.
- Pend a KVM_REQ_EVENT request when an exception is morphed into a nested
VM-Exit, otherwise KVM can fail to actually deliver the nested VM-Exit,
e.g. when injecting a page fault.
- Add support for PML on SVM.
- Add support for virtualizing Bus Lock Detect on SVM, and fix a number of
DR6 bugs and warts along the way.
- Misc fixes and cleanups.
|
|
KVM guest_memfd and CoCo changes for 7.4
- Take the invalidate lock when unbinding a dying file to fix a race with
kvm_gmem_invalidate_end() that results in KVM leaving the MMU invalidation
count elevated. Not taking the lock in the first place was pure misguided
paranoid.
- When binding a memslot, purge the bindings if storing the new memslot fails,
as Xarray's "store range" API can subtly fail partway through.
- Establish memslot<=>guest_memfd bindings *after* the memslot is ready, as
the memslot becomes reachable the instant the binding is installed.
- Don't grab a page refcount when reloading an SNP VMSA and stop returning a
"struct page" from guest_memfd's PFN lookup altogether, to ensure that KVM
doesn't hold any unnecessary refcounts (the lifetime of the page is ensured
by MMU notifiers and similar hooks). Preventing unnecessary refcounts
greatly simplifies supporting in-place SHARED=>PRIVATE conversions.
- Add support for in-place PRIVATE<=>SHARED conversion, i.e. allow a single
guest_memfd instance to be used for both PRIVATE and SHARED variations of
CoCo guests' memory (though not at the same time). This eliminates the need
to double allocate guest memory for TDX and SNP, and is basically a hard
requirement for pKVM, which relies on being able to preserve memory contents
on conversions.
- Hack-a-fix a deadlock bug in ->error_remove_folio() by returning -EBUSY if
the filemap invalidate lock is contended. Punt on a proper fix for the
moment, as a proper fix likely involves reworking truncate_error_folio().
- Add a pile of selftests for in-place conversions.
- Misc guest_memfd cleanups
Conflicts in arch/arm64/kvm/mmu.c: convert to use s2vi.
|
|
setup_bpffs() mounts a bpffs on /sys/fs/bpf and records in mounted_bpffs
whether it did, so that cleanup_bpffs() unmounts only a bpffs that the
test mounted itself.
cleanup_bpffs() checks the flag the other way around. As mounting on
/sys/fs/bpf succeeds even when a bpffs is already there, the test never
unmounts the bpffs it mounted, and every run leaves another one stacked
on /sys/fs/bpf of the mount namespace that the test_progs worker keeps
running later tests in.
Unmount the bpffs only when setup_bpffs() mounted it.
Fixes: 88886309d2e8 ("selftests/bpf: add a selftest for cgroup hierarchical stats collection")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20261009-selftests-bpf-cgroup-stats-bpffs-v2-2-f93a111da992@gmail.com
|
|
setup_hierarchy() mounts a bpffs on /sys/fs/bpf with setup_bpffs(), and
then sets up the cgroups with setup_cgroups(), which moves the test into
a private mount namespace.
The bpffs thus lands in the mount namespace that test_progs shares with
the tests running in parallel, on top of the bpffs already there, and
hides what they pin. cleanup_bpffs() later runs in the private mount
namespace and can't unmount it, so it stays there for good.
bpftool_maps_access then fails to pin an iterator because
/sys/fs/bpf/test_bpftool_map doesn't exist anymore, and bpftool_metadata
can't find the program it pinned in /sys/fs/bpf/test_metadata.
Set up the cgroups first, so that the bpffs is mounted in the private
mount namespace.
Fixes: 88886309d2e8 ("selftests/bpf: add a selftest for cgroup hierarchical stats collection")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-selftests-bpf-cgroup-stats-bpffs-v2-1-f93a111da992@gmail.com
|
|
Test that a veth device advertises ndo-xmit when GRO is enabled on its
peer while both are down and only the device is brought up.
Commit 7a6102aa6df0 ("veth: Update XDP feature set when bringing up
device") made veth_open() set the flag in this case, and selftests were
requested in its review [1].
The block now ends with GRO enabled while the peer is down, so the
aggregation check that follows covers NAPI enabled by veth_open() again.
Commit c017800af5c7 ("selftests: net: veth: test peer ndo-xmit after GRO
toggle while down") had moved it to veth_set_features() [2].
Link: https://lore.kernel.org/all/155aabf8b873bb8cdcafbd6139c42b08513e5fe6.camel@redhat.com/ [1]
Link: https://lore.kernel.org/all/179165459278.434549.1711113374323842550@kernel.org/ [2]
Signed-off-by: Tianyi Gao <tianyi@cloudflare.com>
Reviewed-by: Narcisa Vasile <narcisav.kernel@gmail.com>
Link: https://patch.msgid.link/20261010192914.83651-1-tianyi@cloudflare.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add a regression test for reusing a bpf_task_work context while its
scheduling irq_work is still BUSY. A test that schedules from a thread
pinned to one CPU never observes that window: irq_work_queue() self-IPIs
the local CPU and the invocation completes before the thread's next
syscall.
Use an fexit program on task_work_add() to delay the scheduling
handler's return with a time- and iteration-bounded busy-wait, after
the first-round callback was published and before the handler's tail
state transition. The target task attempts to reuse the same context
from a later syscall, strictly after its own callback wrapper returned,
so the context is STANDBY at that point. A valid window requires the
fexit program to observe the attempt's acknowledgment before returning,
which proves the attempt ran while the old handler, and thus the BUSY
irq_work, was still in flight. With the map and program references
held and no competing scheduler, the only possible source of -EBUSY for
that attempt is the BUSY check in the acquire path. Check that the
attempt is rejected with -EBUSY, that scheduling succeeds on a bounded
retry once BUSY clears, and that the second callback runs exactly once.
Each attempt uses dedicated workers and a fresh map value. Reset the
shared slots before enabling the workers. Stop and join them at the
end of the attempt, then wait for their TIDs to disappear from PID
lookup before checking the final callback count or starting another
attempt. join alone can return before exit_task_work(); a signal-zero
tgkill probe still finds the worker until after its exit task_work has
run. Bound that wait and fail on unexpected probe errors. Callback
deliveries are also matched against the window's generation.
Missed windows are counted and retried within a bounded attempt budget;
hard failures fail the test. Record the busy-wait elapsed time after
worker cleanup as a diagnostic, without asserting a wall-clock bound,
since host vCPU preemption can stretch wall-clock time without affecting
the gate behavior under test.
Skip when fewer than two CPUs are available or when the loaded kconfig
extern identifies CONFIG_PREEMPT_RT. Check that extern before attaching
programs or starting workers. Load and attach errors fail the test
instead of being inferred from tracefs file paths and reported as
unsupported.
Signed-off-by: Yun Lu <luyun@kylinos.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261012061915.231254-5-luyun_611@163.com
|
|
Add cross-CPU stress coverage for the captured-task cancellation fix.
Run scheduling and the target callback on different CPUs, rotating
through three map-value lifetime patterns for 1500 rounds.
Reuse rounds keep the map value and start the next generation after the
callback records its completion. The next scheduling attempt can race
with the remaining callback cleanup, checking that the context becomes
reusable and subsequent generations make progress.
Late-deletion rounds remove the value after the callback records its
completion, racing deletion with the remaining callback cleanup and the
scheduling handler's final state transition. Early-deletion rounds
remove it once the BPF program announces that it is about to schedule,
racing deletion with context acquisition and callback publication.
These deletion variants stress the cancellation paths fixed by the
captured-task change: on an unfixed kernel, a deletion observed by a
delayed scheduling handler can route cancellation through an already
cleared ctx->task.
Publish the scheduling result together with its generation in one
packed value, so a stale result cannot be mistaken for the current
round's. The BPF program retries transient -EBUSY results without
publishing them; in early-deletion rounds, accept -ENOENT when
deletion removes the value before a retry looks it up. Check callback
completion separately for each generation so that an earlier callback
cannot satisfy a later round.
This is stress coverage for task-work cancellation races; it does not
force the exact faulting interleavings or validate rejection of context
reuse while the scheduling irq_work is still BUSY.
Signed-off-by: Yun Lu <luyun@kylinos.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261012061915.231254-3-luyun_611@163.com
|
|
stdio_hijack_init() does not set subtest_state->stdout_saved when
running with -v in non-parallel mode, since output goes straight to
stdout. In that case subtest_state->log_buf is left NULL, so the
compare_text_mismatch subtest crashes when inspecting it. Skip it when
stdout was not saved.
Fixes: bd71904697c6 ("selftests/bpf: Add compare_text_to_expected() helper")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261012140202.2754552-4-kafai.wan@linux.dev
|
|
stdio_hijack_init() does not set subtest_state->stdout_saved when
running with -v in non-parallel mode, since output goes straight to
stdout. In that case subtest_state->log_buf is left NULL, so the
expected-messages subtests crash when inspecting it. Skip them when
stdout was not saved.
Fixes: 34c513be3dad ("selftests/bpf: __not_msg() tag for test_loader framework")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261012140202.2754552-3-kafai.wan@linux.dev
|
|
The hardcoded expected skip_cnt/error_cnt in test_prog_tests_framework()
mismatch the real count when the binary is run with -t or -n filters,
since some subtests are not selected. Count the executed subtests and
compare against that value instead.
Fixes: 2324257dbd68 ("selftests/bpf: Refactor prog_tests logging and test execution")
Signed-off-by: KaFai Wan <kafai.wan@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261012140202.2754552-2-kafai.wan@linux.dev
|
|
For standard and PCPU (non-LRU) hash maps with small key sizes (less
than or equal to the word size), comparing keys requires only a single
instruction. Storing a cached 32-bit hash value to shortcut full key
comparisons provides no performance advantage for small keys, and
consumes memory for every element.
This memory can be saved by removing hash from struct htab_elem and
placing it directly before struct htab_elem only for the new
htab_elem_hashed type which is used only when keys are larger than the
word size or for LRU maps. This reduces elem_size by 8 bytes for small
keys while keeping key and hash at constant compile-time offsets from
struct htab_elem across all map types and key sizes.
All element variants requiring hashes get an anonymous
htab_elem_hashed embedding, ensuring that the -8 byte hash offset is
guaranteed for all element types by composition.
Elements can be recycled without a RCU grace period. Before this commit,
alloc_htab_elem() overwrote the key before initializing the value/pptr
and wrote the hash last, so during value assignment on a recycled
element, the new key was paired with the old hash preventing lockless
readers from matching either the old key or the new key except when
there was also a hash collision.
When hashes are omitted for small keys, only the key field guards
lookups. Writing the new key before value/pptr initialization would
allow a concurrent lookup of the new key to match an uninitialized value
or dereference a stale or freed pptr. So move the new key assignment to
the end of alloc_htab_elem() where the hash assignment was done. Like
the hash assignment it replaces, it does not enforce value store
ordering before key stores on weakly ordered architectures for lockless
readers that were already traversing l_new via a stale pointer before
hlist_nulls_add_head_rcu().
Before this commit htab_mem_dtor() was invoked with the start of the
allocation, which is no longer always the address of struct htab_elem
now that elem_offset can be nonzero for non-preallocated, non-per-CPU
maps. So the key_size field of struct htab_btf_record has been replaced
with a value_offset computed at map creation and passed through
bpf_ma_set_dtor(). That reduces the destructor to a call to
bpf_obj_free_fields() at a fixed offset, which is exactly what
rhtab_mem_dtor() did, so rhtab_mem_dtor() has been removed and replaced
with the new implementation of htab_mem_dtor() for BPF_MAP_TYPE_RHASH.
Together with the previous patch, this reduces the minimum standard and
preallocated hash map element size from 64 bytes down to 32 bytes, and
non-preallocated per-CPU element size from 64 bytes down to 40 bytes.
Signed-off-by: T.J. Mercier <tjmercier@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261008104108.993791-3-tjmercier@google.com
|
|
The htab_elem struct is used as the per-element type for all BPF hash
map types and includes bpf_lru_node in a union with a ptr_to_pptr
pointer. For standard (non-LRU, non-PCPU) hash maps, the 24 byte union
allocated for every element is entirely unused. For non-preallocated
PCPU maps, ptr_to_pptr only requires 8 bytes, leaving 16 bytes of unused
overhead in the union. For preallocated PCPU maps ptr_to_pptr is unused
since elements are freed to the PCPU freelist.
Eliminate this per-element memory overhead by splitting htab_elem into
dedicated structures for each map type:
- struct htab_elem: Minimal structure for standard hash maps and
preallocated PCPU maps (saves 24 bytes per element).
- struct htab_elem_pcpu: Structure for non-preallocated PCPU maps
containing ptr_to_pptr (saves 16 bytes per element).
- struct htab_elem_lru: Retains struct bpf_lru_node for LRU maps.
Place lru_node and ptr_to_pptr before struct htab_elem in
htab_elem_lru and htab_elem_pcpu respectively, and track the offset of
htab_elem from the start of the element allocation in htab->elem_offset.
This keeps key and value at constant compile-time offsets from struct
htab_elem across all hash map types, avoiding dynamic key offset
calculations on lookups.
Use sizeof(struct htab_elem_lru) for the element size rollover check in
htab_map_alloc_check() since it is now the largest element variant.
Signed-off-by: T.J. Mercier <tjmercier@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://patch.msgid.link/20261008104108.993791-2-tjmercier@google.com
|
