summaryrefslogtreecommitdiff
path: root/kernel
AgeCommit message (Collapse)Author
2026-10-18bpf: Don't let an extension reach further into the context than its targetAlexei Starovoitov
How large the context of a raw tracepoint, tracepoint or syscall program is depends on what the program is attached to, or is run with. bpf_probe_register(), perf_event_set_bpf_prog() and the test runs compare that with max_ctx_offset of the program. An extension of the program, or of one of its global functions, gets the same context. How far it reaches is noted in its own max_ctx_offset, which nobody looks at. On a tracepoint with one argument it reads the stack behind the arguments, in a test run the memory behind the context, or from NULL if there is none: BUG: KASAN: stack-out-of-bounds in bpf_prog_954b5d2051843629_ts28_ext_far+0x7e/0x113 Read of size 8 at addr ffffc90000def7b8 by task ts28/104 BUG: KASAN: slab-out-of-bounds in bpf_prog_954b5d2051843629_ts28_ext_far+0x7e/0x113 Read of size 8 at addr ffff88810712c3e8 by task ts28/104 Refuse to attach an extension that reaches further than the program which it extends. A program that leaves the context to its extensions has to touch as much of it as they do. Fixes: be8704ff07d2 ("bpf: Introduce dynamic program extensions") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017230950.802052-3-alexei.starovoitov@gmail.com
2026-10-18bpf: Don't take a program with a target program for an iteratorAlexei Starovoitov
bpf_iter_prog_supported() goes by the name of the attach function. Commit 7cc639dc45c5 ("bpf: Don't take an extension for an iterator program") keeps out extensions, for which that is the name of a function of another program. A tracing program with BPF_TRACE_ITER that is loaded with attach_prog_fd gets there as well: unlike for BPF_TRACE_RAW_TP, bpf_check_attach_target() does not mind the target program. If the function is called like that of an iterator target, bpf_iter_bpf_map for example, its id in the BTF of the program is cached as the id of the target, even if the load fails later. Programs of that target which are loaded already cannot be attached any more. And the program itself can be attached as an iterator of that target, with a context that the verifier has taken from the prototype of that function: BUG: KASAN: stack-out-of-bounds in bpf_prog_66d4234e2e50f5b6_ts26_forged+0x12a/0x20b Read of size 8 at addr ffffc90000d9fc48 by task ts26/104 Refuse every program that has a target program. That covers extensions. Fixes: 15d83c4d7cef ("bpf: Allow loading of a bpf_iter program") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017230950.802052-2-alexei.starovoitov@gmail.com
2026-10-17sched_ext: Add scx_bpf_dsq_reenq_task() to re-enqueue single tasks from a ↵Tejun Heo
DSQ iteration scx_bpf_dsq_reenq() re-enqueues a whole DSQ, but a scheduler that loses the cpus serving a DSQ, such as a sub-scheduler after a cap revoke, needs only the stranded tasks placed again. Add scx_bpf_dsq_reenq_task(), which marks a task from a DSQ iteration. When the iteration ends, the DSQ's deferred re-enqueue walk is scheduled once and re-enqueues only the marked tasks. The kfunc locks the task's rq and the DSQ like scx_bpf_dsq_move() and is callable from the same contexts. The mark is cleared when the task leaves the DSQ, so a task consumed before the walk is not re-enqueued. An iteration's DSQ can be destroyed before the iteration ends, so schedule_dsq_reenq() now skips a destroyed DSQ instead of aborting the scheduler. scx_bpf_dsq_reenq() racing scx_bpf_destroy_dsq() takes the same path. Declare the kfunc for the tools and add a compat probe. v2: Note in the kerneldoc that another iteration's re-enqueue may take a marked task early, and that a task on another scheduler returns false (Andrea). Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com>
2026-10-17sched_ext: Factor out the rq lock switch of scx_dsq_move()Tejun Heo
Factor the rq lock switch on entry to scx_dsq_move() and its undo on exit out into kf_switch_rq_lock() and kf_restore_rq_lock() so that other kfuncs operating on a task from a DSQ iteration can share them. No functional change. Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com>
2026-10-17sched_ext: Merge branch 'for-7.3-fixes' into for-7.4Tejun Heo
Pull to receive 680edc7f23ad ("sched_ext: Don't queue balance callbacks from a core-sched pick") and resolve its conflict with the proxy execution work here so that for-next merges clean. Conflicts: - do_pick_task_scx(): 680edc7f23ad samples sched_core_enabled() into a new local next to the prev declaration, which 3d6577ea93c9 ("sched_ext: Split curr|donor references properly") changed to rq->donor. Kept rq->donor and added the local. Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-17sched_ext: Don't queue balance callbacks from a core-sched pickAndrea Righi
Commit ff0545e0c263 ("sched_ext: Don't leave balance callbacks queued across a pick retry") queues the deferred work as a balance callback only when do_pick_task_scx() returns a final pick. However, under core scheduling, a pick that returns a task isn't final either: pick_next_task() goes on to pick the SMT siblings, and their dispatch can release the shared rq lock with the callback still queued, triggering the rq_pin_lock() warning. Moreover, dispatch_core_pick() returns early when another dispatch is in flight on the rq, and do_pick_task_scx() then consumes %SCX_RQ_BAL_CB_PENDING set by that dispatch while it's still running. This can be reproduced running scx_qmap -I on SMT siblings with coresched-tagged workloads that frequently sleep, mixed with untagged CPU-bound tasks. Fix this by always deferring the pending work to the irq_work path from dispatch_core_pick() under core scheduling, for both the local and the sibling picks, and by leaving %SCX_RQ_BAL_CB_PENDING alone in do_pick_task_scx(), so that it's handled by the dispatch that set it. Sample the core scheduling state once at the start of the pick, since core scheduling can get enabled while a non-core dispatch has the rq lock released, and the follow-up work must match the dispatch path that ran. Fixes: 4c95380701f5 ("sched/ext: Fold balance_scx() into pick_task_scx()") Cc: stable@vger.kernel.org # v6.19+ Signed-off-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-17bpf: Free what map_free() unlinks for sleepable readersJosef Bacik
bpf_local_storage_map_free() takes the elements of the map off the lists of their owners, which are alive, and frees them, and the storage of an owner with its last element, behind a plain RCU grace period. Nobody uses the map any more, but the other maps of these owners are in use. A sleepable program calls the get and delete helpers under rcu_read_lock_trace() only. While it walks the list of an owner for its own map, or is about to lock the storage, the element it stands on, or the storage, can be freed under it. Whatever else takes an element off the list of a live owner frees it behind an RCU Tasks Trace grace period, which implies a plain one. Do so in bpf_local_storage_map_free() as well. It waits for these callbacks before it frees the map. This does not help a sleepable program that hands an owner to the helpers which nobody holds, and which may be gone by then. Fixes: dcf456c9a095 ("bpf: Fix usage of trace RCU in local storage.") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-31-alexei.starovoitov@gmail.com
2026-10-17bpf: Keep iterator programs off special map fieldsJosef Bacik
The bpf_map_elem iterator, for all but the per-cpu maps, and the bpf_sk_storage_map iterator hand the live value of an element to the program as a writable buffer. The verifier does not know the map when it checks the program, so nothing keeps the program from writing over a bpf_spin_lock, a bpf_timer, a kptr or another special field, which neither a map update nor a pointer to a map value can touch. With CAP_BPF and CAP_PERFMON, or a BPF token, a program can so make the kernel follow a pointer of its choice: after a write over a bpf_timer the kernel crashes on the written value when the element is removed. Offsets into a buffer are constants. Let the verifier record the range of bytes that a program may write through a read-write buffer, and refuse the program with -EACCES, at attach and at a link update, if a special field of the map lies in that range. A program that only reads, or that writes beside the special fields, works as before. One that writes on both sides of a special field is refused, since the range covers the field. Fixes: d6c4503cc296 ("bpf: Implement bpf iterator for hash maps") Fixes: 5ce6e77c7edf ("bpf: Implement bpf iterator for sock local storage map") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-30-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't take an extension for a writable raw tracepoint programAlexei Starovoitov
bpf_probe_register() compares max_tp_access of the program that is attached with the size of the buffer that the tracepoint lets programs write to. An extension of that program, or of one of its global functions, gets the same context and the same pointer to the buffer. What it accesses is noted in its own max_tp_access, which nobody looks at. So it reads and writes behind the buffer, of a tracepoint that may have none: BUG: KASAN: stack-out-of-bounds in bpf_prog_5843f2b820eb4929_tr26_ext_far+0xf4/0x10c Write of size 4 at addr ffffc90000d8f894 by task wext/103 Refuse such extensions. Fixes: be8704ff07d2 ("bpf: Introduce dynamic program extensions") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-29-alexei.starovoitov@gmail.com
2026-10-17bpf: Assume no bounds for a writable tp with tail callsJosef Bacik
A writable raw tracepoint hands a buffer to the program. The verifier notes how far the program reaches into it, and bpf_probe_register() refuses a program that reaches further than the tracepoint allows. A program that does not touch the buffer passes, and can tail-call one of its type that reads and writes behind it. Take a BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE program with a tail call to reach as far as can be, as for the stack, the packet and the buffers of iterators. It cannot be attached any more. Plain raw tracepoint programs, for which the limit is zero, cannot get at the buffer, and a tail call stays within one type: they are left alone. Fixes: 9df1c28bb752 ("bpf: add writable context for raw tracepoints") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-28-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't take an extension for an iterator programJosef Bacik
An extension inherits the expected_attach_type of the program it extends. If that is BPF_TRACE_ITER, check_attach_btf_id() hands the extension to bpf_iter_prog_supported(), which goes by the name of the attach function. For an extension that is the name of the subprogram it replaces, so the load fails with -EINVAL, unless that subprogram is called like the function of a target, bpf_iter_bpf_map_elem for example. Then the extension gets the context of that target. If the subprogram has as many arguments, it can use the key and the value as buffers, but how far it reaches is noted in the extension, where the test of the iterator link does not look: it reads and writes behind the elements of the map. And the id of the subprogram in the BTF of the program is cached as the id of the target, so that programs of that target which are loaded already cannot be attached. Refuse all extensions here. Fixes: 15d83c4d7cef ("bpf: Allow loading of a bpf_iter program") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-27-alexei.starovoitov@gmail.com
2026-10-17bpf: Assume no bounds for buffers behind a tail callJosef Bacik
The bpf_map_elem, bpf_sk_storage_map and sockmap iterators hand the key and the value of an element to the program as buffers. The verifier notes how far a program reaches into them, and the iterator compares that with the map when the program is attached. A program that does not touch the buffers passes. It can tail-call another program of the same target, which gets the same context, and whose reach nobody compares with the map: BUG: KASAN: slab-out-of-bounds in ___bpf_prog_run+0xa46a/0xae80 The program array can be filled at any time, so there is nothing to look at when the link is made. For the same reason a program with a tail call is taken to use the whole stack and the whole packet. Take it to reach as far as can be into the buffers as well, so that these iterators refuse it with -EACCES. Fixes: d6c4503cc296 ("bpf: Implement bpf iterator for hash maps") Fixes: 5ce6e77c7edf ("bpf: Implement bpf iterator for sock local storage map") Fixes: 0365351524d7 ("net: Allow iterating sockmap and sockhash") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-26-alexei.starovoitov@gmail.com
2026-10-17bpf: Check the maps that sleepable callbacks useJosef Bacik
Sleepable programs can only use the kinds of map that do without an RCU read side critical section. check_map_prog_compatibility() sees to that when a map is bound to the program, and asks whether the program is sleepable. That misses the bpf_wq or bpf_task_work callback of a program that is not, which runs sleepable all the same, and it misses an inner map, which is not bound. The bpf_wq callback of a tc program that looks a key up in a sockhash gets as far as WARNING: net/core/sock_map.c:903 at __sock_hash_lookup_elem+0x18f/0x1e0 Ask the same question for the map that is handed to a helper in sleepable context, unless the RCU read lock is held: by bpf_rcu_read_lock(), or by the helper that runs the callback, as bpf_for_each_map_elem() does for a hash map. Programs that take the lock go on working. Disabled preemption or interrupts are not enough, since the maps ask for rcu_read_lock_held(). The other parts of the program may use such a map as before. A global function that is called with preemption or interrupts disabled is verified for the non-sleepable context, in which nothing is known of all that. Verify it for a context of its own: not sleepable, and without the RCU read lock. It differs in what maps can be used, and in nothing else. What is refused from now on runs into the warning above, or uses what the map frees behind a plain RCU grace period: an entry of a devmap, of a perf event array or of a cgroup array. A bloom filter does without RCU. Add it to the list, so that callbacks which use one go on loading. As a side effect sleepable programs can use bloom filters as well. A pointer to the value of a devmap which a program keeps beyond bpf_rcu_read_unlock() is not dealt with here. Fixes: 638e4b825d52 ("bpf: Allows per-cpu maps and map-in-map in sleepable programs") Fixes: 8e83da9732d9 ("bpf: add bpf_wq_start") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-25-alexei.starovoitov@gmail.com
2026-10-17bpf: Count sleepable callbacks as sleepable map usersJosef Bacik
map->sleepable_refcnt tells bpf_map_fd_put_ptr() whether an inner map that is taken out of its outer map has to wait for an RCU Tasks Trace grace period as well. It counts the sleepable programs that use the outer map. The bpf_wq or bpf_task_work callback of a program that is not sleepable runs under rcu_read_lock_trace() only, and is not counted. Such a callback can look an inner map up and keep it while user space deletes it from the outer map. The inner map is freed behind a plain RCU grace period, under the callback: BUG: KASAN: slab-use-after-free in array_map_update_elem+0x6bc/0x7d0 Count a program with such a callback. The maps are bound before the verifier meets the callback, so raise the count of those that are bound when it does. Fixes: 8e83da9732d9 ("bpf: add bpf_wq_start") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-24-alexei.starovoitov@gmail.com
2026-10-17bpf: Keep a program until its bpf_wq callback has endedJosef Bacik
bpf_wq_work() runs the callback under rcu_read_lock_trace() only, since the callback may sleep. The program it belongs to need not be sleepable, and such a program is freed behind a plain RCU grace period. The reference that the bpf_wq holds is dropped when the map element goes, or when another callback is set, no matter whether the work is running. If that was the last reference the program is freed under the running callback, and the worker goes on in the freed JIT image: Oops: int3: 0000 [#1] SMP NOPTI RIP: 0010:0xffffffffc0600947 All it takes is to close the map and the program while the callback runs. Note in the program that it has a callback that runs sleepable, and wait for an RCU Tasks Trace grace period, which implies a plain one, before such a program is freed. Let bpf_wq_work() enter its read side critical section before it reads callback_fn. bpf_task_work holds a reference to the program until the callback is done. Fixes: 8e83da9732d9 ("bpf: add bpf_wq_start") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-23-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't bind a program to a netdev that goes awayJosef Bacik
bpf_prog_dev_bound_init() looks the device up, takes bpf_devs_lock and registers the device for the program. The reference it holds does not keep the device registered. If bpf_dev_bound_netdev_unregister() has run by the time the lock is taken, nobody takes the program off the device any more: the bpf_offload_netdev and offload->netdev point to a net_device that is freed as soon as the reference is dropped. bpf_dev_bound_resolve_kfunc() dereferences it while the program is verified, and a new device at the same address finds the old entry. unregister_netdevice_many_notify() leaves NETREG_REGISTERED before it calls bpf_dev_bound_netdev_unregister(), which takes bpf_devs_lock. Look at reg_state under that lock, and treat a device that goes like one that is not there. Fixes: 2b3486bc2d23 ("bpf: Introduce device-bound XDP programs") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-22-alexei.starovoitov@gmail.com
2026-10-17bpf: Fix stale offload->prog pointer after reallocJosef Bacik
bpf_prog_realloc() moves a program that the rewrites of the verifier make outgrow its pages. It updates aux->prog, but not the offload->prog of a device-bound program. When the device is unregistered, __bpf_offload_dev_netdev_unregister() hands the stale pointer to __bpf_prog_offload_destroy(), which faults on the freed program: BUG: unable to handle page fault for address: ffffc90000071050 RIP: 0010:__bpf_prog_offload_destroy+0x3f/0x310 It takes the right to load a device-bound XDP program, CAP_BPF and CAP_NET_ADMIN or a BPF token, and a device that can be removed. Commit a1aa9ef47c29 ("bpf: Fix stale offload->prog pointer after constant blinding") repaired the other place that replaces a program. Do the same here. Nothing keeps the device while a program is loaded, so it can also be unregistered at that very moment. That frees the offload, and it may have read offload->prog already when the old copy of the program is freed. In both places update the pointer under bpf_devs_lock, before the old copy goes. Fixes: ab3f0063c48c ("bpf: offload: add infrastructure for loading programs for a specific netdev") Fixes: 2b3486bc2d23 ("bpf: Introduce device-bound XDP programs") Fixes: a1aa9ef47c29 ("bpf: Fix stale offload->prog pointer after constant blinding") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-21-alexei.starovoitov@gmail.com
2026-10-17bpf: Check the new program of an iterator link updateJosef Bacik
The bpf_map_elem, bpf_sk_storage_map and sockmap iterators hand the key and the value of an element to the program as buffers. The verifier only records how far a program reaches into them, and ->attach_target() compares that with the key and value size of the map. bpf_iter_link_replace() does not. BPF_LINK_UPDATE installs a program that an attach refuses with -EACCES, and the program then reads, and for a value writes, behind every element that the iterator visits. It takes CAP_BPF and CAP_PERFMON, or a BPF token. Split the test out of the three ->attach_target() and ask it, as ->check_prog(), for the new program of an update as well. Targets without such a test take every program of their kind, as before. Fixes: d6c4503cc296 ("bpf: Implement bpf iterator for hash maps") Fixes: 5ce6e77c7edf ("bpf: Implement bpf iterator for sock local storage map") Fixes: 0365351524d7 ("net: Allow iterating sockmap and sockhash") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-20-alexei.starovoitov@gmail.com
2026-10-17bpf: Bound the nesting that the BTF show code followsJosef Bacik
btf_struct_show() and btf_array_show() recurse for every nested struct and array. MAX_RESOLVE_DEPTH bounds one resolve walk when BTF is loaded, not the nesting: a member whose type is resolved already is not walked again, so BTF that defines its types from the inside out can nest them as deep as it has types. Whoever may load BTF, with CAP_BPF or a BPF token, can give such a type to the value of a map. Reading the file of that map in bpffs then overruns the kernel stack: with a few hundred levels the stack guard page is hit and the kernel panics. Stop at 32 levels. That is MAX_RESOLVE_DEPTH, and __btf_show_indent() has as many columns. Nothing is shown for a member that lies deeper, and the status becomes -E2BIG, which bpf_snprintf_btf() and bpf_seq_printf_btf() return. The types of an x86_64 defconfig kernel nest 14 deep. Fixes: b00b8daec828 ("bpf: btf: Add pretty print capability for data with BTF type info") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-19-alexei.starovoitov@gmail.com
2026-10-17bpf: Link the first element after its storage is publishedAlexei Starovoitov
bpf_local_storage_alloc() links the first element of a new storage to the bucket of its map, and then publishes the storage in the owner with cmpxchg(). If another cpu has been faster, the element is taken out of the bucket again and is freed together with the storage. In between the iterator over a sk_storage map can find the element. It stays linked to a storage that points to the socket, but the socket knows nothing of that storage: bpf_sk_storage_free() neither unlinks the element nor waits for anybody who uses the storage. So nothing tells the iterator that the socket has gone away. Three cpus that create the first storage of a new AF_UNIX socket at the same time, and a fourth that reads the iterator, are enough: in 20 seconds its program was shown some 4000 elements of storages that had lost. If the socket is closed in time, the program is handed a freed socket. Link the element once the storage is published. Both happens under the lock of the bucket, which holds off whoever finds the element through the owner and wants to unlink it. Fixes: c69d2ddb2072 ("bpf: Using rcu_read_lock for bpf_sk_storage_map iterator") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-17-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't saturate owner_refcnt on the last unpinJosef Bacik
bpf_local_storage_map_free() pins the owner with refcount_inc_not_zero() while it uncharges an element. The pin counts when bpf_local_storage_destroy() could not get local_storage->lock: destroy() then goes on, drops the owner's reference and spins until the count is zero. So the pin is the last reference whenever it is needed, and refcount_dec() takes that for a bug: it warns "decrement hit 0; leaking memory" and saturates the count. Unless destroy() happens to read the count between the two stores it spins for ever, under rcu_read_lock(). Drop the pin with refcount_dec_and_test(). Fixes: 5d800f87d0a5 ("bpf: Support lockless unlink when freeing map or local storage") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-11-alexei.starovoitov@gmail.com
2026-10-17bpf: Read the probe of a perf event under event_mutexJosef Bacik
For a perf event on a kprobe or uprobe event which was made in tracefs, bpf_get_kprobe_info() and bpf_get_uprobe_info() look the probe up by name in the list of dynamic events, and hand a pointer to its symbol or file name back to the caller. None of the callers holds event_mutex. The perf event keeps its trace event alive, nothing else. Other dynamic events are removed from the list and freed during the walk. And a probe which shares its event with other probes can be deleted while the event is in use, so even the probe which is found can be freed, together with the string, before the caller has measured and copied it. The result is a use-after-free in the walk, in strlen() and in the copy to user space, or a NULL pointer dereference. The readers are BPF_OBJ_GET_INFO_BY_FD and fdinfo of a perf link, for whoever holds a descriptor of the link, and BPF_TASK_FD_QUERY. The other side is anybody who may write to dynamic_events, kprobe_events or uprobe_events. Take event_mutex in bpf_get_perf_event_info() and give the caller a copy of the name, which it has to free. Fixes: 41bdc4b40ed6 ("bpf: introduce bpf subcommand BPF_TASK_FD_QUERY") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-10-alexei.starovoitov@gmail.com
2026-10-17bpf: Look at all 64 bits of a BPF_JSET mask on 32-bitJosef Bacik
In the branch of a BPF_JSET which is taken, the verifier can learn something about the register only if the mask has a single bit: then this bit is set. regs_refine_cond_op() tests this with is_power_of_2(val) and then ORs val into var_off. val is a u64, but is_power_of_2() takes an unsigned long. On a 32-bit kernel it only sees the lower half of the mask. For if r1 & 0xffffffff80000000 goto l which is a BPF_JSET | BPF_K with an immediate of 0x80000000, the lower half is a power of two. So the verifier takes all 33 bits for set at l, while the branch is taken if any of them is. A mask of 0x100000001 from a register does the same for bit 0. The log of an i386 kernel, for r6 = bpf_get_prandom_u32() << 31: 3: (45) if r6 & 0x80000000 goto pc+2 from 3 to 6: R6=0xffffffff80000000 R10=fp0 6: (77) r6 >>= 63 ; R6=1 When the program runs, r6 >> 63 is 0. >From then on the verifier works with known bits which the register does not have at run time. It accepts, for example, an offset into a map value which it derives from these bits and believes to be constant, and the program can access memory outside of the map value. Count the bits of the whole value. Only 32-bit kernels are affected. On a 64-bit kernel unsigned long holds the whole mask, and this patch changes nothing. Fixes: 960ea056561a ("bpf: verifier: teach the verifier to reason about the BPF_JSET instruction") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-8-alexei.starovoitov@gmail.com
2026-10-17bpf: Forbid sleeping in bpf_find_vma() callbacksJosef Bacik
A sleepable program can call bpf_find_vma() and use a sleepable helper such as bpf_copy_from_user() or bpf_copy_from_user_task() in the callback. The verifier accepts this, but bpf_find_vma() runs the callback with the mmap_lock of the mm held for read. If the user copy faults on that mm, the fault handler has to take mmap_lock again. Lockdep reports recursive locking, and if a writer queued up in between, the fault blocks behind the writer, which waits for the read lock the helper already holds: sleepable prog other thread of the same mm bpf_find_vma(current, ...) mmap_read_trylock(mm) mmap_write_lock(mm) <- waits callback: copy_from_user() page fault mmap_read_lock(mm) <- queued behind the writer This is the same problem as with sleepable task_vma iterator programs. The verifier checks synchronous callbacks in the caller's context, so in_sleepable_context() is true in the callback of a sleepable program. Verify the callback like a hash map for_each callback, as running in an RCU read-side critical section where sleeping is not allowed, and actually hold rcu_read_lock() around it, so that global functions the callback calls, which are then verified for a non-sleepable context, see the RCU protection they are verified with. Non-sleepable programs already run the callback under RCU, nothing changes for them. A sleepable program whose bpf_find_vma() callback calls any helper, kfunc or global function that may sleep is now rejected by the verifier, including ones that would be harmless under mmap_lock as they don't touch user memory. Fixes: 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-7-alexei.starovoitov@gmail.com
2026-10-17bpf: Reject sleepable programs for task_vma iteratorJosef Bacik
A sleepable "iter.s/task_vma" program can be attached, but its ->show() callback runs with the mmap_lock of the iterated mm held for read. As soon as the program runs, bpf_iter_run_prog() calls might_fault(), and any bpf_copy_from_user() or bpf_copy_from_user_task() it does may really fault or take mmap_lock again. With lockdep this is reported as recursive locking of mmap_lock. Without lockdep, a fault on the reader's own mm, or access_process_vm() on the iterated mm, has to take mmap_lock for read again. If a writer queued up in the meantime that blocks behind the writer, which in turn waits for the read lock the iterator already holds, and the read() hangs: reader (iter.s/task_vma) other thread of the same mm task_vma_seq_get_next() mmap_read_lock(mm) mmap_write_lock(mm) <- waits bpf prog: copy_from_user() page fault mmap_read_lock(mm) <- queued behind the writer The reader waits killable, so the hang lasts until it gets a fatal signal. bpf_iter_link_attach() allows sleepable programs for the targets that set BPF_ITER_RESCHED, and task_vma sets it because cond_resched() between VMAs is fine with mmap_lock held. Rescheduling between objects and running a sleepable program inside ->show() are different properties, though, and task_vma only has the first one. The open-coded task_vma iterator does not have this problem since it hands out a snapshot of the VMA and holds no lock when it returns to the program. The vma handed to the program is only stable under mmap_lock, so the lock can't be dropped around ->show(). Add a feature flag BPF_ITER_NOSLEEP for targets which can't run sleepable programs although they can reschedule, and set it for task_vma. The check at attach time is not enough. BPF_LINK_UPDATE replaces the program of a link, and bpf_iter_link_replace() only compares the type, the attach type and the BTF id of the two programs, which are the same for a sleepable and a non-sleepable one. Do the check there as well. That also closes this way around the existing rule: so far a sleepable program could be put on a link of a target that doesn't set BPF_ITER_RESCHED, like the hash map element iterator, which runs it under rcu_read_lock(). BPF_LINK_CREATE of a sleepable program for the task_vma target, and BPF_LINK_UPDATE to one, now fail with -EINVAL, also for programs that don't touch user memory and have been working since sleepable iterator programs exist. The programs can still be loaded. They have to be loaded as non-sleepable programs, or use the open-coded iterator. Fixes: b77fb25dcb34 ("bpf: Add support for bpf iterator programs to use sleepable helpers") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-6-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't hand out a linked element as extra elementJosef Bacik
When htab_map_update_elem() replaces an element of a preallocated map, alloc_htab_elem() takes the spare element of this cpu and puts the old element into the per-cpu slot right away, long before the old element is unlinked from its bucket. The bucket lock disables interrupts, but it does not keep out an NMI, or a tracing program on the return of alloc_htab_elem(). bpf_prog_active only helps if both programs are kprobe, tracepoint or perf_event programs. A program that updates an existing key in another bucket of the same map gets to use the slot: htab_map_update_elem(k1) tc, xdp, fentry, raw_tp, ... alloc_htab_elem() l_new = *pl_new *pl_new = l_old still linked in bucket 1 <NMI> htab_map_update_elem(k2) perf_event program l_new = *pl_new the l_old of the outer update hlist_nulls_add_head_rcu(l_new, bucket 2) hlist_nulls_del_rcu(k2's old element) </NMI> hlist_nulls_add_head_rcu(l_new, bucket 1) hlist_nulls_del_rcu(l_old) If l_old was the first element of bucket 1, the outer add points its pprev at l_new, and the delete takes it out of bucket 1 only. l_new->next now leads into bucket 2, where l_old stays with a poisoned pprev, on which a delete of k2 crashes. If it was not the first, the delete takes it out of bucket 2: k2 is gone from the map, and the chain of bucket 1 still contains l_old, whose next pointer leads into bucket 2. Either way a walk of bucket 1 ends on the nulls value of bucket 2. lookup_nulls_elem_raw() restarts whenever that happens, so every lookup in bucket 1 which does not find its key spins forever: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: __htab_map_lookup_elem+0x354/0x4d0 BUG: KASAN: wild-memory-access in htab_map_delete_elem+0x30c/0x670 Write of size 8 at addr dead000000000122 by task extra/102 An NMI between the load and the store of the slot makes both updates use the same element, with much the same result. Empty the slot when the spare element is taken, in a way that is atomic on this cpu, and put the old element in only after it has been unlinked and its timers and the like have been cancelled. A nested update that finds the slot empty, which is the case that corrupted the map, fails with -EBUSY. Fixes: 8c290e60fa2a ("bpf: fix hashmap extra_elems logic") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-5-alexei.starovoitov@gmail.com
2026-10-17bpf: Don't leak the inode when token creation failsJosef Bacik
bpf_token_create() allocates a bpffs inode and hands it to alloc_file_pseudo(). The inode reference only moves to the new file when that call succeeds: alloc_file_pseudo() leaves the caller's reference alone on failure, and FD_PREPARE() does not even evaluate the file expression when it cannot get a descriptor. Before the conversion to FD_PREPARE() the error path did iput(inode). The conversion dropped it, so each BPF_TOKEN_CREATE that fails with -EMFILE, -ENFILE or -ENOMEM at this point leaks an inode. A task that is allowed to create tokens can hit this at will by lowering RLIMIT_NOFILE, and the leaked inodes make umount of that bpffs instance complain about busy inodes. Put the inode again if no file was created. Fixes: 981bec8f698b ("bpf: convert bpf_token_create() to FD_PREPARE()") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-4-alexei.starovoitov@gmail.com
2026-10-17bpf: Forbid sleeping in hash map for_each callbacksJosef Bacik
A sleepable program (e.g. SEC("syscall"), lsm.s, fentry.s) can pass a callback to bpf_for_each_map_elem() on a hash map and call a sleepable helper or kfunc such as bpf_copy_from_user() from it. The verifier accepts this, but at run time the callback is invoked from inside an RCU read-side critical section: bpf_for_each_hash_elem() rcu_read_lock(); callback_fn(map, key, val, ctx, 0); bpf_copy_from_user() <- may fault and sleep rcu_read_unlock(); bpf_each_rhash_elem() does the same for BPF_MAP_TYPE_RHASH. Sleeping there triggers the "Voluntary context switch within RCU read-side critical section!" warning, delays grace periods, and with CONFIG_PREEMPT_RCU=n silently ends the read-side critical section that the bucket walk relies on. Synchronous callbacks are verified with the caller's state, so kernel/bpf/verifier.c:in_sleepable_context() only looks at the program's sleepability and the locks the program took itself, and does not know that the helper holds rcu_read_lock() around the callback. With a SEC("syscall") program whose callback reads a page that has to be read from disk first: BUG: sleeping function called from invalid context at include/linux/uaccess.h:169 RCU nest depth: 1, expected: 0 __might_fault+0x29/0x80 _copy_from_user+0x26/0x90 bpf_copy_from_user+0x23/0x60 bpf_prog_17d44fa39eebd6cc_cb+0x35/0x46 bpf_for_each_hash_elem+0xdb/0x280 Fix by marking the callback frame of hash map iteration as running under RCU and making in_sleepable_context() false while such a frame is on the call stack, so sleepable helpers, kfuncs and global functions are rejected there as inside bpf_rcu_read_lock(). The rules for trusting RCU protected pointers are left alone, which is conservative and keeps callbacks that verify today and do not sleep working. The flag is compared in func_states_equal() so that pruning never mixes the two contexts. Array map callbacks run without any lock and keep the caller's context. Fixes: 314ee05e2fc6 ("bpf: Add hashtab support for bpf_for_each_map_elem() helper") Fixes: 818e00848227 ("bpf: Implement iteration ops for resizable hashtab") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-3-alexei.starovoitov@gmail.com
2026-10-17bpf: Use the frame mem type in devmap egress XDPJosef Bacik
A devmap egress program that removes a whole fragment of a redirected xdp_frame with bpf_xdp_adjust_tail() or bpf_xdp_pull_data() frees it with page_frag_free() whatever the frame's memory type is. For a MEM_TYPE_PAGE_POOL frame from a page_pool based driver this drops the page reference owned by the pool while the pool still accounts the fragment ("page_pool leak" bad page, leaked pp reference). kernel/bpf/devmap.c:dev_map_bpf_prog_run() points xdp.rxq at an on-stack struct xdp_rxq_info rxq = { .dev = rx_dev }; whose mem.type is 0, i.e. MEM_TYPE_PAGE_SHARED, and net/core/filter.c:bpf_xdp_shrink_data() releases frags according to rxq->mem.type. cpumap, which runs programs on xdp_frames the same way, already copies xdpf->mem_type into its private rxq for each frame. With 9000 byte frames from virtio_net and an egress program that cuts the fragments off: BUG: Bad page state in process swapper/3 pfn:10660b page dumped because: page_pool leak bad_page+0xbd/0x190 __free_frozen_pages+0x5c7/0x900 bpf_xdp_adjust_tail+0x3be/0x480 bpf_prog_dad43f894a3faff7_egress+0x15c/0x1cd dev_map_bpf_prog_run+0x288/0x610 The pool waits for these pages for ever when its device is gone. That is the harmless case. A page of the pool can hold more than one fragment while its refcount is 1: the first page_frag_free() frees it under the fragment which is still in use, and the second one runs into page dumped because: VM_BUG_ON_PAGE(page_ref_count(page) == 0) kernel BUG at include/linux/mm.h:1846! veth is not affected. It takes references of its own on the pages and redirects the frame as MEM_TYPE_PAGE_SHARED. Fix by setting rxq.mem.type from each frame before running the devmap program, as cpu_map_bpf_prog_run_xdp() does. Fixes: ca9984c5f0ab ("bpf: devmap: provide rxq after redirect") Signed-off-by: Josef Bacik <josef@toxicpanda.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261017163745.3341787-2-alexei.starovoitov@gmail.com
2026-10-17sched_ext: Don't leave balance callbacks queued across a pick retryAndrea Righi
The dispatch path can request deferred work to be run from a balance callback and pick_task_scx() queues rq->scx.deferred_bal_cb right after dispatching. However, a balance callback must be consumed before the rq lock is released and pick_task_scx() is not necessarily the end of the pick: - it returns RETRY_TASK when a higher priority class task has been enqueued while the dispatch had the rq lock released; - the dl_server pick can return NULL, in which case the lower classes are picked next. In both cases the callback stays queued while the core keeps picking, and the rq lock can be released again, either by prev_balance() or by another dispatch. For example, with an RT task as prev: __schedule() __pick_next_task() pick_task_scx() dispatch_pick() maybe_queue_balance_callback() <- deferred_bal_cb queued return RETRY_TASK prev_balance() balance_rt() pull_rt_task() double_lock_balance() <- rq lock released Any CPU acquiring the rq lock in that window finds a pending balance callback and triggers the following warning in rq_pin_lock(): WARNING: kernel/sched/sched.h:1919 at rq_lock+0x7f/0xe0, CPU#10: stress-ng-sleep/533 CPU: 10 UID: 0 PID: 533 Comm: stress-ng-sleep Not tainted 7.2.0-rc6-virtme #61 PREEMPT(full) Sched_ext: qmap (enabled+all) Call Trace: <IRQ> try_to_wake_up+0x2c1/0x5c0 hrtimer_wakeup+0x22/0x30 __hrtimer_run_queues+0x1d5/0x350 hrtimer_interrupt+0x106/0x1e0 __sysvec_apic_timer_interrupt+0x6f/0x210 sysvec_apic_timer_interrupt+0x71/0x90 </IRQ> This can be reproduced running scx_qmap -I together with a SCHED_FIFO workload that frequently sleeps, e.g.: $ vng -v --cpus 16 --memory 4G -- \ 'tools/sched_ext/build/bin/scx_qmap -I >/dev/null & \ stress-ng --sleep 4 --sched fifo --sched-prio 1 -t 30 -q' Fix this by queueing the balance callbacks only when the pick is final. Otherwise, defer the pending work to the irq_work path, which doesn't depend on the current rq lock section, and leave the kick sync wait pending for the next pick. Fixes: 4c95380701f5 ("sched/ext: Fold balance_scx() into pick_task_scx()") Cc: stable@vger.kernel.org # v6.19+ Signed-off-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-17sched_ext: Merge branch 'for-7.3-fixes' into for-7.4Tejun Heo
Pull to receive 30fb2b1e4d41 ("sched_ext: Fix the running task keeping its protection into bypass") and 5809f749fbdb ("sched_ext: Fix a bypassed task stalling on a cpu its scheduler has no caps on") and resolve their conflicts with the proxy execution work here. Conflicts: - scx_bypass(): 30fb2b1e4d41 drops the protection strip in the runnable_list walk, which never sees the running task, and strips rq->curr ahead of the walk instead. 3d6577ea93c9 ("sched_ext: Split curr|donor references properly") moved the walk's strip to task_current_donor(), as protection belongs to the scheduling context. set_next_task_scx() takes the donor off the runnable_list, so that strip is dead too. Dropped it, and the new strip takes rq->donor. - scx_bypass(), merged without a marker: 5809f749fbdb evicts the running task when bypass lifts if it can't stay on the cpu, on rq->curr. 3d6577ea93c9 moved the cap revoke eviction of the running task in reenq_local() to rq->donor. The lift eviction takes rq->donor too, and both comments in the block say the current donor. Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-17sched_ext: Fix a bypassed task stalling on a cpu its scheduler has no caps onTejun Heo
A bypassing sub-scheduler's tasks are run by its nearest ancestor that isn't bypassing. The ancestor puts them on the local DSQ under its own caps. Whether such a task may stay on the cpu and whether a cap revoke reenqueues it are still checked against the caps of the task's own scheduler. The bypass load balancer moves tasks between cpus without looking at caps, so a bypassed task can run on a cpu where its own scheduler holds none. Once its slice runs out, it can't stay. It goes to the ancestor's bypass DSQ without a kick, and the cpu can go idle with it queued until something else reschedules the cpu. Make both checks against the caps of the scheduler running the task. When bypass lifts, the running task keeps its slice without a cap check, so evict it then if it can't stay on the cpu. Fixes: 75a8c8202c91 ("sched_ext: Add reject DSQ for cap-rejected dispatches") Fixes: 46a85ae6fe5b ("sched_ext: Tie cpu occupancy to SCX_CAP_BASE through the task slice") Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com>
2026-10-17sched_ext: Fix the running task keeping its protection into bypassTejun Heo
Protection ends when the scheduler enters bypass, so that bypass scheduling can cut every task's slice. scx_bypass() clears it for the queued tasks by cycling them, and strips the running task's in the same walk over rq->scx.runnable_list. The running task is never on that list. set_next_task_scx() takes it off and put_prev_task_scx() puts it back, so the strip never runs. A protected running task then keeps its slice into bypass, where the tick's attempts to cut it are refused. Strip the running task's protection outside the walk. Fixes: 9cfc6ab34a31 ("sched_ext: Add SCX_TASK_PROTECTED") Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com>
2026-10-17bpf: Don't echo user-supplied map info fields back to userspaceDaniel Borkmann
Commit ea2e6467ac36 ("bpf: Return hashes of maps in BPF_OBJ_GET_INFO_BY_FD") started copying struct bpf_map_info in from userspace in order to read the new hash and hash_size input fields. Before that, the kernel ignored the contents of the user buffer and returned a zeroed structure with only the output fields filled in. Some output fields are only set for certain maps: btf_id, btf_key_type_id and btf_value_type_id when the map has BTF (and btf_vmlinux_id for struct_ops maps, ifindex, netns_dev and netns_ino when the map is offloaded). For all other maps, these fields now return whatever userspace passed in before. This breaks userspace which reuses one bpf_map_info for several queries. Restore the original behavior by zeroing the structure and copying in only hash and hash_size from userspace before the output fields are filled in. Fixes: ea2e6467ac36 ("bpf: Return hashes of maps in BPF_OBJ_GET_INFO_BY_FD") Reported-by: Quentin Monnet <qmo@kernel.org> Reported-by: Tianyi Chen <hi@tychen.cc> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261016191635.662690-1-daniel@iogearbox.net
2026-10-16Merge tag 'printk-for-7.3-hotfix' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux Pull printk fixes from Petr Mladek: - Switch NBCON consoles to the emergency mode in emergency_restart() to see messages even in this shutdown code path - Postpone the start of printk kthreads after smp_init(). It prevents boot stalls on some HW - Typo fix in a previous hot-fix patch * tag 'printk-for-7.3-hotfix' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux: printk: Postpone kthread startup printk: fix kernel-doc name for nbcon_write_context_is_braille nbcon/reboot: Switch NBCON consoles into the emergency mode in emergency_restart()
2026-10-16Merge tag 'trace-v7.3-rc7' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing fixes from Steven Rostedt: - Fix NULL dereference when copying histogram field variable If event_hist_trigger_parse() creates an action that uses one of pseudo fields for its keys, the hist_data field element will have a NULL for its field member. When looking to match that key, the lookup uses key_field->field->name which will dereference a NULL pointer. Instead use the helper function that handles such cases: expr_field_str(). - Take the trace_types_lock when dropping the buffer_meta reference The tracing_buffer_meta_open() error return drops the trace_array reference with __trace_array_put() instead of trace_array_put() where the former doesn't take the lock and the latter does. - Fix NULL pointer dereference when reading per_cpu/cpuN/buffer_meta These files are created for every possible CPU. But it only contains data for CPUs that have come online. If one is read for a CPU that hasn't come online yet, it triggers a NULL pointer dereference. Have reading of CPUs that have not come on line yet return -ENODEV. - Fix an out-of-bounds shift in the log2 histogram modifier Histograms can take a log2 modifier to have the buckets defined by log2 increments. If a shift is greater than ULONG_MAX or above 2^63, the key falls into a lower order bucket. Use order_base_2() instead of "ilog2(n - 1) + 1" to have all overflowed keys go into the top bucket. - Fix the persistent ring buffer rb_meta_subbuf_idx() accounting The algorithm for finding the index of a sub-buffer in a persistent ring buffer buffer did not take into account the meta data at the beginning of the buffer causing the wrong index to be returned. As rb_subbufs_from_meta() does the proper accounting to find the first sub-buffer, use that function instead of repeating the logic (incorrectly) and make the calculations based off of that. - Fix the symbol name checks for the remote buffer code The remote ring buffer has a build time check to make sure its not using any code that will not be in the hypervisor. For various architectures, there has been false positive reports from this checker. Fix them. - Fix false warning in ring_buffer_map_get_reader() There is a WARN_ON() after rb_get_reader_page() if no reader is returned. But there are legitimate cases where this can happen. Remove the WARN_ON() and also a check that covers one of the legitimate cases that tries to exit out early. There's no need for the check if it just simply allows the return of NULL from rb_get_reader_page(). - Fix the ignored return value from resize_buffer_duplicate_size() Some clean up of the code in tracing_snapshot_write() removed checking the return value of resize_buffer_duplicate_size() and if that failed it would not be detected. - Fix the histogram timestamps for buffered events The histogram timestamps are taken from the events that are recorded. It currently reads ring_buffer_event_time_stamp() but if the event is buffered it does not read it properly. Use tracing_event_time_stamp() which handles both buffered and normal events. - Do not initialize tracefs if tracing is disabled at boot If tracing fails to allocate or some other error is detected, tracing_disabled is set. Do not initialize tracefs in this case as it may expect data to be initialized but is not. - Free eventfs inode after an RCU grace period Currently eventfs_inodes are freed after a SRCU grace period as the iterations in the code use SRCU. There are locations that the inode may still be accessed from the VFS that uses RCU which does not have the same synchronization as SRCU. Add an RCU wait before free eventfs inodes. - Do not fail persistent and remote buffers under memory pressure As persistent and remote buffers have their own pre-allocated memory, there's no reason to check how much memory is available before created them. By checking the memory pressure, these buffers may fail to be created even though their memory is already present. - Account for subbuffer size when checking how much memory is available Before allocating a ring buffer, the size is checked against the current memory pressure of the system. But it assumes every sub-buffer is of page size. This is no longer the case. Use the actual sub-buffer size when calculating how much memory will be used. - Reject non-stack fields for synthetic stacktrace fields Stack traces are special fields and the checks currently do not do enough to make sure that the fields being processed as stack traces are indeed stack traces. Update the logic to properly detect what is or is not a stack trace. - Fix a crash in print_graph_function_event() When a tracer does not define static or default flags the tr->current_trace_flags is set to NULL. If the ring buffer is not cleared when switching tracers, the previous tracers events could be read. The function_graph entries will dereference the tr->current_trace_flags and cause a crash. Set the tr->current_trace_flags to a empty set of flags to prevent it from being NULL. - Separate tracing filter predicate count from allocation The amount allocated for the predicate program is the number of predicates defined plus two for adding "true" and "false" at the end. The nr_preds is incremented by two to accommodate the booleans but is then used as the limit check for number of predicates. This is incorrect as it is now two higher than what is expected. Use a separate variable for the size of the predicate array. - Fix the filter operator stack check The operator stack increments the pointer and then writes the content onto the stack. The check for overflow compared the current pointer to see if it was greater than the stack size before doing so. This is an off by 2 error as the write will be at the size - 1 and the compare is done before the increment. Adjust the check accordingly. * tag 'trace-v7.3-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: tracing: Check operator stack capacity before pushing tracing: Keep the predicate limit separate from program capacity tracing: Fix crash in print_graph_function_event() tracing/histograms: Reject non-stacktrace fields for synthetic stacktrace fields ring-buffer: Use subbuf_order in available memory estimation ring-buffer: Skip si_mem_available() for persistent and remote buffers eventfs: Free the eventfs_inode after an rcu grace period too tracing: Do not initialize tracefs work if tracing is disabled tracing: Fix hist trigger timestamps for buffered events tracing: Fix ignored return value of resize_buffer_duplicate_size() ring-buffer: Fix false warning in ring_buffer_map_get_reader() tracing: Fix simple_ring_buffer undef check on sparc64 tracing: Fix simple_ring_buffer undef check on parisc tracing: Fix empty symbol names in simple_ring_buffer undef check ring-buffer: Fix the sub-buffer array base in rb_meta_subbuf_idx() tracing: Fix the shift out of bounds in the log2 histogram modifier ring-buffer: Do not show buffer_meta for CPUs without a buffer tracing: Take trace_types_lock when dropping the buffer_meta reference tracing: Fix NULL dereference when copying keys for a field variable
2026-10-16Merge tag 'probes-fixes-v7.3-rc7' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull probes fix from Masami Hiramatsu: - error out if BTF is required but unavailable Fix to return an error instead of NULL if BTF is not available when user specify '$args*' meta fetch argument for probe events. * tag 'probes-fixes-v7.3-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: tracing: probes: error if BTF is required but unavailable
2026-10-16Automated merge of 'lsm/dev' into 'stable-7.4'Paul Moore
Manual merge fixup in ns_common_types.h:ns_common to work around a conflict with the nsfs RCU grace period fix. * lsm/dev: lsm: remove redundant NULL check in security_init() selftests/landlock: Check full ioctl commands in audit records lsm: Preserve full ioctl commands in audit records doc: fix LSM ordering description for /sys/kernel/security/lsm doc: describe CONFIG_LSM and lsm= as the selection mechanism selftests/bpf: verify mount idmaps reach inode hooks lsm: expose mount idmaps to inode hooks rust: lsm: replace `core::mem::zeroed` with `pin_init::zeroed` cred: prevent slab cache merging for cred_jar lsm: don't call security_backing_file_free() multiple times lsm: delete duplicate assignment in lsm_prepare() lsm: update the BUILD_BUG_ON() in audit_log_lsm_data() lsm: add LSM_AUDIT_DATA_NS for namespace audit records lsm: add LSM blob and hooks for namespaces ns: Free anonymous mount namespaces via ns_common_free() cred: clarify that task_struct::cred is only for the current task
2026-10-16bpf: Refuse a trampoline on an unreliable static function with stack argsYonghong Song
When the BTF of a function in the target program is unreliable, the model of the function given to its trampoline is MAX_BPF_FUNC_REG_ARGS u64 arguments. The verifier takes a static function's stack arguments from its BTF before it finds the BTF unreliable, though, so such a function can still take more: one with six or more arguments, one of them a plain pointer, for example. The trampoline saves and restores five registers then. After an fentry program, the rest of the register arguments, r9 on x86-64 and x5 to x7 on arm64, hold whatever __bpf_prog_exit() left there, and for fexit the stack arguments are not copied either, so the function goes on with garbage in arguments the verifier checked, pointers included. Refuse the attach when such a function has more argument slots in its BTF than fit in registers. The main program is left alone: it takes only its context, and the verifier gives it no stack arguments when its BTF is unreliable. Fixes: 0f6bd5e7a804 ("bpf: Support stack arguments for bpf functions") Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261016183120.1136008-1-yonghong.song@linux.dev
2026-10-16bpf: Describe a BPF function to its trampoline as one u64 per slotYonghong Song
A trampoline on a BPF function, fentry, fexit or fsession attached through attach_prog_fd, takes the function's arguments from where BPF code left them. A call from one BPF function to another passes slot N where the kernel convention passes its Nth u64 argument, in a register or a stack slot, whatever the types of the arguments: the x86-64, arm64 and riscv JITs all place a stack argument that way. The model of the function, distilled from its BTF, still carries those types, so each arch places the arguments by its own rules for them. For a 16-byte argument at an odd slot, such as s in u64 f(u64 a, struct { __int128 v; } s, u64 b) the arm64 trampoline reads s from x2 and x3, where the caller left it in x1 and x2, and b from x4. And an x86-64 trampoline puts a 16-byte argument at slot 6 wholly on the stack, while x86-64 BPF code splits it between r9 and the stack. The s390 trampoline, which takes a 16-byte argument by reference, loads s through its first half as if that were a pointer, and saves and restores only three of the four registers the caller passed. Rewrite the model of a target program's function in bpf_check_attach_target() as one u64 argument per argument slot. Every convention places N u64 arguments in slot order, so all of the above take them from where BPF code put them, and no arch has to know that the function is a BPF one. The model no longer gives the size and flags of each argument, so an arch that sign-extends a narrow signed argument, or clears the unused bytes of a narrow stack argument, for the program does not do that for a BPF function any more: the program gets the eightbyte BPF code passed, as it does on x86-64, and fexit hands it back unchanged. Fixes: afdbb4cdd3d2 ("bpf, arm64: Place trampoline arguments by the arm64 calling convention") Fixes: 4dd676ca5b7a ("bpf: Support by-value struct arguments up to 16 bytes") Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261016183100.1134528-1-yonghong.song@linux.dev
2026-10-16bpf: Claim the per-CPU send_signal irq_work before filling itYuan Chen
irq_work_is_busy() cannot see the per-CPU send_signal_work while it is being filled: the check only matches after irq_work_queue() has claimed the work. A concurrent caller therefore passes it, both callers race for the same irq_work, and the loser's signal is silently lost along with its task reference while the queued work runs with a mix of both callers' fields. kprobe, tracepoint and perf_event programs exclude each other on the same CPU through bpf_prog_active, so for two callers to meet, one of the programs has to be a raw tracepoint or an fentry one. And since commit 87c544108b61 ("bpf: Send signals asynchronously if !preemptible") this path runs with IRQs enabled too, so a hard IRQ can interrupt the fill as well, not only an NMI. Found by code inspection and reproduced with a perf_event NMI program racing an fentry one. Claim the work through work->task itself: cmpxchg() it from NULL to the target task before touching the other fields, and set it back to NULL once the callback has consumed them. A context finding the work claimed returns the documented -EBUSY. Fixes: 1bc7896e9ef4 ("bpf: Fix deadlock with rq_lock in bpf_send_signal()") Signed-off-by: Yuan Chen <chenyuan@kylinos.cn> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261009022427.968304-1-chenyuan@kylinos.cn
2026-10-16tracing: Save the event file instead of the compatible histogramDonggeun Yoo
create_field_var_hist() creates a histogram on the matched event to supply a field variable, and records it on the target so it can be torn down later: hist_data = find_compatible_hist(target_hist_data, file); ... /* Save the compatible histogram information */ var_hist->hist_data = hist_data; hist_data is not the histogram it creates. It is one that was already there, whose keys the new one copies. The saved pointer has a single user, unregister_field_var_hists(), which runs when the target is removed: file = hist_data->field_var_hists[i]->hist_data->event_file; find_compatible_hist() matches on keys alone, so it can pick one with no variables of its own. check_var_refs() then returns false and the user can remove it while the target still points at it: BUG: KASAN: slab-use-after-free in event_hist_trigger_free+0x2b2/0x320 Read of size 8 at addr ffff8880093b90e0 by task init/1 Freed by task 1: kfree+0x154/0x420 event_hist_trigger_free+0x1d5/0x320 event_hist_trigger_parse+0x35f3/0x69e0 trigger_process_regex+0x1a6/0x250 Save the event file instead. It is all the dereference ever wanted, and it does not go away when the user removes a trigger. Cc: stable@vger.kernel.org Reported-by: Sashiko <sashiko-bot@kernel.org> Link: https://lore.kernel.org/all/20260914110753.221E61F000FF@smtp.kernel.org/ Link: https://patch.msgid.link/20260914234754.3676905-1-donggeunyoo.kernel@gmail.com Fixes: 02205a6752f2 ("tracing: Add support for 'field variables'") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Acked-by: Tom Zanussi <zanussi@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-16bpf: Free the subprog jump tables once the CFG passes are doneDaniel Borkmann
The jump tables which compute_subprog_jts() builds are only read through bpf_insn_successors() once CFG discovery is done. Its last caller is bpf_compute_scev(), which runs after bpf_compute_live_registers(), so free the tables right after it instead of keeping them until the verifier environment is torn down. With large insn_array maps, this also drops them before do_check() starts to allocate states. Dead code removal runs after do_check(), so a removed subprogram no longer owns a table and adjust_subprog_starts_after_remove() can go back to not caring about the table pointers. Suggested-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261016155559.652461-1-daniel@iogearbox.net
2026-10-16bpf: Drop unnecessary ftrace_location() in ↵Leon Hwang
register/unregister/modify_fentry_multi() Since 'tr->ip' is already a 'ftrace_location()'-ed IP in bpf_trampoline_lookup(), it is unnecessary to call ftrace_location() again in register/unregister/modify_fentry_multi(). No functional changes intended. Signed-off-by: Leon Hwang <leon.hwang@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://patch.msgid.link/20261016142014.12420-3-leon.hwang@linux.dev
2026-10-16bpf: Initialize ftrace_managed in bpf_trampoline_get()Leon Hwang
A trampoline managed by ftrace is determined by its target, that said ftrace_managed is already known when the trampoline is initialized. so, set 'ftrace_managed = tr->ip != 0' in bpf_trampoline_get() instead of relying on the first regular or tracing_multi attachment to set it. This makes the backend selection independent of attachment order and keeps the registration callbacks focused on installing the trampoline. No functional change intended. Signed-off-by: Leon Hwang <leon.hwang@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://patch.msgid.link/20261016142014.12420-2-leon.hwang@linux.dev
2026-10-16tracing: Hold trace_types_lock when resetting last boot data from readersDavid Carlier
update_last_data_if_empty() is called from tracing_open() with O_TRUNC, tracing_read_pipe() and tracing_buffers_read() without trace_types_lock. Two readers can both pass the LAST_BOOT test and free module_delta twice, and a module notifier can update module_delta while it is being freed. Take trace_types_lock in those paths and keep the callers that already hold it on an unlocked variant. Cc: stable@vger.kernel.org Fixes: 32dc0042528d ("tracing: Reset last-boot buffers when reading out all cpu buffers") Link: https://patch.msgid.link/20261016061638.214847-2-devnexen@gmail.com Signed-off-by: David Carlier <devnexen@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-16tracing: Hold trace_types_lock when updating modules in persistent instancesDavid Carlier
trace_module_record() walks ftrace_trace_arrays from the module notifier without trace_types_lock. A concurrent rmdir of an instance removes it from the list and frees it under that lock, so a module load or unload can walk into a freed trace_array. Cc: stable@vger.kernel.org Fixes: 5f3719f697c3 ("tracing: Update modules to persistent instances when loaded") Link: https://patch.msgid.link/20261016061638.214847-1-devnexen@gmail.com Signed-off-by: David Carlier <devnexen@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-16Merge branch 'for-7.3-hotfix' into for-linusPetr Mladek
2026-10-16tracing/syscalls: Use copy_from_user for the faultable syscall bufferHaoran Wang
syscall_copy_user_array() copies the buffer argument of faultable syscalls (write, pwrite64, ...) with __copy_from_user(), which skips access_ok(). Use copy_from_user() instead so a non-user address is rejected and the entry is recorded as faulted. It could be a security concern for machines that trust perf a bit too much. Cc: stable@vger.kernel.org Fixes: baa031b7bd2ce ("tracing: Allow syscall trace events to read more than one user parameter") Link: https://patch.msgid.link/6ad19afd.a938ce27.16851e.3440@mx.google.com Signed-off-by: Haoran Wang <haoranwangsec@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-16printk: Postpone kthread startupJohn Ogness
There have been various bug reports related to SMP systems hanging on boot. The reports vary but seem to have some overlap with regard to missed IPI's during SMP setup. The reports also show that reverting NBCON console usage allows the bugs to disappear. The NBCON kthreads for console printing are started very early during boot using early_initcall(). Since these kthreads are asynchronously enabling/disabling interrupts during early boot, they may be interfering with early SMP setup on some hardware platforms. Starting the console printing kthreads later has shown to avoid the hanging on systems that could previously reproduce the issue. Postpone kthread starting until after all arch initcalls have completed. Reported-by: Derek Barbosa <debarbos@redhat.com> Closes: https://lore.kernel.org/linux-rt-users/arPmYPbFLf2EldIl@debarbos-thinkpadt14gen5.rmtusma.csb Fixes: 76f258bf3f2a ("printk: nbcon: Introduce printer kthreads") Signed-off-by: John Ogness <john.ogness@linutronix.de> Tested-by: Derek Barbosa <debarbos@redhat.com> Link: https://patch.msgid.link/20261016085441.857695-1-john.ogness@linutronix.de Signed-off-by: Petr Mladek <pmladek@suse.com>