summaryrefslogtreecommitdiff
path: root/kernel
AgeCommit message (Collapse)Author
2026-10-16Merge tag 'vfs-7.3-rc8.fixes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs Pull vfs fixes from Christian Brauner: - Fix a range of use-after-frees during rcu pathwalk caused by non-rcu frees. Whatever ->permission(), ->d_revalidate(), ->get_link(), ->d_hash() and ->d_compare() read in rcu mode has to be freed after a grace period. umount(2) hides most of it because namespace_unlock() waits for one, but a chdir() out of a lazily unmounted directory or the close() of the last file on it drops the last reference without. * ext4_get_link() reads the sbi and the extent status counters for a symlink stored in a data block. Both are freed after a grace period now. * The unicode map of casefolded ext4, f2fs and tmpfs was freed under generic_ci_d_hash() and generic_ci_d_compare(). * ntfs3 with nocase freed its sbi and mount options under its dentry hash and compare. A remount freed the old options right away too. * eventfs freed an eventfs_inode after an SRCU grace period only while tracefs_d_revalidate() reads it under rcu_read_lock(). * afs and 9p are only saved by the grace period their private bdi gets in bdi_unregister(). They stop depending on it. * create_ipc_ns() leaked the mqueue mount and its superblock on a late failure. A long-term mount needs kern_unmount(). - Don't let idle threads pin the initramfs. The idle threads of the secondary cpus are forked from PID 1 with a private copy of its fs_struct, aren't on the task list so pivot_root() never moves them and never exit. So the initramfs stayed pinned forever on every machine with more than one cpu, 70 to 150 MB in the reports. They share init_fs now like init_task does. - A mount that copy_tree() leaves out can't take over a lock. copy_tree() drops mounts of mount namespace files from the copies it makes for unshare(CLONE_NEWNS) and propagation without looking at MNT_LOCKED. So root in a user namespace could bind a child's /proc/<pid>/ns/mnt beneath a locked cover, let it take the lock over, unshare(CLONE_NEWNS) and read the file. copy_tree() refuses with EPERM now and the move beneath a locked mount is refused up front. - Revert "namei: don't reveal overmounted entries in refwalk" from the rc7 fixes. The bug class is bigger than that patch. - namei: lookup_fast_for_open() checked ->d_inode while traverse_mounts() checks the type in d_flags, which is set second, so a refwalk racing the instantiation could fail an O_CREAT open with ENOENT. Use d_is_negative() in both places. - Drop the smp_mb()s in mntput_no_expire_slowpath() and do_umount() that mnt_get_count()'s own barrier made redundant. - fsnotify: a mark added while fsnotify_destroy_marks() walks the connector could be missed and the connector detached, so an fdinfo read called igrab(NULL). New marks are refused during teardown. - pipe: a failed second fasync enable on an O_RDWR FIFO removed the existing reader entry and cleared FASYNC while the writer entry stayed, a use-after-free in send_sigio() later. - watch_queue: post_one_notification() could see nr_notes before the note pool behind it and dereference a NULL array. An unprivileged user can trigger that with a key watch. Publish nr_notes last. - pid_namespace: zap_pid_ns_processes() waits without bound for a tracer outside the namespace to reap its zombies and neither wait was freezable, so every suspend attempt failed. - exit: is_child_subreaper and has_child_subreaper shared a word with other bitfields written without a common lock, so racing prctl()s lost a bit. They are separate bools now. - autofs: a notification write that failed on a pipe the daemon had already replaced put the superblock into catatonic mode again and closed the replacement. Only do that for the current pipe. - ovl: reject a lowerdir ending with an unescaped backslash with EINVAL instead of a confusing ENOENT later. - ocfs2/dlm: take dlm_reco_state_lock when the debugfs state file walks the recovery participant list. - samples/pidfd: fix the userprogs typo that kept pidfd-metadata from being built. * tag 'vfs-7.3-rc8.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (24 commits) Revert "namei: don't reveal overmounted entries in refwalk" selftests/filesystems: test that a mntns file can't take over a lock fs: let idle threads share init_fs namespace: a mount that copy_tree() leaves out can't take over a lock namei: test for a negative dentry the way traverse_mounts() does fs: pass flags to copy_fs() ovl: reject lowerdir ending with unescaped backslash fsnotify: prevent mark additions during connector teardown pid_namespace: make zap_pid_ns_processes() freezable fs: drop the smp_mb()s that mnt_get_count() made redundant autofs: ignore notification errors from a replaced pipe ntfs3: free replaced mount options after a grace period ntfs3: free the sbi after a grace period unicode: free the map after a grace period ipc: drop the mqueue mount with kern_unmount() on setup failure eventfs: free the eventfs_inode after an rcu grace period too 9p: free the session after a grace period watch_queue: Fix note pool publication race in watch_queue_set_size() afs: free the super info after a grace period pipe: preserve existing fasync readers on writer setup failure ...
2026-10-16Automated merge of 'audit/dev' into 'stable-7.4'Paul Moore
Manual merge fixup in audit_tree.c:kill_rules() to work around a conflict with a UAF fix that was merged for v7.3-rc6. * audit/dev: audit: separate file and process capability storage audit: Fix filter rule accounting after automatic removal audit: use copied skb length in kauditd_send_multicast_skb() audit: add missing unlikely hint to audit_dummy_context() wrappers audit: free proctitle in context so it can be re-set by fork on exec_binprm
2026-10-16Automated merge of 'audit/topic-7.3-audit_fsopen' into 'stable-7.4'Paul Moore
* audit/topic-7.3-audit_fsopen: audit: add FSOPEN record to log filesystem name
2026-10-16sched: Replace nr_iowait_cpu() with a bool accessorChristian Loehle
The menu governor was the last consumer of the per-CPU I/O-wait count that needed its magnitude, apart from nr_iowait() summing it globally. Commit 38f83090f515 ("cpuidle: menu: Remove iowait influence") removed that use. All remaining per-CPU consumers only test for zero/nonzero. Replace nr_iowait_cpu() with bool cpu_has_iowait() and update the generic and s390 idle accounting callers. Have nr_iowait() read the counters directly so that the global total and existing accounting are preserved. Replace the stale menu comment with a description of the predicate. Also convert the remaining open-coded idle accounting checks to use the predicate, as suggested by Prateek. The conversion was done with coccinelle: @ predicate depends on patch @ expression cpu; @@ - nr_iowait_cpu(cpu) > 0 + cpu_has_iowait(cpu) @ callers depends on patch @ expression cpu; @@ - nr_iowait_cpu(cpu) + cpu_has_iowait(cpu) Signed-off-by: Christian Loehle <christian.loehle@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com> Link: https://patch.msgid.link/20261016073341.575472-1-christian.loehle@arm.com
2026-10-16Revert "sched/fair: Remove dead code on enqueue_task_fair()"Kayra Cizmeci
This reverts commit 819224e506bc7c2d61ec6a58ec6e876b505abce9. The cfs_rq->curr == se case can actually happen with CFS_BANDWIDTH, so the removed code is needed. Document this. Fixes: 819224e506bc ("sched/fair: Remove dead code on enqueue_task_fair()") Reported-by: Christian Loehle <christian.loehle@arm.com> Signed-off-by: Kayra Cizmeci <kayracizmeci@gmail.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/lkml/fd3f9d83-cb5b-496b-b7c1-b40f31cf795e@arm.com/ Link: https://patch.msgid.link/20261015082919.4344-1-kayracizmeci@gmail.com
2026-10-16sched/cache: Avoid publishing partial LLC utilizationJemmy Wong
update_sg_lb_stats() sums utilization over the CPUs in both sched_group_span(group) and env->cpus, but takes capacity from group->sgc->capacity for the whole group. record_sg_llc_stats() publishes both. Rejecting an all-affinity-pinned source runqueue removes that CPU from env->cpus and retries group selection. The retry can publish the partial utilization with the full capacity. can_migrate_llc() then treats a busy preferred LLC as having spare capacity. More tasks can aggregate into it, migrations out can be blocked, and two busy LLCs may not fall back to generic balancing. This increases CPU contention and task latency. Found by inspection. Skip publication on affinity retries using LBF_ALL_PINNED, retaining the first-pass snapshots and avoiding repeated cpumask checks. On the first pass, publish either shared LLC statistic only when env->cpus covers the whole group. Load balancing still uses the affinity-filtered statistics. The same window exists during CPU hotplug. env->cpus drops a CPU at set_cpu_active(false), but the group span and sgc->capacity still include it until cpuset_cpu_inactive() rebuilds the domains. Keeping the previous snapshot avoids the mismatched publish. After the rebuild, the next periodic balance records both statistics again. Fixes: f025ef275388 ("sched/cache: Record per LLC utilization to guide cache aware scheduling decisions") Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com> Suggested-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Jemmy Wong <jemmywong512@gmail.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20261013084338.55100-1-jemmywong512@gmail.com
2026-10-16sched/cputime: Don't account idle time twice after dyntick-idleStian Halseth
On idle exit the dyntick-idle accounting accounts the time up to now, then the tick is restarted on its old period. The first tick accounts a whole TICK_NSEC to whatever runs, although the part of that period before the idle exit has just been accounted as idle time, or with IRQ_TIME_ACCOUNTING as IRQ time. That is up to a full tick per idle exit, and /proc/stat reports more idle time than wall time. Record how much of the current tick period dyntick-idle has accounted, and leave it out of the tick that ends the period. Fixes: cf6444c3e1bb7 ("tick/sched: Unify idle cputime accounting") Signed-off-by: Stian Halseth <stian@itx.no> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Frederic Weisbecker <frederic@kernel.org> Link: https://lore.kernel.org/all/20261004142724.3896396-1-stian@itx.no/ Link: https://patch.msgid.link/20261007142950.547604-1-stian@itx.no
2026-10-16sched/psi: Remove the RCU wait from trigger destructionShakeel Butt
In the Meta fleet, we found PSI trigger destruction waiting for an RCU grace period while holding kernfs node_mutex, blocking other users [1]. File close and node drain hold this mutex while calling the release callback. Cgroup removal and disabling cgroup.pressure also hold cgroup_mutex during the drain. The wait was added by commit 0e94682b73bf ("psi: introduce psi monitor") to protect worker and trigger lookups. Commit 461daba06bdc ("psi: eliminate kthread_worker from psi trigger scheduling mechanism") replaced the worker queue with a group timer. The scheduler now only checks whether rtpoll_task is NULL; it uses neither the task nor the trigger. Commit a06247c6804f ("psi: Fix uaf issue when psi trigger is destroyed while being polled") removed trigger replacement and polling's RCU lookup. Trigger lists are protected by mutexes. Proc poll/select entries are removed before their file references are dropped. eventpoll_release() removes epoll entries before the file release callback. Commit aff037078eca ("sched/psi: use kernfs polling functions for PSI trigger polling") moved cgroup polling to a kernfs waitqueue whose lifetime follows the file. A late timer firing only wakes the group waitqueue. The system group is static, and commit 5457025fa8ca ("sched/psi: Shut down rtpoll_timer in psi_cgroup_free()") shuts down the cgroup timer before freeing the group. Cgroup reclamation already waits for scheduler readers. Remove synchronize_rcu() from psi_trigger_destroy() and use rcu_access_pointer() for the worker NULL check. Keep kthread_stop() outside the trigger mutex. Tested trigger churn, notifications and cgroup removal in an 8-CPU VM with KASAN, lockdep, and full and lazy preemption. No warnings were found. Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Tejun Heo <tj@kernel.org> Link: https://github.com/bpftrace/user-tools/tree/master/runnablelockmonitor [1] Link: https://patch.msgid.link/20261010191500.3895736-1-shakeel.butt@linux.dev
2026-10-16perf: Update cgroup time when rotating multiplexed eventsAntoine Huret
Since commit bd2756811766 ("perf: Rewrite core context handling"), perf_rotate_context() only updates the context time before scheduling out the flexible events. It no longer updates the cgroup time, which ctx_sched_out() used to do. Cgroup events take their times from the per-cpu cgroup clock, so between two cgroup switches the rotations see a frozen clock. The events rotated out lose the running time they accumulated, and the next update, usually a read, credits the whole interval to the events that happen to be on the PMU at that moment. As a result, the time_running of multiplexed cgroup events is wrong, and so are their scaled counts, as long as no other cgroup runs on the CPU, e.g. a busy job alone on its CPUs. Counting cycles over one second of a busy loop in a cgroup gives from half to several times the max frequency of the CPU. Update the cgroup time before rotating the CPU context, as was done before the rewrite. Fixes: bd2756811766 ("perf: Rewrite core context handling") Closes: https://lore.kernel.org/r/PR3PR07MB69223331EED5BEBE150375B7849C2@PR3PR07MB6922.eurprd07.prod.outlook.com/ Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Antoine Huret <antoine.huret@bull.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20261015-perf-cgroup-rotation-time-v1-1-93e177dfee5d@bull.com
2026-10-16perf: Fix perf_pmu_migrate_context() leaving events behindHaris Okanovic
__perf_pmu_remove() walks the group leaders of a CPU and PMU with perf_event_groups_for_cpu_pmu(), removing each one as it goes. perf_remove_from_context() takes the leader out of the groups rbtree, after which rb_next() on that node returns NULL, prematurely ending the walk. Other groups stay behind in the source context. On CPU hotplug, they are then torn down with the dying CPU. They keep running until teardown, while the PMU's interrupt and migrated events are on a new CPU, which drivers rely on never happening. Walk ctx->event_list instead and pick out the PMU's events by their pmu_ctx. Fixes: bd2756811766 ("perf: Rewrite core context handling") Signed-off-by: Haris Okanovic <harisokn@amazon.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20261013185603.1347265-2-harisokn@amazon.com
2026-10-16perf: Track the perf mlock gift separatelyLorenzo Stoakes (ARM)
The locked_vm field in struct user_struct was originally introduced by commit 789f90fcf6b0 ("perf_counter: per user mlock gift") to provide an mlock-like budget for perf measured against mm->pinned_vm (since 2011). perf grants each user a 'gift' of perf_event_mlock_kb * online CPUs of buffer pages which is charged to user->locked_vm without any RLIMIT_MEMLOCK check. Only pages beyond the gift are checked against RLIMIT_MEMLOCK, per-mm, via mm->pinned_vm, which makes it different from the standard mlock() check, which is per-mm and made against RLIMIT_MEMLOCK. However, since this was introduced, a number of other components have utilised this field where a shared resource needed to be limited against RLIMIT_MEMLOCK. The other users are currently MSG_ZEROCOPY, io_uring, AF_XDP, iommufd, s390 KVM zPCI and most recently, secretmem in commit 97d34aa65c29 ("mm/secretmem: properly account locked pages"). This field is shared between all of these, but because the gift is not checked against RLIMIT_MEMLOCK, user->locked_vm alone can exceed it - 16 CPUs * 516 KiB is already more than the default 8 MiB - after which every other user of the field fails unconditionally. This was reported by a user who saw secretmem failing while running a simultaneous perf record. Resolve the issue by simply giving perf its own field separate from the rest. BPF used user->locked_vm up until commit 80ee81e0403c ("bpf: Eliminate rlimit-based memory accounting infra for bpf maps") which landed in 5.11. Thus also eliminate the now-defunct ifdef for CONFIG_BPF_SYSCALL for user->locked_vm. This issue exists between all of the components which use user->locked_vm and perf, however secretmem is the first user who has caused a report over many years and has been backported, so target the fix at that. Fixes: 97d34aa65c29 ("mm/secretmem: properly account locked pages") Closes: https://lore.kernel.org/20261002175651.811343-1-ameer.hamza@truenas.com Reported-by: Ameer Hamza <ameer.hamza@truenas.com> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20261003-perf-locked-vm-fix-v1-1-d214dcaf0d76@kernel.org
2026-10-16sched/fair: Avoid overflow in place_entity()Hui Su
place_entity() inflates an entity's virtual lag with lag * (load + weight) / load Commit 4823725d9d1d ("sched/fair: Increase weight bits for avg_vruntime") replaced the scaled-down weights used in this calculation with avg_vruntime_weight(), removing the arithmetic headroom provided by scale_load_down(). The multiplication can overflow s64 before the division even when the final quotient is representable. A task's hierarchical effective weight can be as small as 2 after avg_vruntime_weight() scaling under group scheduling, while entity_lag() bounds |vlag| by calc_delta_fair(cfs_rq_max_slice(cfs_rq) + TICK_NSEC, se) For example, with HZ=1000, NICE_0_LOAD=1048576, and the minimum scaled hierarchical weight scale_load(2)=2048, this bound is 891289600. With 114 maximum-weight entities, the runqueue load is 10361604096. The intermediate product is: 891289600 * (10361604096 + 2) = 9235189971864780800 > S64_MAX The signed overflow corrupts the entity's placement. Rewrite lag * (load + weight) / load as lag + lag * weight / load The two expressions are equivalent, but the latter avoids multiplying the virtual lag by the total runqueue weight. entity_lag() bounds vlag against the entity's effective weight, so the remaining lag * weight product stays bounded. rescale_entity() preserves this scale across h_load changes and already relies on the same product when rescaling vlag. Keep the original zero-load warning semantics by multiplying lag by weight after replacing the divisor with one. Fixes: 4823725d9d1d ("sched/fair: Increase weight bits for avg_vruntime") Signed-off-by: Hui Su <sh_def@163.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260928124754.1692077-1-sh_def@163.com
2026-10-16bpf: Prevent task work context reuse while irq_work is busyYun Lu
A bpf_task_work context can return to STANDBY before its scheduling irq_work handler returns. Both callback completion and task_work_add() failure can publish STANDBY while the irq_work is still BUSY. A new round can then reinitialize and queue the same irq_work on another CPU while the old handler is still running. One concrete interleaving is: 1. CPU 0 successfully queues ctx->work with task_work_add() and pauses before its SCHEDULING-to-SCHEDULED cmpxchg. 2. The target task on CPU 1 completes the callback, resets ctx->task and publishes STANDBY. 3. CPU 2 starts a new round, changes STANDBY to PENDING and queues the same irq_work. The new handler changes PENDING to SCHEDULING, but task_work_add() returns -ESRCH because its target task has exited. The handler has not yet reset the context or rolled back the state. 4. CPU 0 resumes and changes the new round's SCHEDULING to SCHEDULED, even though no task work was queued for that round. 5. The new handler resets ctx->task, but its SCHEDULING-to-STANDBY cmpxchg fails. The context remains SCHEDULED with a NULL task, and subsequent attempts to acquire this context fail with -EBUSY. 6. Map-value deletion observes SCHEDULED and queues asynchronous cancellation, which passes the NULL ctx->task to task_work_cancel(). Overlapping invocations also share one BUSY bit. An invocation can clear that bit while the other invocation is still running. Initialize the scheduling irq_work when the context is created and stop reinitializing it for each round. After claiming the context with the STANDBY-to-PENDING cmpxchg, check BUSY and reject reuse with -EBUSY if a previous invocation is still in progress. Checking after the cmpxchg orders the BUSY read after the STANDBY publication. No irq_work has been queued for the attempted round at this point, so the rejection can roll PENDING back to STANDBY. Use cmpxchg for that rollback to preserve FREED if a concurrent deletion has published it. Once BUSY clears, scheduling can be retried. Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs") Signed-off-by: Yun Lu <luyun@kylinos.cn> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Mykyta Yatsenko <yatsenko@meta.com> Link: https://patch.msgid.link/20261012061915.231254-4-luyun_611@163.com
2026-10-16bpf: Fix NULL task dereference in task work cancellationYun Lu
bpf_task_work_irq() publishes ctx->work with task_work_add() before changing the context from SCHEDULING to SCHEDULED. The target task can run the callback on another CPU before the handler reaches its final state transition. The handler then reloads ctx->task for cancellation if its final cmpxchg observes FREED. By that point the callback or context destruction may already have cleared ctx->task, causing task_work_cancel() to dereference a NULL task pointer. One concrete interleaving is: 1. CPU 0 changes PENDING to SCHEDULING and successfully queues ctx->work with task_work_add(), but has not attempted the final state cmpxchg. 2. The target task on CPU 1 runs the callback. It changes SCHEDULING to RUNNING, executes the BPF subprogram, resets ctx->task, publishes STANDBY and drops the callback's context reference. 3. CPU 2 deletes the map value. It changes STANDBY to FREED and drops the map's context reference without queuing asynchronous cancellation. 4. CPU 0 resumes. Its SCHEDULING-to-SCHEDULED cmpxchg observes FREED, and the cancellation attempt passes NULL to task_work_cancel(). If deletion publishes FREED before the callback claims RUNNING, the callback takes its FREED exit and drops its context reference. Context destruction can then clear ctx->task before CPU 0 attempts cancellation. A controlled reproducer with a delay before the final cmpxchg triggered the following fault on a v7.3-rc3 based x86-64 KVM guest: BUG: kernel NULL pointer dereference, address: 0000000000000890 RIP: 0010:task_work_cancel+0xd/0xa0 RDI: 0000000000000000 CR2: 0000000000000890 Call Trace: <IRQ> bpf_task_work_irq+0x95/0x100 irq_work_run_list+0x4f/0x90 irq_work_run+0x18/0x50 </IRQ> 0x890 is the task_struct::task_works offset in that build, read through the NULL task argument. Capture ctx->task before task_work_add() publishes the callback and use that pointer for the handler's cancellation attempt. The round still holds its task reference when the pointer is captured. Releasing that reference uses bpf_task_release(), which calls put_task_struct_rcu_user(), so the handler's existing RCU read-side section keeps the captured task alive even if ctx->task is subsequently cleared. The asynchronous cancellation path reads ctx->task directly, which is safe there: queuing it requires deletion to observe SCHEDULED, and that xchg wins against the callback's transition to RUNNING, so the callback takes its FREED exit without resetting ctx->task, while the map reference transferred to the cancellation handler prevents context destruction until cancellation completes. The handler-tail and asynchronous cancellation attempts are mutually exclusive: asynchronous cancellation is queued only if deletion observes SCHEDULED after the scheduling handler's final cmpxchg has succeeded. If that cmpxchg instead observes FREED, the scheduling handler attempts cancellation with the captured pointer. Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs") Reported-by: Jackie Liu <liuyun01@kylinos.cn> Suggested-by: Mykyta Yatsenko <mykyta.yatsenko5@gmail.com> Signed-off-by: Yun Lu <luyun@kylinos.cn> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261012061915.231254-2-luyun_611@163.com
2026-10-16bpf: htab: Reduce elem_size by 8 bytes for small key sizesT.J. Mercier
For standard and PCPU (non-LRU) hash maps with small key sizes (less than or equal to the word size), comparing keys requires only a single instruction. Storing a cached 32-bit hash value to shortcut full key comparisons provides no performance advantage for small keys, and consumes memory for every element. This memory can be saved by removing hash from struct htab_elem and placing it directly before struct htab_elem only for the new htab_elem_hashed type which is used only when keys are larger than the word size or for LRU maps. This reduces elem_size by 8 bytes for small keys while keeping key and hash at constant compile-time offsets from struct htab_elem across all map types and key sizes. All element variants requiring hashes get an anonymous htab_elem_hashed embedding, ensuring that the -8 byte hash offset is guaranteed for all element types by composition. Elements can be recycled without a RCU grace period. Before this commit, alloc_htab_elem() overwrote the key before initializing the value/pptr and wrote the hash last, so during value assignment on a recycled element, the new key was paired with the old hash preventing lockless readers from matching either the old key or the new key except when there was also a hash collision. When hashes are omitted for small keys, only the key field guards lookups. Writing the new key before value/pptr initialization would allow a concurrent lookup of the new key to match an uninitialized value or dereference a stale or freed pptr. So move the new key assignment to the end of alloc_htab_elem() where the hash assignment was done. Like the hash assignment it replaces, it does not enforce value store ordering before key stores on weakly ordered architectures for lockless readers that were already traversing l_new via a stale pointer before hlist_nulls_add_head_rcu(). Before this commit htab_mem_dtor() was invoked with the start of the allocation, which is no longer always the address of struct htab_elem now that elem_offset can be nonzero for non-preallocated, non-per-CPU maps. So the key_size field of struct htab_btf_record has been replaced with a value_offset computed at map creation and passed through bpf_ma_set_dtor(). That reduces the destructor to a call to bpf_obj_free_fields() at a fixed offset, which is exactly what rhtab_mem_dtor() did, so rhtab_mem_dtor() has been removed and replaced with the new implementation of htab_mem_dtor() for BPF_MAP_TYPE_RHASH. Together with the previous patch, this reduces the minimum standard and preallocated hash map element size from 64 bytes down to 32 bytes, and non-preallocated per-CPU element size from 64 bytes down to 40 bytes. Signed-off-by: T.J. Mercier <tjmercier@google.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20261008104108.993791-3-tjmercier@google.com
2026-10-16bpf: htab: Split htab_elem_lru and htab_elem_pcpu off of htab_elemT.J. Mercier
The htab_elem struct is used as the per-element type for all BPF hash map types and includes bpf_lru_node in a union with a ptr_to_pptr pointer. For standard (non-LRU, non-PCPU) hash maps, the 24 byte union allocated for every element is entirely unused. For non-preallocated PCPU maps, ptr_to_pptr only requires 8 bytes, leaving 16 bytes of unused overhead in the union. For preallocated PCPU maps ptr_to_pptr is unused since elements are freed to the PCPU freelist. Eliminate this per-element memory overhead by splitting htab_elem into dedicated structures for each map type: - struct htab_elem: Minimal structure for standard hash maps and preallocated PCPU maps (saves 24 bytes per element). - struct htab_elem_pcpu: Structure for non-preallocated PCPU maps containing ptr_to_pptr (saves 16 bytes per element). - struct htab_elem_lru: Retains struct bpf_lru_node for LRU maps. Place lru_node and ptr_to_pptr before struct htab_elem in htab_elem_lru and htab_elem_pcpu respectively, and track the offset of htab_elem from the start of the element allocation in htab->elem_offset. This keeps key and value at constant compile-time offsets from struct htab_elem across all hash map types, avoiding dynamic key offset calculations on lookups. Use sizeof(struct htab_elem_lru) for the element size rollover check in htab_map_alloc_check() since it is now the largest element variant. Signed-off-by: T.J. Mercier <tjmercier@google.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Mykyta Yatsenko <yatsenko@meta.com> Link: https://patch.msgid.link/20261008104108.993791-2-tjmercier@google.com
2026-10-15ftrace: Add max field to function profiler statsYun Zhou
The function profiler (trace_stat/function<cpu>) reports Hit, Time, Avg and s^2 per function. Add a "max" field recording the maximum single-call duration, which is valuable for real-time tuning and for investigating occasional latency spikes. Like Time/Avg, it honors the graph-time and sleep-time options, so it reflects entry-to-return wall-clock latency. Link: https://patch.msgid.link/20260924030108.2673654-1-yun.zhou@windriver.com Signed-off-by: Yun Zhou <yun.zhou@windriver.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Print a signed histogram key as signedDonggeun Yoo
A histogram sorts its rows by the key named in sort=, and for a key over a signed event field it sorts them signed: create_tracing_map_fields() picks the comparison function with else cmp_fn = tracing_map_cmp_num(field->size, field->is_signed); hist_trigger_print_key() does not look at is_signed at all. Every numeric key is printed with %llu, so a sorted histogram over a signed field comes out looking unsorted. syscalls/sys_exit_lseek carries "long ret", which SYSCALL_FIELD() marks signed and which is negative on every error: # echo 'hist:keys=ret:sort=ret' > events/syscalls/sys_exit_lseek/trigger # echo 1 > events/syscalls/sys_exit_lseek/enable # (five lseek(2) calls returning -ESPIPE, -EINVAL, -EBADF, 0 and 100) # cat events/syscalls/sys_exit_lseek/hist { ret: 18446744073709551587 } hitcount: 1 { ret: 18446744073709551594 } hitcount: 1 { ret: 18446744073709551607 } hitcount: 1 { ret: 0 } hitcount: 1 { ret: 100 } hitcount: 1 The rows are in ascending order and the numbers are not, because the three leading ones are -29, -22 and -9 rendered as their two's complement. Take the rendering from the same place the sort order comes from: the event field when the key has one, and the hist_field itself when it does not. The same histogram then reads { ret: -29 } hitcount: 1 { ret: -22 } hitcount: 1 { ret: -9 } hitcount: 1 { ret: 0 } hitcount: 1 { ret: 100 } hitcount: 1 A ".buckets" key stays unsigned here, and not because it is right. hist_field_bucket() divides as unsigned, so the group holding S64_MAX also holds S64_MIN, reported at https://lore.kernel.org/all/20260914054614.82A9B1F000FF@smtp.kernel.org/ Printing that group signed would name a range that does not contain S64_MIN, so the grouping has to be fixed with it, in a patch of its own. Link: https://patch.msgid.link/20260917045918.370993-1-donggeunyoo.kernel@gmail.com Fixes: 7ef224d1d0e3 ("tracing: Add 'hist' event trigger command") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Fix 64-bit and signed histogram fields on 32-bit kernelsDonggeun Yoo
DEFINE_HIST_FIELD_FN() reads a trace event field and widens it to the u64 the histogram uses as a key or a value: type *addr = (type *)(event + hist_field->field->offset); return (u64)(unsigned long)*addr; unsigned long is 32 bits on a 32-bit kernel, so the intermediate cast discards the upper half of a 64-bit field, and widens a signed field by zero extension from 32 bits rather than sign extension from its own width. Five of the eight instantiations are wrong there: s64, u64, s32, s16 and s8. The cast came in with commit 79e577cbce4c ("tracing: Support string type key properly"), which also added hist_field_dynstring() and hist_field_pstring(). Those convert a char * to u64 and do warn without an unsigned long in between on 32-bit. DEFINE_HIST_FIELD_FN() converts an integer and never a pointer, so it gains nothing from the cast and only loses information. Drop the intermediate cast and let the usual arithmetic conversions widen *addr according to its own type. unsigned long is 64 bits on LP64, so the two forms are identical there and 64-bit kernels are unaffected. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260917015532.103081-1-donggeunyoo.kernel@gmail.com Fixes: 79e577cbce4c ("tracing: Support string type key properly") Reported-by: sashiko-bot@kernel.org Closes: https://lore.kernel.org/all/20260914054614.82A9B1F000FF@smtp.kernel.org/ Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ring-buffer: Take cpus_read_lock() when changing the subbuf orderDavid Carlier
A CPU coming online for the first time gets its per CPU buffer allocated by trace_rb_cpu_prepare(), which then adds it to buffer->cpumask. If that happens while ring_buffer_subbuf_order_set() is running, the CPU shows up after the allocation loop and the install loop walks it with an empty new_pages list, corrupting the buffer. Take cpus_read_lock() as ring_buffer_resize() does. Cc: stable@vger.kernel.org Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page") Link: https://patch.msgid.link/20260916192628.274136-1-devnexen@gmail.com Signed-off-by: David Carlier <devnexen@gmail.com> Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ring-buffer: Remove unnecessary cpumask_test_cpuVincent Donnefort
for_each_buffer_cpu() already iterates over buffer->cpumask. Remove the unnecessary cpumask_test_cpu() found in ring_buffer_subbuf_order_set(). Link: https://patch.msgid.link/20260916152813.105082-3-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ring-buffer: Clean up resize_disabled checksVincent Donnefort
ring_buffer_subbuf_order_set() checks for resize_disabled twice under the same buffer->mutex hold. Moreover, this check duplicates the logic in ring_buffer_resize(). Create a common helper rb_resize_disabled() to factor out this code. Link: https://patch.msgid.link/20260916152813.105082-2-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15fgraph: Enhance funcgraph-retval with BTF-based type-aware outputpengdonglin
The current funcgraph-retval implementation suffers from two accuracy issues: 1. Void-returning functions still print a return value, creating misleading noise in the trace output. 2. For functions returning narrower types (e.g., char, short), the displayed value can be incorrect because high bits of the register may contain undefined data. This patch addresses both problems by leveraging BTF to obtain the exact return type of each traced kernel function. The key changes are: 1. Void function filtering: Functions with void return type no longer display any return value in the trace output, eliminating unnecessary clutter. 2. Type-aware value formatting: The return value is now properly truncated to match the actual width of the return type before being displayed. Additionally, the value is formatted according to its type for better human readability. Here is an output comparison: Before: # perf ftrace -G vfs_read --graph-opts retval ... 1) | touch_atime() { 1) | atime_needs_update() { 1) 0.069 us | make_vfsuid(); /* ret=0x0 */ 1) 0.067 us | make_vfsgid(); /* ret=0x0 */ 1) | current_time() { 1) 0.197 us | ktime_get_coarse_real_ts64_mg(); /* ret=0x187f886aec3ed6f5 */ 1) 0.352 us | } /* current_time ret=0x69380753 */ 1) 0.792 us | } /* atime_needs_update ret=0x0 */ 1) 0.937 us | } /* touch_atime ret=0x0 */ After: # perf ftrace -G vfs_read --graph-opts retval ... 2) | touch_atime() { 2) | atime_needs_update() { 2) 0.070 us | make_vfsuid(); /* ret=0x0 */ 2) 0.070 us | make_vfsgid(); /* ret=0x0 */ 2) | current_time() { 2) 0.162 us | ktime_get_coarse_real_ts64_mg(); 2) 0.312 us | } /* current_time ret=0x69380649(trunc) */ 2) 0.753 us | } /* atime_needs_update ret=false */ 2) 0.899 us | } /* touch_atime */ Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com> Link: https://patch.msgid.link/20260916133218.1282590-4-dolinux.peng@gmail.com Signed-off-by: pengdonglin <pengdonglin@xiaomi.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ftrace: Build trace_btf.c when CONFIG_DEBUG_INFO_BTF is enabledpengdonglin
The trace_btf.c file provides BTF helper functions used by the ftrace subsystem. Make the compilation of trace_btf.c solely depend on CONFIG_DEBUG_INFO_BTF, allowing features like funcgraph-retval to also utilize these helpers. Additionally, the redundant dependency on CONFIG_PROBE_EVENTS_BTF_ARGS is removed, as CONFIG_DEBUG_INFO_BTF already depends on CONFIG_BPF_SYSCALL. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com> Link: https://patch.msgid.link/20260916133218.1282590-3-dolinux.peng@gmail.com Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Avoid BTF lookup in atomic contextpengdonglin
The function graph trace formatter can be called from ftrace_dump_one() with interrupts disabled. When ftrace_dump_on_oops is enabled, the trace panic notifier can reach ftrace_dump() from the panic path, including the nmi_panic() path used by the hard lockup detector. btf_find_func_proto() calls bpf_find_btf_id(), which may initialize vmlinux BTF under a mutex. Skip the BTF lookup when running in an atomic context or with interrupts disabled, and fall back to the existing return value formatting. Link: https://patch.msgid.link/20260916133218.1282590-2-dolinux.peng@gmail.com Fixes: ebeed8d4a555 ("tracing/probes: Move finding func-proto API and getting func-param API to trace_btf") Signed-off-by: pengdonglin <pengdonglin@xiaomi.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Clamp the printed end of a histogram bucket at U64_MAXDonggeun Yoo
A key with the ".buckets=N" modifier is stored as the start of its bucket, and hist_trigger_print_key() prints the range it covers as start to start + N - 1. That addition overflows once the start is within N of U64_MAX: # echo 'hist:keys=offset.buckets=10' >> \ events/syscalls/sys_enter_lseek/trigger # ./seeker 5 -1 # lseek(0, 5) and lseek(0, -1) # cat events/syscalls/sys_enter_lseek/hist ... { offset: ~ 18446744073709551610-3 } hitcount: 1 { offset: ~ 0-9 } hitcount: 1 ... 18446744073709551610 + 10 - 1 does not fit in 64 bits, so the range ends below where it starts. The bucket does end at U64_MAX: it holds every value hist_field_bucket() maps onto its start, and there is no value above U64_MAX. Clamp the printed end there. Printed honestly the group is shorter than the size asked for, which histogram.rst does not mention, so say it there. Link: https://patch.msgid.link/20260914132346.2234731-1-donggeunyoo.kernel@gmail.com Fixes: de9a48a360b7 ("tracing: Add linear buckets to histogram logic") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Unregister field variable histograms when the command failsDonggeun Yoo
An action that names a field on another event needs that field as a variable, so create_field_var_hist() registers a second hist trigger on that event to provide it, and records it in target_hist_data->field_var_hists[]. If a later part of the same command fails, create_actions() returns an error and event_hist_trigger_parse() jumps to out_free. The write fails with -EINVAL and sched_switch gets no trigger, but: # echo 'hist:keys=pid:ts0=common_timestamp.usecs' >> \ events/sched/sched_waking/trigger # echo 'my_synth u64 lat; int a; int b' >> synthetic_events # echo 'hist:keys=next_pid:wakeup_lat=common_timestamp.usecs-$ts0:\ onmatch(sched.sched_waking).my_synth($wakeup_lat,prio,nosuchfld)'\ >> events/sched/sched_switch/trigger # cat events/sched/sched_waking/trigger hist:keys=pid:vals=hitcount:ts0=common_timestamp.usecs:sort=hitcount:size=2048:clock=global [active] hist:keys=pid:vals=hitcount:synthetic_prio=prio:sort=hitcount:size=2048 [active] The second histogram is the one created for 'prio'. It is still active and still counting. out_free reaches destroy_field_var_hists(), which frees the field_var_hists[] entries, but never unregister_field_var_hists(), which is what removes the triggers and runs only from event_hist_trigger_free(). The entry is freed and the trigger is left with nothing tracking it. Unregister the field variable histograms in out_free as well, and record each one as soon as its trigger is registered rather than after the lookup that follows it, so no exit can leave a registered trigger untracked. The entry then belongs to field_var_hists[] and is released by destroy_field_var_hists(), so freeing it on that exit would be a double free. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260914104036.1768529-1-donggeunyoo.kernel@gmail.com Fixes: 02205a6752f2 ("tracing: Add support for 'field variables'") Reported-by: sashiko-bot@kernel.org Closes: https://lore.kernel.org/all/20260913204302.DB2551F000FF@smtp.kernel.org/ Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Only report the stacktrace modifier on a real fieldDonggeun Yoo
get_hist_field_flags() returns "stacktrace" for any field carrying HIST_FIELD_FL_STACKTRACE, including the common_stacktrace pseudo-field, which has no ftrace_event_field behind it. parse_field() no longer takes the modifier there: if (stack_modifier && (!field || field->filter_type != FILTER_STACKTRACE)) { hist_err(tr, HIST_ERR_BAD_FIELD_MODIFIER, errpos(field_str)); so expr_field_str() renders "common_stacktrace.stacktrace", a spelling that cannot be parsed back. Report the modifier only when there is a field to report it for. hist_field_print(), the other caller of get_hist_field_flags(), excludes HIST_FIELD_FL_STACKTRACE before it calls, so this is confined to expr_field_str(), whose only key renderer arrives in the next patch. Link: https://patch.msgid.link/20260914053443.981201-4-donggeunyoo.kernel@gmail.com Fixes: a5e70ba87ca8 ("tracing: Fix memory corruption from the histogram stacktrace modifier") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Add the bucket size to expr_field_str()Donggeun Yoo
expr_field_str() renders a HIST_FIELD_FL_BUCKET field as "pid.buckets", dropping the count, and parse_field() rejects that spelling. get_hist_field_flags() returns the bare string "buckets" and keeps the count in hist_field->buckets. hist_field_print() appends it, expr_field_str() never did. Nothing reaches it today, since .buckets is refused on a value and no expression can carry one, but the next patch renders histogram keys with expr_field_str(), where a .buckets key is legal. Append the count, as hist_field_print() does. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260914053443.981201-3-donggeunyoo.kernel@gmail.com Fixes: de9a48a360b7 ("tracing: Add linear buckets to histogram logic") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Print the bucket size as unsignedDonggeun Yoo
A key's bucket size reads back negative once it exceeds LONG_MAX: # echo 'hist:keys=next_pid.buckets=18446744073709551615' > trigger # cat trigger hist:keys=next_pid.buckets=-1:vals=hitcount:... parse_field() takes the size with kstrtoul() and rejects only zero, so the whole unsigned long range is accepted and stored, but hist_field_print() renders it with %ld. Print it with %lu. Link: https://patch.msgid.link/20260914053443.981201-2-donggeunyoo.kernel@gmail.com Fixes: de9a48a360b7 ("tracing: Add linear buckets to histogram logic") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Assisted-by: Claude:claude-fable-5 Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ring-buffer: Unify ring buffer minimum page calculationsVincent Donnefort
The codebase scatters hardcoded minimum page limits across multiple places. Introduce RING_BUFFER_MIN_NR_PAGES and rb_calculate_nr_pages() to formalize the requirement of two pages in a ring buffer. Link: https://patch.msgid.link/20260911193937.602202-4-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Rename tracing_entries_fops to tracing_capacity_fopsVincent Donnefort
Since commit 8a5f63637890 ("ring-buffer: Fix subbuf resize race with ring buffer readers"), "capacity" refers to the memory available for events, while "entries" refers to the number of events in a ring buffer. As a consequence, rename trace_array::entries to capacity to align with this convention. While at it, rename also tracing_total_entries_fops. It is the aggragate of all ring buffer capacities. Link: https://patch.msgid.link/20260908090937.838544-4-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Rename trace_array::entries to capacityVincent Donnefort
Since commit 8a5f63637890 ("ring-buffer: Fix subbuf resize race with ring buffer readers"), "capacity" refers to the memory available for events, while "entries" refers to the number of events in a ring buffer. As a consequence, rename trace_array::entries to capacity to align with this convention and avoid any confusion. Rename also the helper functions trace_set_buffer_entries and update_buffer_entries. Link: https://patch.msgid.link/20260908090937.838544-3-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ring-buffer: Rename ring_buffer_size() to ring_buffer_capacity()Vincent Donnefort
Since commit 8a5f63637890 ("ring-buffer: Fix subbuf resize race with ring buffer readers"), "capacity" refers to the memory available for events (excluding subbuffer headers) while "size" refers to the raw allocation. ring_buffer_size() actually calculates this event capacity. Rename it to ring_buffer_capacity() to align with this convention. Link: https://patch.msgid.link/20260908090937.838544-2-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Parse filter from boot event stringThomas Weißschuh
Make it possible to specify a trace event filter from the kernel command line. Example: trace_event=syscalls:sys_enter_clock_nanosleep:filter:which_clock==0 Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-7-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Add a filter argument to __ftrace_set_clr_event()Thomas Weißschuh
Make it possible to directly enable a filter when enabling an event. For now this is unused, an upcoming patch will wire it up. Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-6-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Add a test for ftrace_parse_event_string()Thomas Weißschuh
The parsing logic is a bit complicated and about to become more so. Add a unit test to avoid regressions. Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-5-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Split the event string parsing logic into a dedicated functionThomas Weißschuh
This new function can be tested with a KUnit test to prevent regressions when adding new fields to the format. Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-4-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Stop modifying the input buffer in ftrace_set_clr_event()Thomas Weißschuh
ftrace_set_clr_event() modifies its input buffer during parsing. Before returning to the caller these modifications are undone. This is restoration is error prone and was forgotten before. Upcoming changes will also make it more complicated. Copy the input argument and modify that copy and drop the restoration logic. Also mark the input argument as const. Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-3-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Remove duplicate declaration of ftrace_set_clr_event()Thomas Weißschuh
The function is also declared in include/linux/trace_events.h which is even included from kernel/trace/trace.h. Having two declarations creates churn when changing the signature. Remove the unnecessary, duplicate declaration. Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-2-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15ftrace: Have stacktrace option work with func-argsSteven Rostedt
When both the func_stack_trace option is enabled first, the func-args option can not be enabled. Same if the func-args option is enabled first, the func_stack_trace option can not be enabled. Allow the two to work together. The stack trace is already slow and testing the current tracer flags should be in the noise of overhead. Link: https://patch.msgid.link/20260825191036.11b19036@gandalf.local.home Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracnig: Add proper error message for glob use with "within" filterSteven Rostedt
If a user were to use a glob expression with the "within" filter, it currently just fails with an errno return but does not update the error_log to tell the user why it failed. Update it to use the parsing error log to tell the user why it failed. # echo 'within ~ "*_read"' > events/kmem/kmem_cache_alloc/filter -bash: echo: write error: Invalid argument # cat error_log [ 4676.606481] event filter parse error: error: 'within' filter cannot take glob expressions Command: within ~ "*_read" ^ Link: https://patch.msgid.link/20260821233050.488131029@kernel.org Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Add "within" filter for call-stack-based event filteringChen Jun
Low-level kernel functions are called from many different paths. When debugging, it is often useful to filter trace events to only those occurring within a specific call chain. Add a "within" filter predicate that tests whether a given function appears in the current call stack at event time. The function name is resolved to its address range via kallsyms during filter setup; at runtime, stack_trace_save() captures the call stack and compares each return address against the stored range. Example: echo 'within == "vfs_read"' > events/sched/sched_switch/filter Only "==" and "!=" operators are supported. The filter depends on CONFIG_STACKTRACE. Link: https://patch.msgid.link/20260821233050.335986786@kernel.org Link: https://patch.msgid.link/20260608145556.94931-1-chenjun102@huawei.com Signed-off-by: Chen Jun <chenjun102@huawei.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-16genirq/proc: Fix missing CPU header in /proc/interruptsHaris Okanovic
irq_seq_show() must be idempotent since it may be called multiple times to render the same record, e.g. when seq_file's buffer is exceeded. It emits the CPU header behind the one-shot print_header flag, which the first pass clears, so the header is lost with the discarded buffer. This can be seen on large systems, such as 8g.48xl (192 CPUs) and 8i.96xl (384 CPUs) AWS instance types: $ head -1 /proc/interrupts 9: 0 0 ... 0 GICv3 25 Level vgic Replace print_header with the position of the first record and include the header whenever that record is emitted. Fixes: 171cc0d9eed1 ("genirq/proc: Speed up /proc/interrupts iteration") Signed-off-by: Haris Okanovic <harisokn@amazon.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Tested-by: Michael Kelley <mhklinux@outlook.com> Reviewed-by: Radu Rendec <radu@rendec.net> Link: https://patch.msgid.link/20260911-dev-harisokn-proc-interrupts-fix-b4-v1-1-f9ff208efcfa@amazon.com
2026-10-16genirq: Fix typos in commentsHemanth Selam
Correct the spelling of "descriptor" and "preceded" in the kernel-doc for msi_insert_desc() and msi_next_desc(). Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Reviewed-by: Radu Rendec <radu@rendec.net> Link: https://patch.msgid.link/20261009064807.70345-1-hemanth.selam@gmail.com
2026-10-16genirq: Move interrupt thread exit_work object to irqactionRichard Patel
irq_thread() registers irq_thread_dtor() as exit task_work using a stack-allocated callback_head. On normal interrupt thread exit, this work gets cancelled and never runs. On x86, an abnormal interrupt thread exit (e.g. handler oops) discards the stack, including the callback_head, while that callback_head is still pending. By the time exit_task_work() runs, the stack memory formerly backing callback_head is already clobbered. irq_thread() <-- on_exit_work allocated ** IRQ handler faults ** oops_end() rewind_stack_and_make_dead() <-- on_exit_work freed make_task_dead() do_exit(SIGSEGV) ... exit_shm() <-- on_exit_work clobbered ... exit_task_work() <-- stack use-after-free exit_shm()+0x30 <-- wild jump to return addr Thus, a benign threaded IRQ oops (e.g. a NULL pointer deref) is turned into a wild jump. This is caught by kernel IBT: ** first fault ** Oops: general protection fault, kernel NULL pointer dereference 0x3c0: 0000 [#1] SMP NOPTI ... ** second fault ** Missing ENDBR: exit_shm+0x30/0x1c0 kernel BUG at arch/x86/kernel/cet.c:133! ... ? task_work_run+0x5c/0x90 ? do_exit+0x209/0x460 ? rewind_stack_and_make_dead+0x16/0x20 ... note: irq/165-AudioDS[1427] exited with irqs disabled Fixing recursive fault but reboot is needed! Fix this by storing the callback_head in struct irqaction. The irqaction is only freed once irq_thread_dtor() finishes. Fixes: 2deb4be28077 ("x86/dumpstack: When OOPSing, rewind the stack before do_exit()") Signed-off-by: Richard Patel <ripatel@wii.dev> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20261009145842.1616117-3-ripatel@wii.dev
2026-10-16genirq: Defer kfree(irqaction) until irq_thread_dtor() finishesRichard Patel
When an interrupt thread exits abnormally via do_exit() (e.g. the interrupt handler oopses) without returning from irq_thread(), a concurrent call to free_irq() could free the irqaction before the irq_thread_dtor() exit task work runs. irq thread (oopsed) free_irq() ------------------- ---------- do_exit() exit_mm() complete_vfork_done() -------> kthread_stop() returns exit_files() ... kfree(action) exit_task_work() irq_thread_dtor() action->... <- use-after-free Track the pending exit work with the IRQTF_EXIT_WORK thread flag, and have __free_irq() wait for the bit to clear before kfree(). irq_thread_dtor() only runs on irq_thread abnormal exit. If that dtor also faults, free_irq() will hang indefinitely, but double IRQ thread faults require a reboot anyways. [ tglx: Add missing include ] Fixes: ed3e694d78cc ("move exit_task_work() past exit_files() et.al.") Signed-off-by: Richard Patel <ripatel@wii.dev> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20261009145842.1616117-2-ripatel@wii.dev
2026-10-15tracing: Check operator stack capacity before pushingTaeyang Lee
predicate_parse() uses the first operator stack entry before processing any opening parentheses. A push then increments top before writing. The existing check still permits that write when top already points to the last allocated entry, which can access beyond the operator stack. Reject the push when top reaches the last allocated entry. Cc: stable@vger.kernel.org Fixes: 39f7c41c908b ("tracing/filters: Enable filtering a cpumask field by another cpumask") Link: https://patch.msgid.link/atELO9L0I9kg5CpC@Taeyangs-MacBook-Pro.local Signed-off-by: Taeyang Lee <0wn@theori.io> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Keep the predicate limit separate from program capacityTaeyang Lee
predicate_parse() adds two to nr_preds to reserve the terminal TRUE and FALSE entries. It also uses that increased value as the limit for parsed predicates. If the initial count is smaller than the number actually parsed, the program can fill the allocation and subsequent accesses can go past its end. Keep nr_preds as the predicate limit and use a separate value for the allocation size. This leaves space for both terminal entries and a zeroed entry when parsing stops at the limit. Cc: stable@vger.kernel.org Fixes: 39f7c41c908b ("tracing/filters: Enable filtering a cpumask field by another cpumask") Link: https://patch.msgid.link/atELCd1XjKVhrkFL@Taeyangs-MacBook-Pro.local Signed-off-by: Taeyang Lee <0wn@theori.io> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-10-15tracing: Fix crash in print_graph_function_event()Hao Ge
tracing_set_tracer() assigns the flags of the newly selected tracer with: tr->current_trace_flags = t->flags ? : t->tracer->flags; If the new tracer defines neither static flags nor default flags, like hwlat, irqsoff or wakeup, add_tracer() has no per instance flags copy for it, and both operands are NULL, so tr->current_trace_flags ends up being NULL as well. The ring buffer is not cleared when switching tracers: it is reset inside tracer_init(), after current_trace_flags was already assigned. If the buffer still holds function_graph entries of the previous tracer, printing one of them dereferences the NULL pointer. This was hit by cat trace during an ftrace stress test: BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode RIP: 0010:print_graph_function_event+0xe/0x20 Call Trace: s_show+0x4d/0x110 seq_read_iter+0x2d4/0x490 seq_read+0x130/0x170 vfs_read+0xb5/0x370 ksys_read+0x6d/0xe0 do_syscall_64+0xbc/0x450 The snapshot buffer is not reset at all, so this can be reproduced without a race: echo function_graph > /sys/kernel/tracing/current_tracer echo 1 > /sys/kernel/tracing/snapshot echo hwlat > /sys/kernel/tracing/current_tracer cat /sys/kernel/tracing/snapshot Rather than teaching every user of tr->current_trace_flags about this corner case, make sure it is never NULL: when the newly selected tracer has no flags at all, tracing_set_tracer() falls back to a static empty no_tracer_flags. run_tracer_selftest() can hit the same case, so the same fallback is added there too. The existing users already handle opts == NULL, so these tracers keep behaving as before. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20261015130658.221198-1-hao.ge@linux.dev Fixes: e29aa918a928 ("tracing: Have function graph tracer define options per instance") Suggested-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Hao Ge <hao.ge@linux.dev> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>