| Age | Commit message (Collapse) | Author |
|
A local DSQ reenq is queued on the rq through a per-sched node and run from
deferred work. When the reenq is requested from dispatch, the work is a
balance callback queued once dispatch is done. A cpu going down installs
balance push early in its teardown, and queue_balance_callback() refuses new
callbacks from then on. ops.dispatch() keeps running on that cpu until the
rq goes offline after an RCU grace period, so a reenq it requests in that
window may never run. The IMMED handling in dispatch_one() requests one the
same way.
A reenq requested from elsewhere for a cpu that is already offline goes
through irq_work_queue_on(), which warns, and its work doesn't run until the
cpu comes back.
Either way the node stays on the rq's list. When the scheduler is freed,
scx_sched_free_rcu_work() warns and frees the pcpu with the node still
linked, and the list is walked again once the cpu comes back.
An inactive cpu's local DSQ holds only tasks that are bound to it or are
being moved off, so the reenq is pointless. Ignore reenqs of an inactive
cpu's local DSQ.
The test is lockless. The hotplug thread of a cpu going down runs on that
cpu, and it marks the cpu inactive before it installs balance push. A
dispatch on the cpu runs to completion before that thread can run there, so
a dispatch that saw the cpu active has queued its callback by then. Requests
from outside dispatch run from the wakeup path or through irq_work, which
balance push doesn't affect. The grace period after the cpu goes inactive
waits for any requester that saw it active, so that irq_work is queued while
the cpu is online.
Fixes: 238eba8c210d ("sched_ext: Use schedule_deferred_locked() in schedule_dsq_reenq()")
Fixes: 76edc2761ab8 ("sched_ext: Use irq_work_queue_on() in schedule_deferred()")
Cc: stable@vger.kernel.org # v7.1+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Tested-by: Andrea Righi <arighi@nvidia.com>
|
|
* for-next/fw-rmi:
: RMM v2.0 base RMI support
firmware: arm_rmm: hotplug: Skip memory added to ZONE_MOVABLE
firmware: arm_rmm: Add wrappers for Realm related RMI commands
arm64: Block hibernate and kexec while RMM is active
PM: hibernate: Add an arch hook for preventing hibernation
firmware: arm_rmm: Ensure the RMM has GPT entries for memory
firmware: arm_rmm: Activate the RMM
firmware: arm_rmm: Add support for SRO
firmware: arm_rmm: Configure the RMM with the host's page size
firmware: arm_rmm: Check for RMI support at init
firmware: arm_rmm: Add SMC definitions for calling the RMM
arm64: mm: Handle Granule Protection Faults (GPFs)
|
|
The vDSO basetime of a clock is stored in the scaled nanoseconds of
tkr_mono, so that the reader can floor the base and the cycle delta
together in vdso_calc_ns().
vdso_time_update_aux() instead shifts the base down to nanoseconds, adds
the offset, and shifts it back up, which zeroes the fractional nanoseconds
of xtime_nsec. The reader then floors the base and the delta separately:
ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift)
vdso: base + (xtime_nsec >> shift)
+ ((delta * mult) >> shift)
Since floor(a) + floor(b) <= floor(a + b), the vDSO reports 0 or 1 ns
below the syscall for the same clock. It is not a monotonicity problem:
across an update the step is floor(a + d) - floor(a) - floor(d), which is
0 or 1, never negative.
Add the offset in scaled nanoseconds as the other high resolution clocks
do, and normalise with __iter_div64_u64_rem() so that the stored base
stays below one second and the userspace fast-path does not iterate more
in __iter_div_u64_rem().
Only the sub-second field changes. (a + (b << shift)) >> shift is exactly
(a >> shift) + b, so the seconds carried out of the normalisation are the
same as before; what the old form dropped was the low shift bits of the
remainder.
monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays
below NSEC_PER_SEC even for a negative offset, and the sum stays below
2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift()
can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19.
Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the datapage")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-4-zhanxusheng@xiaomi.com
|
|
The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are stored in
the scaled nanoseconds of tkr_mono, so normalising them requires a
division by NSEC_PER_SEC << shift. That divisor does not fit the u32
parameter of __iter_div_u64_rem(), so update_vdso_time_data() open-codes
the same iterative division twice.
Repeated subtraction is the appropriate form at these two sites because
the quotient never exceeds one. accumulate_nsecs_to_secs() keeps
xtime_nsec below one scaled second, and the offset added to it is a
normalised timespec64 fraction, so the dividend stays below twice the
divisor. That bound holds on every architecture, which matters more here
than the cost of a division on any particular one.
Add __iter_div64_u64_rem(), the u64-divisor counterpart of
__iter_div_u64_rem(), and use it at both sites. Return the quotient as a
u32 for consistency with the u32-divisor variant; the bound above leaves
no use for a wider type.
Store the remainder directly into the basetime, as the coarse clocks
already do. The CLOCK_BOOTTIME copy of the CLOCK_MONOTONIC values then
takes both fields from the same place.
Folding the two open-coded loops into one inlined helper removes 16 bytes
of vsyscall.o text on x86-64 with gcc 13.
No functional change.
Suggested-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-3-zhanxusheng@xiaomi.com
|
|
__arch_update_vdso_clock() and __arch_sync_vdso_time_data() are only
used in vsyscall.c. vsyscall.h has become a dumping ground for different
kinds of fallbacks.
Move the fallback function definitions into vsyscall.c.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260909-vdso-asm-generic-vsyscall-h-v1-4-63ae856cc659@linutronix.de
|
|
The vdso/ header namespace is meant for symbols which are usable from
vDSO userspace code. However vdso_update_begin() and vdso_update_end()
are only usable from the kernel.
Move them into a new header in the linux/ namespace. Also rename them to
make clear that they are only for time-related data.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260909-vdso-asm-generic-vsyscall-h-v1-1-63ae856cc659@linutronix.de
|
|
In check_synth_field(), when a synthetic event field is a stacktrace
(field->is_stack) and the histogram field is not a "long[" stacktrace
array, validation falls through to the generic size and signedness
comparison.
Because parse_synth_field() sets a stacktrace field's type to
"__data_loc unsigned long[]", its size is sizeof(u64) and
synth_field_signed() marks it as signed. Consequently, any 8-byte
signed scalar histogram field passes the fallback check, and its scalar
value is subsequently dereferenced as a kernel stacktrace pointer in
get_field_size() and trace_stack().
Furthermore, checking strstr(hist_field->type, "long[") is insufficient
because tracepoint fields defined with __bitmask() or __rel_bitmask()
also have type "__data_loc unsigned long[]" or "__rel_loc unsigned
long[]", whilst create_hist_field() treats them as 4-byte scalar values
without HIST_FIELD_FL_STACKTRACE.
Propagate HIST_FIELD_FL_STACKTRACE to variable references and field
variables (whilst excluding variable aliases from per-element
stacktrace capture), and reject any mismatch between
HIST_FIELD_FL_STACKTRACE and field->is_stack in check_synth_field().
Cc: stable@vger.kernel.org
Fixes: 00cf3d672a9d ("tracing: Allow synthetic events to pass around stacktraces")
Link: https://patch.msgid.link/20261013112449.1006088-1-lee@kernel.org
Signed-off-by: Lee Jones <lee@kernel.org>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
To avoid failing in the middle of an allocation of a ring-buffer,
__rb_allocate_pages() looks at the available memory estimation with
si_mem_available(). Use the subbuffer order with this estimation to fail
early if necessary for ring-buffers with an order > 0.
Link: https://patch.msgid.link/20261013102047.1488347-3-vdonnefort@google.com
Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page")
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
|
|
__irq_exit_rcu() removes HARDIRQ_OFFSET from preempt_count() before the
interrupt exit work. On the outermost interrupt exit, with no NMI or
softirq state remaining, this makes context checks report task context. The
softirq-dispatch, timer-wakeup and tick-exit checks account for the early
decrement, but other code reached in this window sees the wrong context:
- ftrace, perf and the ring buffer record task context and use the
task recursion and context slots.
- KCSAN attributes accesses to the interrupted task. KMSAN uses and
changes the interrupted task's KMSAN context. KCOV and the printk
caller id see a task.
- On PREEMPT_RT can_spin_trylock() and local_trylock() reject hard
interrupt context to avoid interfering with PI when the interrupted
task is blocked on a lock. Those checks do not reject calls made in
this window. BPF programs attached to sched_waking or sched_wakeup
can reach the trylock path through kmalloc_nolock().
- An oops kills the interrupted task instead of ending in "Fatal
exception in interrupt".
The early decrement predates git. No failure caused by this
misclassification is known. lockdep is unaffected because
lockdep_hardirq_exit() is the last operation in irq_exit().
Keep HARDIRQ_OFFSET until the end of __irq_exit_rcu(), including
tick_irq_exit(). Softirq handlers must run in BH context, so replace
HARDIRQ_OFFSET with SOFTIRQ_OFFSET while they run and restore it
afterwards. Use one raw counter update for each transition to preserve
the preemption disable location recorded by irq_enter_rcu(), and update
lockdep explicitly. Detect IRQ exit with in_hardirq() in
softirq_handle_begin() because __do_softirq() can be reached through
do_softirq_own_stack() and cannot take an argument.
Adjust the exit checks for HARDIRQ_OFFSET still being present. Use
irq_count() == HARDIRQ_OFFSET for softirq dispatch, as in irq_enter_rcu().
Use hardirq_count() == HARDIRQ_OFFSET for the timer thread wakeup and
tick_irq_exit(). NMI exits use a separate path. An IRQ nested in an NMI
has an extra HARDIRQ_OFFSET from NMI entry, so it is excluded too. The
timer thread check must ignore softirq_count() to allow wakeups when an
IRQ interrupts softirq processing or a BH-disabled section. The timer
thread handles all pending softirqs, including any raised during its
wakeup.
A softirq raised from a tracepoint on the final preempt_count_sub()
waits for the next interrupt exit and can trigger NOHZ tick-stop
warnings. Handling that instrumentation case does not justify a check
on every interrupt exit. A tracepoint on tick_irq_exit() can also raise
a softirq after the normal softirq handling has finished.
The number of counter updates and interrupt time accounting are
unchanged. The preemptoff tracer now reports the interrupt and subsequent
softirq processing as one section. With nofuncgraph-irqs, function graph
tracing also skips the IRQ exit work, including the __do_softirq() frame.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: LLM
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://lore.kernel.org/r/20260813130826.GW687043@noisy.programming.kicks-ass.net
Link: https://patch.msgid.link/20261011060742.30584-1-kmehltretter@gmail.com
|
|
Some nodes do not allocate any early boot memory. An example is
hotpluggable nodes with movable_node enabled. So the bootmem size for
them ends up being zero.
Zero size allocations normally don't play well with memblock. Memblock
returns an address, but then doesn't actually reserve anything. This
address ends up being recorded in kho_bootmem with a size of 0. In case
of hotpluggable nodes with movable_nodes, the allocation on the node
actually fails because memblock refuses to allocate there, and so KHO
fails to initialize.
Skip bootmem reservation for such nodes entirely.
To keep the code more robust, avoid overloading kho_bootmem_cnt. use
max_nr_bootmem to store the size of kho_bootmem array. This ensures that
the location of kho_bootmem_cnt assignment does not affect the cleanup
paths.
Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
Reported-by: Bing Jiao <bingjiao@google.com>
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20261014131547.259138-1-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Get the upstream modifications in softirq.c to avoid conflicts with pending
patches,
|
|
traceprobe_expand_meta_args expands probe arguments based on BTF info.
Return an error if we have arguments that need expanding but BTF is not
supported.
The params variable is now unused, it can be removed.
Link: https://lore.kernel.org/all/20261014083838.31020-1-martin@kaiser.cx/
Fixes: d157d7694460 ("tracing/probes: Support BTF field access from $retval")
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
The instruction tests of the test_bpf module live in the BPF selftests
now, as the jit_insns test of test_progs, which BPF CI runs on every
architecture it covers, and its skb_segment tests have a KUnit successor
in net/core/net_test.c.
Remove the module together with its Kconfig entry, the test_kmod.sh and
test_bpf.sh wrappers of the bpf and net selftests, the config fragments
and defconfigs that enabled it, as well as all other references to it and
symbol exports in core kernel.
For archs wanting to support BPF JITs, it's time they move over to the BPF
selftests for proper coverage.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261013171230.523677-19-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
Pull workqueue fix from Tejun Heo:
- Fix a NULL dereference in the check for whether the current task is
running a memory reclaim work item when called from a kworker outside
work item execution
* tag 'wq-for-7.3-rc7-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
workqueue: Fix NULL current_pwq deref in mem-reclaim helper
|
|
Persistent and remote buffer memory is pre-allocated by the time
__rb_allocate_pages() is called. Skip si_mem_available() to avoid
wrongly failing the allocation under memory pressure.
For the same reason, unset user_thread.
Link: https://patch.msgid.link/20261013102047.1488347-2-vdonnefort@google.com
Fixes: be68d63a139b ("ring-buffer: Add ring_buffer_alloc_range()")
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
If trace buffer allocation fails during early boot (e.g. in
early_trace_init() -> allocate_trace_buffers()), tracing_disabled is
left set to 1 and the global trace array events list remains
uninitialized.
Later during boot, tracer_init_tracefs() (invoked via fs_initcall)
queues tracerfs_init_work asynchronously without checking if tracing was
disabled. When tracer_init_tracefs_work_func() runs, it calls
event_trace_init() and init_tracer_tracefs(&global_trace, NULL)
unconditionally. This leads to __find_event_file() dereferencing an
uninitialized tr->events list, causing an early boot NULL pointer
dereference Oops:
BUG: Kernel NULL pointer dereference on read at 0x00000010
Faulting instruction address: 0xc000000000485be0
Oops: Kernel access of bad area, sig: 7 [#1]
...
NIP [c000000000485be0] __find_event_file+0x70/0x3c0
LR [c000000000448874] init_tracer_tracefs+0x274/0xc80
Call Trace:
init_tracer_tracefs+0x274/0xc80
tracer_init_tracefs_work_func+0x50/0x320
process_one_work+0x1e8/0x5c0
worker_thread+0x1dc/0x3d0
kthread+0x194/0x1b0
start_kernel_thread+0x14/0x18
Check tracing_disabled at the beginning of tracer_init_tracefs_work_func()
and return immediately if tracing is disabled or failed to initialize,
preventing the crash.
Cc: stable@vger.kernel.org
Fixes: 6621a7004684 ("tracing: make tracer_init_tracefs initcall asynchronous")
Link: https://patch.msgid.link/20261009163557.88467-2-amachhiw@linux.ibm.com
Reported-by: Michal Suchánek <msuchanek@suse.de>
Closes: https://lore.kernel.org/all/arYskzbiaNzBR9MD@kunlun.suse.cz/
Signed-off-by: Amit Machhiwal <amachhiw@linux.ibm.com>
Tested-by: Michal Suchánek <msuchanek@suse.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Syzkaller triggered a WARN_ON_ONCE(!nest) warning:
WARNING: kernel/trace/ring_buffer.c:821 at ring_buffer_event_time_stamp
Call trace:
ring_buffer_event_time_stamp
hist_field_timestamp
hist_fn_call
event_hist_trigger
event_triggers_call
__event_trigger_test_discard
trace_event_buffer_commit
do_trace_event_raw_event_sched_switch
trace_event_raw_event_sched_switch
The warning is probabilistically reproducible and is related to syzkaller
executing the following commands:
echo 'prev_pid == 999999' > events/sched/sched_switch/filter
echo 'hist:keys=common_timestamp' > events/sched/sched_switch/trigger
The specific triggering process is as follows:
CPU 0 CPU 1
(context switching) (echo 'hist:keys=common_timestamp' > \
events/sched/sched_switch/trigger)
do_trace_event_raw_event_##call
trace_event_buffer_lock_reserve
if (!tr->no_filter_buffering_ref &&
trace_file->flags & EVENT_FILE_FL_FILTERED)
return entry; // return directly
event_hist_trigger_parse
hist_register_trigger
tracing_set_filter_buffering(file->tr, true);
tr->no_filter_buffering_ref++;
__trace_buffer_lock_reserve
rb_start_commit
local_inc(&cpu_buffer->committing); // skipped and not executed
trace_event_buffer_commit
__event_trigger_test_discard
event_triggers_call
event_hist_trigger
hist_fn_call
hist_field_timestamp
ring_buffer_event_time_stamp
nest = local_read(&cpu_buffer->committing);
WARN_ON_ONCE(!nest) // trigger warning
The root cause is that when an event file has a filter attached and the
hist trigger has not yet been attached, trace_event_buffer_lock_reserve()
writes the event into the per-CPU trace_buffered_event temp buffer instead
of reserving it in the ring buffer, so cpu_buffer->committing is not
incremented. When the event is later processed at commit time by a hist
trigger, hist_field_timestamp() -> ring_buffer_event_time_stamp() sees a
zero committing count, triggering WARN_ON_ONCE(!nest).
tracing_event_time_stamp() was added by commit d8279bfc5e959 ("tracing:
Add tracing_event_time_stamp() API") for exactly this case: if the event
is the per-CPU trace_buffered_event, it returns the current ring buffer
timestamp. But it never gained a caller. Use it in hist_field_timestamp(),
so that a hist trigger that races onto a buffered event records the
current time, which is only marginally later than when the event was
recorded, instead of warning and recording a bogus timestamp.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260929104015.1689921-1-wutengda@huaweicloud.com
Fixes: b94bc80df648 ("tracing: Use a no_filter_buffering_ref to stop using the filter buffer")
Signed-off-by: Tengda Wu <wutengda@huaweicloud.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When tr->allocated_snapshot is true, resize_buffer_duplicate_size() is
called to update the snapshot buffer size. However, its return value was
not checked. In case of a failed resize this could lead next snapshot
being incomplete.
Add resize_buffer_duplicate_size() result check and return error if
resize fails.
Compile tested only.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260928152634.4354-1-d.kandybka@gmail.com
Fixes: 180e4e390978 ("tracing: Add snapshot refcount")
Signed-off-by: Dmitry Kandybka <d.kandybka@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The mmap reader can warn when it catches up with a writer that is still
committing events. rb_get_reader_page() returns NULL in this case, but
ring_buffer_map_get_reader() treats that result as an error.
The initial rb_per_cpu_empty() check filters out an empty buffer, but it
does not guarantee that the next reader page has committed data. Writers
do not take reader_lock and can advance commit_page before publishing the
committed length on the new page. rb_get_reader_page() can swap to that
page, catch up with the writer and return NULL.
Handle NULL through the existing no-data exit without warning. Remove the
caller's reader_page == commit_page check as well, since the reader-page
helper already handles that case. The existing metadata update and return
path remain in place.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260924084946.20104-1-krystianmkaniewski@gmail.com
Fixes: 117c39200d9d ("ring-buffer: Introducing ring-buffer mapping functions")
Assisted-by: Gemini:gemini-3.8-flash syzbot
Reported-by: syzbot+de3d7f9bcc9212f3fae1@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=de3d7f9bcc9212f3fae1
Link: https://syzkaller.appspot.com/ai_job?id=488adc18-a10e-42d2-a205-25327c38837f
Signed-off-by: Krystian Kaniewski <krystianmkaniewski@gmail.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
simple_ring_buffer relies on headers that end up including
<asm/thread_info.h>. On sparc64, this emits the undefined symbol
current_thread_info_reg. Allow this symbol.
Link: https://patch.msgid.link/20260923074213.3077182-4-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
simple_ring_buffer uses local_set() and cmpxchg() on unsigned long. Add
them to undefsyms_base.c to extend the allowlist for the parisc
architecture.
Link: https://patch.msgid.link/20260923074213.3077182-3-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Sparc64 emits empty symmbol names. Skips them when establishing the
allowlist and when extracing undefined symbols.
Link: https://patch.msgid.link/20260923074213.3077182-2-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- A task that skipped the BPF scheduler on its way back to its CPU's
local queue could stall there until the watchdog fired, as the CPU
had already decided to go idle.
- Use-after-free of a destroyed user dispatch queue by a deferred
re-enqueue request.
- A task could be moved off a dispatch queue while the dispatching CPU
was still running ops.dequeue() on it.
- Bypass mode mishandled a sub-scheduler bypass overlapping an
ancestor's, as across a system suspend. Descendants could leave
bypass mid-teardown and tasks could be stranded on bypass queues.
- The idle CPU selection helper could return a CPU outside the allowed
CPUs of a pinned or migration-disabled task.
* tag 'sched_ext-for-7.3-rc7-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Wait for SCX_OPSS_DISPATCHING before moving tasks off DSQs
sched_ext: Rename scx_reenq_wait_dispatching() to scx_wait_dispatching()
sched_ext: idle: Respect task affinity in restricted CPU selection
sched_ext: Fix the bypass dispatch claim for schedulers bypassed by an ancestor
sched_ext: Propagate every bypass depth change to the descendants
sched_ext: Fix use-after-free of a destroyed user DSQ by a deferred reenqueue
sched_ext: Fix stall when a task enqueued with SCX_ENQ_LAST lands back on its CPU's local DSQ
|
|
Array traversal logic in btf_struct_access() starts from reg_smin().
Update comment to match the logic.
Fixes: 0892ef2a7ef3 ("bpf: Representation for intervals with steps")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261013-scev-minimal-v3-followups-v2-3-6d9d61965486@gmail.com
|
|
The following types can be returned by the btf_prepare_func_args():
- ARG_PTR_TO_CTX
- ARG_PTR_TO_BTF_ID (+ flags)
- ARG_PTR_TO_MEM (+ flags)
- ARG_PTR_TO_ARENA
- ARG_PTR_TO_DYNPTR
- ARG_SCALAR
Of these, valid programs can pass frame pointer only as
ARG_PTR_TO_{MEM,BTF_ID,DYNPTR}. Additionally, pointer arithmetic
allows the following trick:
0: (bf) r1 = r10
1: (07) r1 += -8
2: (1f) r1 -= r10
3: (85) call pc+1 // r1=fp0 ?
4: (95) exit
This commit changes bpf_global_subprog_stack_access_bytes() to handle
ARG_SCALAR and ARG_PTR_TO_DYNPTR, thus addressing the full set of
possible returned values.
Fixes: 8212c1c2986d ("bpf: Derive global subprog stack liveness from BTF signatures")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261013-scev-minimal-v3-followups-v2-1-6d9d61965486@gmail.com
|
|
On a persistent ring buffer, rb_meta_subbuf_idx() can return an index one
sub-buffer too high. rb_setup_ids_meta_page() then fills subbuf_ids[]
shifted by one and leaves subbuf_ids[0] NULL, which __rb_map_vma()
dereferences when userspace maps trace_pipe_raw. rb_update_meta_reader()
writes two such indices into meta->buffers[], which rb_cpu_meta_valid()
would reject on the next boot, discarding the previous boot's trace.
A CPU's sub-buffer array starts after its meta header and the nr_subbufs
integers that follow, aligned up to a sub-buffer. rb_meta_subbuf_idx()
inverts that calculation but skips only the integers, not the header, so
its base is one sub-buffer low whenever the omitted header size crosses
the alignment boundary.
Invert the calculation with rb_subbufs_from_meta() rather than repeating
it. rb_range_buffer() maps an index back to an address through the same
helper, so the two directions can no longer disagree.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260918071507.1019251-1-donggeunyoo.kernel@gmail.com
Fixes: b14d032973d4 ("ring-buffer: Add ring_buffer_meta data")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The .log2 key modifier files a value under its base-2 order:
# echo 'hist:keys=bytes_req.log2' > events/kmem/kmalloc/trigger
hist_field_log2() computes that as ilog2(roundup_pow_of_two(val)). The
out-of-line form of roundup_pow_of_two() is
return 1UL << fls_long(n - 1);
which shifts by 64 on a 64-bit kernel both when n is 0, where n - 1
wraps to ULONG_MAX, and when n is above 2^63. Both are ordinary keys,
because the field fetch hands every value over as a u64: a syscall
argument of 0 reaches the first, and one above 2^63 reaches the second.
Keying sys_enter_lseek on offset.log2 and passing 0, 1, 2, 3, 1000,
2^63+1 and U64_MAX gives
{ offset: ~ 2^1 } hitcount: 1
{ offset: ~ 2^10 } hitcount: 1
{ offset: ~ 2^2 } hitcount: 1
{ offset: ~ 2^0 } hitcount: 4
Keys above 2^63 land in the lowest bucket. With CONFIG_UBSAN_SHIFT=y
the shift is also reported:
UBSAN: shift-out-of-bounds in include/linux/log2.h:57:13
shift exponent 64 is too large for 64-bit type 'long unsigned int'
__ubsan_handle_shift_out_of_bounds.cold+0xdd/0x1cb
hist_fn_call.cold+0x8b/0xd9
event_hist_trigger+0x1cc/0x790
ftrace_syscall_enter+0x197/0x360
do_syscall_64+0x402/0x4b0
Use order_base_2(), which is ilog2(n - 1) + 1 for n > 1 and 0 below
that. It returns what the old expression returned on every input the
old one was defined for. Where it was not, 0 stays in bucket 0, which
is where x86_64 had been putting it, and the keys above 2^63 move into
bucket 64.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260916002735.808520-1-donggeunyoo.kernel@gmail.com
Fixes: 4b94f5b7b4a5 ("tracing: Add hist trigger 'log2' modifier")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The per_cpu/cpuN/buffer_meta files are created for every possible CPU,
but buffer->buffers[cpu] is only allocated once that CPU comes online.
Reading the file of a CPU that never came online dereferences a NULL
cpu_buffer in rbm_start().
Fail the open with -ENODEV when the CPU has no buffer, as the other
per CPU ring buffer interfaces do.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914133506.4865-3-devnexen@gmail.com
Fixes: 950032ffcee7 ("ring-buffer: Add output of ring buffer meta page")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260913134906.6E8A01F000FF@smtp.kernel.org/
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
tracing_buffer_meta_open() drops the trace_array reference with
__trace_array_put() when ring_buffer_meta_seq_init() fails, without
holding trace_types_lock. Use trace_array_put() like every other open
error path.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914133506.4865-2-devnexen@gmail.com
Fixes: 950032ffcee7 ("ring-buffer: Add output of ring buffer meta page")
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
In event_hist_trigger_parse() where it needs to create actions like
"onmatch", it calls:
event_hist_trigger_parse() {
create_actions() {
action_create() {
trace_action_create() {
trace_action_create_field_var() {
create_field_var_hist()
Where create_field_var_hist() does a loop on the hist_data
representing the keys. The issue is, if the keys uses one of the
pseudo field types (like common_cpu), the hist_data field element
will have NULL for its field member causing a NULL pointer
dereference when accessing the key_field->field->name.
# echo 'hist:keys=common_cpu:ts0=common_timestamp.usecs' > \
events/sched/sched_waking/trigger
# echo 'my_synth u64 lat; int prio' > synthetic_events
# echo 'hist:keys=common_cpu:wakeup_lat=common_timestamp.usecs-$ts0:\
onmatch(sched.sched_waking).my_synth($wakeup_lat,prio)' > \
events/sched/sched_switch/trigger
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
RIP: 0010:create_field_var_hist+0x771/0x1380
Call Trace:
trace_action_create_field_var+0x360/0x600
trace_action_create+0x393/0xe80
event_hist_trigger_parse+0x3e47/0x69e0
trigger_process_regex+0x1a6/0x250
event_trigger_write+0xce/0x160
Instead of accessing it directly, use the proper handler
expr_field_str() to get the name.
The three previous patches are prerequisites: without them a .buckets
key and a common_stacktrace key each render into a command that cannot
be parsed back.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914053443.981201-5-donggeunyoo.kernel@gmail.com
Fixes: 02205a6752f2 ("tracing: Add support for 'field variables'")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Commit ee172227d0dc ("sched_ext: Delegate proxy donor admission to BPF
schedulers") makes put_prev_task_scx() pass a retained proxy donor to
ops.enqueue() with SCX_ENQ_BLOCKED.
Some of these puts are only proxy bookkeeping; proxy_resched_idle()
drops the rq's donor reference before switching to idle when:
- find_proxy_task() cannot run the mutex owner yet and retries through
idle (e.g., the owner is on a remote CPU),
- proxy_migrate_task() detaches the donor before migration,
- proxy_deactivate() blocks a task in the chain whose owner cannot run.
In the first case, BPF has already selected a donor with slice left.
Returning it to ops.enqueue() forces BPF to dispatch it again before
proxy resolution can continue. In the other two cases, the donor is
typically deactivated right after ops.enqueue(), undoing any placement
BPF makes.
As also discussed at the sched_ext microconference at Linux Plumbers
2026, these ops.enqueue/dequeue() invocations do not represent any
meaningful scheduling events, so we should avoid triggering them.
A more reasonable semantic is to keep a donor with slice left at the
head of the local DSQ, so the next pick can resolve its owner, or
deactivation can remove it without an unnecessary BPF handoff.
An IMMED donor can stay there only for a bookkeeping put during proxy
resolution. Mark a blocked pick as awaiting resolution with
SCX_RQ_PROXY_PENDING to distinguish that put from a real preemption and
re-insert the donor with SCX_ENQ_IMMED, so that it only needs
SCX_CAP_ENQ_IMMED rather than SCX_CAP_ENQ. A preempted IMMED donor still
returns to BPF. If a higher priority class takes the CPU once proxy
resolution completes, schedule a local reenqueue, so that a parked IMMED
donor returns to BPF instead of lingering on the local DSQ. Update the
SCX_ENQ_IMMED and SCX_ENQ_BLOCKED documentation accordingly.
Moreover, a donor that has run out of slice now follows the regular put
path, so it can receive SCX_ENQ_LAST when the CPU is about to go idle
and the scheduler needs to arrange a follow-up scheduling event. This
does not apply to bookkeeping puts, where the switch to idle is only
temporary. The blocked-donor specific sanity checks go away together
with the special case.
When a retained donor is removed ahead of a scheduler ownership or
policy change, mark the proxy reset with SCX_RQ_PROXY_BLOCKING and skip
reenqueueing the active donor: sched_proxy_block_task() will dequeue it
next. Report it as not runnable to ops.stopping(), as a regular dequeue
of the current donor does. This avoids a transient ops.enqueue/dequeue()
pair and a false ENQ_LAST warning.
With this, the sched_ext core absorbs the transient proxy-bookkeeping
puts and the BPF scheduler only sees a blocked donor when there is an
actual placement decision to make.
Fixes: ee172227d0dc ("sched_ext: Delegate proxy donor admission to BPF schedulers")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
With nullfs as the real rootfs userspace can pivot_root() away from the
initramfs and unmount it:
chdir(rootfs);
pivot_root(".", ".");
umount2(".", MNT_DETACH);
That's what systemd does and the initramfs is supposed to go away at
that point together with all the memory it uses. But on every machine
with more than one possible cpu the initramfs superblock stays around
forever (cf. [1]). With ramfs as rootfs the unpacked initramfs shows up
as Unevictable in /proc/meminfo, with tmpfs as Shmem, and it is never
freed. In the reports that's between 70MB and 150MB.
The idle threads of the secondary cpus are created by fork_idle() from
smp_init(). That runs in kernel_init(), i.e., in PID 1, after
init_userspace_fs() moved PID 1 from nullfs into the initramfs.
fork_idle() doesn't pass CLONE_FS so each idle thread gets a private
copy of PID 1's fs_struct and with it a reference to the initramfs as
root and as pwd.
pivot_root() relies on chroot_fs_refs() to move everyone whose root or
pwd is the old root over to the new root. Idle threads aren't on the
task list though so chroot_fs_refs() never finds them. They never exit
either. So they keep the initramfs mount and with it the superblock
alive for good. I overlooked this when I added nullfs. Before that the
rootfs could never be unmounted anyway so it didn't matter what pinned
it.
Idle threads are kernel threads and init_task, the idle thread of the
boot cpu, already uses init_fs. So let the idle threads of the other
cpus share init_fs as well instead of copying PID 1's filesystem state.
Note that kernels before 7.3 need a different backport. There PID 1
still shares init_fs with all kthreads so passing CLONE_FS in
fork_idle() is enough. They also have a second pin: devtmpfsd unshares
a copy of the initial mount namespace, initramfs mount included. That
was fixed in commit 66d2faeccbff ("devtmpfs: create private mount
namespace").
Reproducer, as /init in an initramfs together with a 128MB ballast
file, booted with root=/dev/sda to get ramfs and with -smp 4:
// SPDX-License-Identifier: GPL-2.0
/* initramfs /init: switch root like systemd, check the old root */
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mount.h>
#include <sys/reboot.h>
#include <sys/sendfile.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/sysmacros.h>
#include <unistd.h>
static void meminfo(void)
{
char line[256];
FILE *f = fopen("/proc/meminfo", "re");
while (f && fgets(line, sizeof(line), f))
if (!strncmp(line, "Unevictable:", 12) ||
!strncmp(line, "Shmem:", 6))
fputs(line, stdout);
if (f)
fclose(f);
}
int main(int argc, char *argv[])
{
struct stat st;
char dev[32], buf[32];
unsigned int maj, min;
int fd, in, out;
long r;
if (argc > 1) {
sleep(2);
meminfo();
/* ustat() only finds live superblocks */
r = syscall(SYS_ustat, (unsigned)atoi(argv[1]), buf);
printf("initramfs superblock %s\n", r ? "freed" : "pinned");
fflush(stdout);
sleep(1);
reboot(RB_POWER_OFF);
}
mount("proc", "/proc", "proc", 0, NULL);
mount("devtmpfs", "/dev", "devtmpfs", 0, NULL);
/* drop the fds console_on_rootfs() opened in the initramfs */
fd = open("/dev/console", O_RDWR);
dup2(fd, 0);
dup2(fd, 1);
dup2(fd, 2);
close(fd);
setvbuf(stdout, NULL, _IOLBF, 0);
meminfo();
stat("/", &st);
maj = major(st.st_dev);
min = minor(st.st_dev);
/* new_encode_dev(), what ustat() expects */
snprintf(dev, sizeof(dev), "%u",
(min & 0xff) | (maj << 8) | ((min & ~0xffu) << 12));
mkdir("/sysroot", 0755);
mount("tmpfs", "/sysroot", "tmpfs", 0, NULL);
mkdir("/sysroot/proc", 0755);
mkdir("/sysroot/dev", 0755);
in = open("/init", O_RDONLY);
out = open("/sysroot/init", O_WRONLY | O_CREAT, 0755);
fstat(in, &st);
sendfile(out, in, NULL, st.st_size);
close(in);
close(out);
mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL);
mount("/proc", "/sysroot/proc", NULL, MS_BIND | MS_REC, NULL);
mount("/dev", "/sysroot/dev", NULL, MS_BIND | MS_REC, NULL);
chdir("/sysroot");
if (syscall(SYS_pivot_root, ".", ".") || umount2(".", MNT_DETACH))
perror("pivot_root");
execl("/init", "/init", dev, NULL);
return 1;
}
Before:
Unevictable: 131812 kB
Shmem: 0 kB
Unevictable: 131812 kB
Shmem: 792 kB
initramfs superblock pinned
After:
Unevictable: 131812 kB
Shmem: 0 kB
Unevictable: 0 kB
Shmem: 792 kB
initramfs superblock freed
With -smp 1 the superblock is freed without this patch as well.
[1]: https://github.com/systemd/systemd/issues/43911
Fixes: 576ee5dfd459 ("fs: add immutable rootfs")
Link: https://patch.msgid.link/20261013-work-fs-idle_fs-v1-2-2063d1bcb5aa@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Turn the umh boolean into a flags argument. Idle threads need their own
case in copy_fs() as well and that's nicer than a second boolean.
No functional changes.
Link: https://patch.msgid.link/20261013-work-fs-idle_fs-v1-1-2063d1bcb5aa@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
When the init of a pid namespace exits, zap_pid_ns_processes() waits
until all other pids in the namespace are freed. A tracer outside the
namespace that does not reap its traced zombies keeps those pids
allocated, so this wait has no upper bound, and while it lasts every
suspend attempt fails:
Freezing user space processes failed after 20.006 seconds (2 tasks refusing to freeze, wq_busy=0):
Neither wait is freezable. The first loop sleeps in do_wait(), which
returns -ERESTARTSYS on the freezer's fake signal; the loop then clears
TIF_SIGPENDING and waits again without ever freezing. The final loop
sleeps in TASK_INTERRUPTIBLE, and since nothing clears TIF_SIGPENDING
there, schedule() returns immediately after the first freeze attempt
and the task spins until the wait ends.
Sleep in TASK_INTERRUPTIBLE | TASK_FREEZABLE in do_wait(), so that the
freezer can freeze tasks waiting there directly, and in TASK_IDLE |
TASK_FREEZABLE in the final loop, like coredump_task_exit() does.
This was hit with Chromium-based browsers: crashpad ptrace-attaches the
threads of a crashed renderer, which is a pid namespace init, and does
not reap those killed during the dump.
Fixes: 3eb07c8c8adb ("pid namespaces: destroy pid namespace on init's death")
Fixes: b9a985db9896 ("pid_ns: Sleep in TASK_INTERRUPTIBLE in zap_pid_ns_processes")
Cc: stable@vger.kernel.org # 6.1+
Suggested-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Aviv Vaknin <vaknins33@gmail.com>
Link: https://patch.msgid.link/20261002202820.3433318-1-vaknins33@gmail.com
Acked-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
There is a race between watch_queue_set_size() and post_one_notification().
watch_queue_set_size() publishes wqueue->notes, notes_bitmap, nr_pages and
nr_notes with plain stores, and post_one_notification() reads them with
plain loads under a different lock, so nothing orders the four stores
against the poster's loads: depending on the build, notes can be stored
last (gcc 14 on x86-64 does this). Since keyctl_watch_key() lets a watch
attach to a queue before it is sized, post_one_notification() can already
be running on another CPU at that moment: it reads nr_notes > 0, passes
its bounds check, and then reads notes while it is still NULL,
dereferencing notes[0].
An unprivileged user triggers this reliably by watching one of its own
keys through a notification pipe and racing ioctl(IOC_WATCH_QUEUE_SET_SIZE)
on that pipe against keyctl(KEYCTL_SETPERM) on the key from another CPU,
triggering:
BUG: kernel NULL pointer dereference, address: 0000000000000000
RIP: 0010:post_one_notification.isra.0+0xa2/0x1e0
Call Trace:
__post_watch_notification+0x148/0x180
keyctl_setperm_key+0x119/0x130
do_syscall_64+0x108/0x4c0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Kernel panic - not syncing: Fatal exception in interrupt
Fix this by publishing nr_notes last with smp_store_release(), and by
reading it once with smp_load_acquire() in post_one_notification() before
the bitmap and the page array are touched. A poster that observes a
non-zero nr_notes then also observes the pool behind it; one that observes
zero drops the notification as before.
Fixes: c73be61cede5 ("pipe: Add general notification queue support")
Reported-by: Dae R. Jeong <threeearcat@gmail.com>
Closes: https://lore.kernel.org/all/ZT-S8Q7tyutcvu_q@dragonet/
Reported-by: AutonomousCodeSecurity@microsoft.com
Suggested-by: Dae R. Jeong <threeearcat@gmail.com>
Signed-off-by: Cen Zhang (Microsoft) <cenzhang@linux.microsoft.com>
Link: https://patch.msgid.link/20261009224715.46121-1-cenzhang@linux.microsoft.com
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The current value of sg_policy->tunables->rate_limit_us is used in
sugov_start() without locking to update sg_policy->freq_update_delay_ns,
so if rate_limit_us_store() runs in parallel with it, the rate limit
update may go unnoticed and the new freq_update_delay_ns value may be
stale until the rate limit is updated again.
To avoid that, acquire the update_lock mutex of the rate limit sysfs
attribute, that is held by rate_limit_us_store() during rate limit
updates, around the freq_update_delay_ns in sugov_start().
Fixes: 9bdcb44e391d ("cpufreq: schedutil: New governor based on scheduler utilization data")
Reviewed-by: Zhongqiu Han <zhongqiu.han@oss.qualcomm.com>
Signed-off-by: Hui Su <sh_def@163.com>
Cc: All applicable <stable@vger.kernel.org>
[ rjw: Subject adjustment, new changelog, edits of the new comment ]
[ rjw: Add empty code line after guard()() ]
Link: https://patch.msgid.link/20261013034235.2678177-1-sh_def@163.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
|
|
wq_worker_running() increments pool->nr_running with only preemption
disabled, while kick_pool_pick() reads it via need_more_worker() under
pool->lock, so the two can run concurrently on different CPUs. KCSAN
reports:
BUG: KCSAN: data-race in kick_pool_pick / wq_worker_running
write to 0xffff888059c2af24 of 4 bytes by task 3007 on cpu 0:
wq_worker_running+0x114/0x130
schedule_timeout+0x16e/0x180
io_schedule_timeout+0x4c/0x70
submit_bio_wait+0xd7/0x110
blkdev_issue_flush+0x4c/0x70
ext4_sync_file+0x60d/0x6c0
aio_fsync_work+0x7f/0x270
process_one_work+0x4ae/0xa00
read to 0xffff888059c2af24 of 4 bytes by task 3001 on cpu 1:
kick_pool_pick+0x7a/0x3a0
__queue_work+0x72d/0xcf0
queue_work_on+0x9d/0xb0
schedule_on_each_cpu+0x11a/0x2d0
vmstat_refresh+0x23/0x140
proc_sys_call_handler+0x2b9/0x4f0
The race is benign. The increment in wq_worker_running() is the only
lockless write to nr_running; decrements and the reset are done under
pool->lock, so a reader holding the lock sees either the old or the new
count. A stale 0 costs at most a spurious wakeup of an idle worker. A
nonzero count is correct because the incrementing worker is running and
re-checks the worklist under pool->lock before it stops counting. The
increment can also go 1 -> 2, e.g. when a worker sleeping in a work item
wakes up after kick_pool() woke another worker.
The lockless increment is part of how nr_running is designed to work,
as the comment above the field documents. Mark the field __data_racy
so that every reader is covered, not just need_more_worker().
Reported-by: syzbot+8922e6b8343f70be92e8@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=8922e6b8343f70be92e8
Signed-off-by: Nguyen Ngoc Thang <ngocthang2710.1999@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
A FUT_OFF_MMSHARED futex is a shared futex that refers to an MM.
It is possible for a process to wait on a shared futex with a different MM
because a FUT_OFF_INODE waiter can be requeued onto a FUT_OFF_MMSHARED
futex by another process.
This can cause FUT_OFF_MMSHARED waiters on a freed MM to consume
wakeups intended for a newly allocated MM at the same address.
Fix it by keying FUT_OFF_MMSHARED using a unique 64-bit per-MM ID.
Leave private futexes as before to avoid influencing the performance of the
hotpath.
(Multi-threaded processes typically implicitly use FUT_OFF_MMSHARED by
setting clear_child_tid such that it points into anonymous memory, which
causes mm_release() in a multi-threaded mm to perform FUTEX_WAKE.)
Fixes: 222993395ed3 ("futex: Remove pointless mmgrap() + mmdrop()")
Closes: https://lore.kernel.org/r/CAG48ez0dLBpc3QtbAhMMVNHwHL94iHh2G+h-=BVFR4dDuzZr1g@mail.gmail.com/
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260929-futex-mmshared-fix-v1-1-262b1ffeb3d0@google.com
Cc: stable@vger.kernel.org
|
|
Paul reported recent RCU changes exposed a lost resched where CPU idles
with a rcutorture thread waiting while being completely runnable.
Dumping the rq state at the time of idling with runnable tasks showed
the following:
rq->next_class = &fair_sched_class
rq->nr_running = 1
rq->cfs.h_nr_queued = 0
rq->rt.rt_nr_running = 1
Although the waiting task was an RT task, "rq->next_class" did not point
to rt_sched_class and was stuck at the lower priority fair_sched_class.
Inspecting sched_change_end() revealed the following race with
__sched_setscheduler() for queued, but non-running tasks:
CPU0 CPU1
==== ====
__schedule()
/*
* rq->donor is a fair task that blocks.
* rq->next_class = &fair_sched_class
*/
rq_lock(rq0)
pick_next_task()
/* nr_running = 0 */
pick_task_fair()
sched_balance_newidle()
raw_spin_rq_unlock(rq0)
rq_lock(rq0)
activate_task(rq0, p)
/* p is a fair task */
rq0->nr_running++;
rq0->cfs.h_nr_queued++;
wakeup_preempt(rq, p)
rq->next_class = &fair_sched_class
resched_curr()
rq_unlock(rq0)
__sched_setscheduler(p)
/* p is moved to SCHED_RR */
task_rq_lock(p)
sched_change_begin()
rq0->cfs.h_nr_queued--;
rq0->rt.rt_nr_running++;
sched_change_end()
/*
* p->on_rq = 1 (ctx->queued)
* but p is not rq->donor
* (!ctx->running) hence
* resched_curr() is and
* rq->next_class upgrade is
* skipped.
*/
task_rq_unlock()
raw_spin_rq_lock(rq0)
/*
* rq->nr_running = 1
* rq->cfs.h_nr_queued = 0
* rq->rt.rt_nr_running = 1
* rq->next_class = &fair_sched_class
*/
if (rq->cfs.h_nr_queued && ...)
/* Above is false */
if (rq_modified_above(this_rq, &fair_sched_class))
/* Also false since rq->next_class is still fair */
/*
* sched_balance_newidle() is considered to
* have failed to pull any fair tasks and no
* other wakeups were detected.
*/
return NULL; /* pick_task_fair() */
pick_task_idle()
return rq->idle;
next = rq->idle;
/* Preemption indicators are lost */
clear_tsk_need_resched(prev);
clear_preempt_need_resched();
rq->curr = rq->idle;
!!! rq->idle switches in with rq->running == 1 but !need_resched() !!!
In simple words, enqueue, followed by a class change of the newly queued
task goes unnoticed in rq->next_class indicator and
sched_balance_newidle() sees no fair task, nor any higher priority task
enqueued.
The above sequence will lead to CPU idling with a runnable task on the
CPU.
Call wakeup_preempt() for ctx->queued cases in sched_change_end() to
re-evaluate rq->next_class after class change (ENQUEUE_CLASS) and set
the NEED_RESCHED indicator accordingly.
Newidle balance will correctly see an enqueue of a higher priority
task if a queued task was upgraded when the rq_lock() was dropped and
will retry the pick.
Closes: https://lore.kernel.org/lkml/8fa7f795-518c-4507-b684-60b0c1cac2c5@paulmck-laptop/
Fixes: 704069649b5b ("sched/core: Rework sched_class::wakeup_preempt() and rq_modified_*()")
Reported-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Phil Auld <pauld@redhat.com>
Link: https://patch.msgid.link/20261013152455.9039-1-kprateek.nayak@amd.com
|
|
Mapping one MMIO dma-buf for many RDMA importers overflows dma-debug's
cacheline overlap counter and triggers a warning. DMA_ATTR_MMIO mappings
must never be CPU-cacheable, so CPU cacheline overlap tracking does not
apply to them.
Skip MMIO entries in both cacheline insertion and removal. Keep them in
the DMA entry hash so mapping and unmapping checks still run.
The reported warning was:
------------[ cut here ]------------
DMA-API: exceeded 7 overlapping mappings of cacheline 0x000000e040000400
WARNING: kernel/dma/debug.c:465 at add_dma_entry+0x3b5/0x470, CPU#58: dtr/5264
Modules linked in: xt_conntrack xt_MASQUERADE ip6table_nat ip6table_filter ip6_tables iptable_nat nf_nat xt_addrtype rpcsec_gss_krb5 krb5 auth_rpcgss oid_registry overlay rpcrdma rdma_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm nvidia(O) mlx5_fwctl nvidia_drm(O) mlx5_ib ib_uverbs_support ib_core zram zsmalloc fuse mlx5_core
CPU: 58 UID: 0 PID: 5264 Comm: dtr Tainted: G O J 7.3.0-rc6_for_upstream_debug_2026_10_08_13_46 #1.fc40 PREEMPTLAZY
Tainted: [O]=OOT_MODULE, [J]=FWCTL
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
RIP: 0010:add_dma_entry+0x3bf/0x470
Code: ac e8 f5 89 6e 00 e9 47 fe ff ff 41 f7 c6 00 18 00 00 75 9f 83 fb 07 7e 9a 48 8d 3d eb 02 5f 04 be 07 00 00 00 48 8d 54 24 30 <67> 48 0f b9 3a eb 82 0f 0b 48 c7 c7 80 d3 2a 85 4c 89 e6 e8 a9 1f
RSP: 0018:ff110001bcb7f230 EFLAGS: 00010002
RAX: 0000000000000006 RBX: 0000000000000008 RCX: 0000000000000000
RDX: ff110001bcb7f260 RSI: 0000000000000007 RDI: ffffffff85cfd070
RBP: 000000e040000400 R08: 0000000000000000 R09: ffe21c0044b8abf5
R10: ff11000225c55faf R11: 0000000000000000 R12: 0000000000000246
R13: 1fe220003796fe48 R14: 0000000000000400 R15: 00000000ffffffff
FS: 00007fa8f91e6b80(0000) GS:ff11000eb81cc000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa8f5ca80f0 CR3: 0000000164871004 CR4: 0000000000373eb0
Call Trace:
<TASK>
? check_sync+0x2070/0x2070
? find_held_lock+0x2b/0x80
? debug_dma_map_phys+0x588/0x840
debug_dma_map_phys+0x267/0x840
? dma_debug_add_bus+0xd0/0xd0
? __sg_alloc_table+0x2b7/0x4d0
dma_map_phys+0x10a/0x330
? sg_miter_start+0x130/0x130
dma_buf_phys_vec_to_sgt+0x565/0xed0
uverbs_dmabuf_map+0x158/0x340 [ib_uverbs]
? uverbs_dmabuf_unpin+0x10/0x10 [ib_uverbs]
dma_buf_map_attachment+0x138/0x690
ib_umem_dmabuf_map_pages+0x249/0xc90 [ib_core]
pagefault_dmabuf_mr+0x172/0xf80 [mlx5_ib]
? do_raw_spin_unlock+0x54/0x210
mlx5_ib_init_dmabuf_mr+0x13e/0x200 [mlx5_ib]
[... '?' frames and user-space registers omitted ...]
reg_user_mr_dmabuf+0x3bf/0x760 [mlx5_ib]
mlx5_ib_reg_user_mr_dmabuf+0x2dd/0x13b0 [mlx5_ib]
ib_uverbs_handler_UVERBS_METHOD_REG_MR+0x76d/0xd90 [ib_uverbs]
ib_uverbs_cmd_verbs+0x2337/0x31c0 [ib_uverbs]
ib_uverbs_ioctl+0x1c7/0x350 [ib_uverbs]
__x64_sys_ioctl+0x72c/0x11f0
do_syscall_64+0xa1/0x4f0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
---[ end trace 0000000000000000 ]---
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Fixes: 3aa31a8bb11e ("dma-buf: provide phys_vec to scatter-gather mapping routine")
Link: https://lore.kernel.org/r/20261013-dma-debug-fixes-v1-1-1ad1f471cc61@nvidia.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Use the function name in the kerneldoc description.
Fixes: c83e3d291243 ("braille: nbcon: Allow to use a serial console with NBCON API as Braille console")
Signed-off-by: Muhammad Mahdi Amirpour <ampr.mhd@gmail.com>
Reviewed-by: John Ogness <john.ogness@linutronix.de>
Link: https://patch.msgid.link/20261013151300.75966-1-ampr.mhd@gmail.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
emergency_restart()
When the system is going down a normal way, the NBCON kthreads are
flushed and stopped properly via a syscore shutdown callback, see
printk_kthreads_shutdown().
One missing piece is emergency_restart() where the kthreads are not
stopped. It might be called from several non-panic code paths,
e.g. sysrq-o, kdb_reboot(), or some watchdogs.
Fix it by explicitly entering the emergency (atomic) mode and trying
to flush the consoles there.
Note that printk_trigger_flush() must be used instead of pr_flush()
because the later is not safe in an atomic context.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/all/202608061008.48a1e76e-lkp@intel.com/
Suggested-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: John Ogness <john.ogness@linutronix.de>
Link: https://patch.msgid.link/20261013154141.94016-1-pmladek@suse.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
try_add_tainted_module() allocates the tracking record without zeroing
it and initializes the fields in this order:
mod_taint = kmalloc_obj(*mod_taint);
strscpy(mod_taint->name, mod->name, MODULE_NAME_LEN);
mod_taint->taints = mod->taints;
list_add_rcu(&mod_taint->list, &unloaded_tainted_modules);
mod_taint->count = 1;
The record is published by list_add_rcu() before the count field is
written. The readers, unloaded_tainted_modules_seq_show() (debugfs,
under rcu_read_lock()) and print_unloaded_tainted_modules() (panic
path, under guard(rcu)), traverse the list under RCU and read
mod_taint->count. module_mutex, which the writer holds, does not
exclude RCU readers.
A reader that observes the new node between the list_add_rcu() and
the count store reads an uninitialized u64 left over from previous
slab use. This is a data race and leaks kernel heap residue through
the debugfs file and the panic message.
The race window is two adjacent instructions; it is reachable when a
tainted module is unloaded (rmmod) concurrently with a read of
/sys/kernel/debug/module/unloaded_tainted, or with the module list
printout of an oops on another CPU.
Fix this by initializing count before the record is published.
Found during a static-analysis-assisted audit: candidate sites were
surfaced by a Coccinelle rule, and the reader/writer paths were
confirmed by LLM-assisted (GLM-5.3) code inspection. The issue was
not observed at runtime; it is identified by code inspection only.
Fixes: 99bd9956551b ("module: Introduce module unload taint tracking")
Signed-off-by: Binbin Deng <18983559317@163.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
Commit 6164be01f179 ("watchdog/perf: optimize bytes copied and remove
manual NUL-termination") swapped strscpy(buf, str, sizeof(buf)) plus a
manual buf[len] = 0 for strscpy(buf, str, len), and that count is one
short. strscpy() keeps the last byte of the destination for the NUL, so
the config loses its final digit, nmi_watchdog=r300,panic for example ends
up with buf = "30" and arms the raw event with the wrong config.
The empty case falls over too, strscpy() with a zero count writes nothing
at all, so nmi_watchdog=r,1 leaves buf uninitialized and kstrtoull() reads
whatever stack garbage is sitting there.
The old code was safe on both by accident, it filled the whole buffer
first and buf[len] = 0 then stomped the comma position, so the worst you
got was a truncated parse failure.
Pass len + 1 so the copy includes the character the NUL replaces, and
reject len >= sizeof(buf) like it was before the optimization, which also
turns an empty config back into a clean parse failure.
Link: https://lore.kernel.org/20261003110200.3-brads@mainlining.org
Fixes: 6164be01f179 ("watchdog/perf: optimize bytes copied and remove manual NUL-termination")
Signed-off-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Cc: Thorsten Blum <blum@kernel.org>
Cc: Thomas Gleixner <tglx@kernel.org>
|
|
The KCOV interrupt selftest enables KCOV_MODE_TRACE_PC without a coverage
area so that spurious coverage causes a fault. If the fault path is
instrumented, the coverage callback faults recursively. Observed failure
modes include a stack overflow on x86_64 and arm32 getting stuck in abort
handling, neither of which identifies the original coverage event.
The recursive failure is not new, but commit 9a79524d1420 ("kcov: use
WRITE_ONCE() for selftest mode stores") made the selftest effective on
configurations where the compiler had previously removed the mode store,
exposing it more broadly.
Replace the open-coded kcov_mode stores with kcov_start() and kcov_stop(),
passing a 16-word buffer that records up to 15 spurious PCs, with the
first word holding the count. Keep KCOV enabled for the original 300 ms
window, then disable it, print all recorded PCs and panic if any were
recorded. Once the buffer fills, further PCs are discarded by the
existing coverage callback. This preserves the selftest's hard failure
while avoiding the recursive fault path.
Add decanonicalize_ip() as the inverse of canonicalize_ip(), which
subtracts kaslr_offset() from recorded PCs. Restore that offset before
printing the PCs with %pB, since KCOV records return addresses.
Link: https://lore.kernel.org/20261002181357.14293-1-kmehltretter@gmail.com
Fixes: 6cd0dd934b03 ("kcov: Add interrupt handling self test")
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Alexander Potapenko <glider@google.com>
Tested-by: Bradley Morgan <brads@mainlining.org> # Power10
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Assisted-by: LLM
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Bradley Morgan <include@grrlz.net>
Cc: Thomas Gleixner <tglx@kernel.org>
|
|
RESOURCE_KUNIT_TEST selects GET_FREE_REGION even when no other option
needs it. Remove the selection to follow the dependency rule in
Documentation/dev-tools/kunit/style.rst.
Skip resource_test_region_intersects() when GET_FREE_REGION is disabled.
GET_FREE_REGION has no prompt, so configurations without a production
consumer cannot enable it. Most configurations will therefore skip this
case; this is intentional. The union and intersection tests remain
available.
Since kunit_skip() does not return, the compiler drops the reference to
the unavailable alloc_free_mem_region(), as in the CONFIG_OF_ADDRESS
check in drivers/of/of_test.c.
Link: https://lore.kernel.org/20260926002651.87267-1-kmehltretter@gmail.com
Fixes: 99185c10d5d9 ("resource, kunit: add test case for region_intersects()")
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Tested-by: Bradley Morgan <brads@mainlining.org> # Power10
Assisted-by: LLM
Cc: Ying Huang <huang.ying.caritas@gmail.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Gow <david@davidgow.net>
Cc: Rae Moar <raemoar63@gmail.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
|
|
write_comp_data() reads the comparison record count from area[0] and uses
it to index the coverage buffer:
area = (u64 *)t->kcov_area;
max_pos = t->kcov_size * sizeof(unsigned long);
count = READ_ONCE(area[0]);
/* Every record is KCOV_WORDS_PER_CMP 64-bit words. */
start_index = 1 + count * KCOV_WORDS_PER_CMP;
end_pos = (start_index + KCOV_WORDS_PER_CMP) * sizeof(u64);
if (likely(end_pos <= max_pos)) {
The buffer is mmap'd writable into the collecting process, so count is
under its control and end_pos <= max_pos is its only bound. A count that
wraps the u64 multiply leaves end_pos below max_pos, so the check passes
while the record store lands 24 bytes before the buffer, in the unmapped
vmalloc guard page, and faults:
BUG: unable to handle page fault for address: ffa0000000b60fe8
#PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
Oops: 0002 [#1] SMP KASAN NOPTI
RIP: 0010:write_comp_data+0x7e/0xa0
...
Kernel panic - not syncing: Fatal exception
Bound count first: only max_pos / (sizeof(u64) * KCOV_WORDS_PER_CMP)
records fit, so a larger count is not a valid index and is dropped.
kcov_move_area() bounds the same untrusted count this way, and no count
the end_pos <= max_pos check accepts reaches that limit, so no valid
record is lost.
kcov is a root-only debugfs file (debugfs_create_file_unsafe("kcov", 0600,
...)) and write_comp_data() exists only under
CONFIG_KCOV_ENABLE_COMPARISONS, so this is a local, debug-kernel
robustness fix: the process corrupts its own buffer and the kernel oopses.
It crosses no privilege boundary. Additional details at [1]
Link: https://lore.kernel.org/20260917104306.22145-1-fangxy@xiaopeng.com [1]
Fixes: ded97d2c2b2c ("kcov: support comparison operands collection")
Signed-off-by: Fang Xieyan <fangxy@xiaopeng.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Alexander Potapenko <glider@google.com>
Assisted-by: Hawkeye:GLM-5.3-flash
Assisted-by: Qoder:Qwen3.8-Max
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Marco Elver <elver@google.com>
Cc: Victor Chibotaru <tchibo@google.com>
Cc: <stable@vger.kernel.org>
|
|
The redirect buffer is a disgusting terrible hack. panic_force_buf is
kmalloc'ed in a late_initcall, and until then the redirect delivers this
as the panic message:
Redirected panic (buffer unavailable)
The whole point of the redirect is to hand the panic message to the target
CPU, so the crash kernel boots knowing it panicked and not why. And that
window is the entire boot, from the early_param to the late_initcall,
which is exactly when you most want the message.
Make it a static 1KB buffer and kill the initcall. The cost is 1KB of
.bss in SMP crash dump builds, and it is only ever touched when
panic_force_cpu= is set anyway. The local msg variable is gone too, the
buffer goes directly to panic_smp_redirect_cpu().
Link: https://lore.kernel.org/20260916182957.7788-7-brads@mainlining.org
Signed-off-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Cc: Guenetr Roeck <linux@roeck-us.net>
Cc: Sashiko <sashiko-bot@kernel.org>
Cc: Wang Jinchao <wangjinchao600@gmail.com>
Cc: Wim Van Sebroeck <wim@linux-watchdog.org>
Cc: <stable@vger.kernel.org>
|
|
nmi_panic() claims panic_cpu via panic_try_start() before calling panic().
When the panic later reaches panic_try_force_cpu(), the
panic_in_progress() check sees panic_cpu set and refuses to redirect. The
crash kernel runs on the CPU that took the NMI instead of the CPU
requested with panic_force_cpu=:
nmi_panic()
panic_try_start() wins, panic_cpu = X
panic("%s", msg)
vpanic()
panic_try_force_cpu()
panic_in_progress() true, panic_cpu is X
return false redirect bypassed
panic_try_start() already won
__crash_kexec() on X, not the requested CPU
Try the redirect before claiming panic_cpu instead, as suggested by Petr
Mladek. nmi_panic() now calls panic_try_force_cpu() first and claims
panic_cpu only when no redirect happened. The requested CPU claims
panic_cpu itself when it runs panic(), so panic_cpu does not need to be
handed off. panic_try_force_cpu() copies the arguments before formatting
(patch 3), so nmi_panic() can pass them to vpanic() again when no redirect
happens.
The redirect IPI is sent with smp_call_function_single_async(), which is
not guaranteed to work from NMI context. Treat it as best effort. It is
worth the risk because the redirection is only used when the crash kernel
would not work on the panicking CPU anyway.
Keep returning when the panic is already running on this CPU. A nested
NMI, for example with unknown_nmi_panic while this CPU is inside panic(),
must return and let the interrupted panic() continue instead of parking
the CPU in nmi_panic_self_stop().
Mark the redirecting CPU offline before stopping it, like vpanic() does,
so that panic_other_cpus_shutdown() on the target CPU does not wait for
it.
Link: https://lore.kernel.org/20260916182957.7788-6-brads@mainlining.org
Fixes: 2e171ab29f91 ("panic: add panic_force_cpu= parameter to redirect panic to a specific CPU")
Signed-off-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260708164312.19044-1-include@grrlz.net
Reviewed-by: Petr Mladek <pmladek@suse.com>
Cc: Guenetr Roeck <linux@roeck-us.net>
Cc: Wang Jinchao <wangjinchao600@gmail.com>
Cc: Wim Van Sebroeck <wim@linux-watchdog.org>
Cc: <stable@vger.kernel.org>
|