linux-toradex.git/kernel/smp.c, branch v6.4-rc1

trace,smp: Trace all smp_function_call*() invocations

2023-03-24T10:01:30+00:00

(Ab)use the trace_ipi_send_cpu*() family to trace all
smp_function_call*() invocations, not only those that result in an
actual IPI.

The queued entries log their callback function while the actual IPIs
are traced on generic_smp_call_function_single_interrupt().

Signed-off-by: Peter Zijlstra (Intel)

trace: Add trace_ipi_send_cpu()

2023-03-24T10:01:29+00:00

Because copying cpumasks around when targeting a single CPU is a bit
daft...

Tested-and-reviewed-by: Valentin Schneider 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lkml.kernel.org/r/20230322103004.GA571242%40hirez.programming.kicks-ass.net

sched, smp: Trace smp callback causing an IPI

2023-03-24T10:01:29+00:00

Context
=======

The newly-introduced ipi_send_cpumask tracepoint has a "callback" parameter
which so far has only been fed with NULL.

While CSD_TYPE_SYNC/ASYNC and CSD_TYPE_IRQ_WORK share a similar backing
struct layout (meaning their callback func can be accessed without caring
about the actual CSD type), CSD_TYPE_TTWU doesn't even have a function
attached to its struct. This means we need to check the type of a CSD
before eventually dereferencing its associated callback.

This isn't as trivial as it sounds: the CSD type is stored in
__call_single_node.u_flags, which get cleared right before the callback is
executed via csd_unlock(). This implies checking the CSD type before it is
enqueued on the call_single_queue, as the target CPU's queue can be flushed
before we get to sending an IPI.

Furthermore, send_call_function_single_ipi() only has a CPU parameter, and
would need to have an additional argument to trickle down the invoked
function. This is somewhat silly, as the extra argument will always be
pushed down to the function even when nothing is being traced, which is
unnecessary overhead.

Changes
=======

send_call_function_single_ipi() is only used by smp.c, and is defined in
sched/core.c as it contains scheduler-specific ops (set_nr_if_polling() of
a CPU's idle task).

Split it into two parts: the scheduler bits remain in sched/core.c, and the
actual IPI emission is moved into smp.c. This lets us define an
__always_inline helper function that can take the related callback as
parameter without creating useless register pressure in the non-traced path
which only gains a (disabled) static branch.

Do the same thing for the multi IPI case.

Signed-off-by: Valentin Schneider 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/20230307143558.294354-8-vschneid@redhat.com

smp: reword smp call IPI comment

2023-03-24T10:01:28+00:00

Accessing the call_single_queue hasn't involved a spinlock since 2014:

  6897fc22ea01 ("kernel: use lockless list for smp_call_function_single")

The llist operations (namely cmpxchg() and xchg()) provide similar ordering
guarantees, update the comment to lessen confusion.

Signed-off-by: Valentin Schneider 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/20230307143558.294354-7-vschneid@redhat.com

smp: Trace IPIs sent via arch_send_call_function_ipi_mask()

2023-03-24T10:01:27+00:00

This simply wraps around the arch function and prepends it with a
tracepoint, similar to send_call_function_single_ipi().

Signed-off-by: Valentin Schneider 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (Google) 
Link: https://lore.kernel.org/r/20230307143558.294354-4-vschneid@redhat.com

sched, smp: Trace IPIs sent via send_call_function_single_ipi()

2023-03-24T10:01:27+00:00

send_call_function_single_ipi() is the thing that sends IPIs at the bottom
of smp_call_function*() via either generic_exec_single() or
smp_call_function_many_cond(). Give it an IPI-related tracepoint.

Note that this ends up tracing any IPI sent via __smp_call_single_queue(),
which covers __ttwu_queue_wakelist() and irq_work_queue_on() "for free".

Signed-off-by: Valentin Schneider 
Signed-off-by: Peter Zijlstra (Intel) 
Reviewed-by: Steven Rostedt (Google) 
Acked-by: Ingo Molnar 
Link: https://lore.kernel.org/r/20230307143558.294354-3-vschneid@redhat.com

kernel/smp: Make csdlock_debug= resettable

2023-03-24T10:01:26+00:00

It is currently possible to set the csdlock_debug_enabled static
branch, but not to reset it.  This is an issue when several different
entities supply kernel boot parameters and also for kernels built with
CONFIG_CSD_LOCK_WAIT_DEBUG_DEFAULT=y.

Therefore, make the csdlock_debug=0 kernel boot parameter turn off
debugging.  Last one wins!

Reported-by: Jes Sorensen 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Juergen Gross 
Link: https://lore.kernel.org/r/20230321005516.50558-4-paulmck@kernel.org

locking/csd_lock: Remove per-CPU data indirection from CSD lock debugging

2023-03-24T10:01:26+00:00

The diagnostics added by this commit were extremely useful in one instance:

a5aabace5fb8 ("locking/csd_lock: Add more data to CSD lock debugging")

However, they have not seen much action since, and there have been some
concerns expressed that the complexity is not worth the benefit.

Therefore, manually revert the following commit preparatory commit:

de7b09ef658d ("locking/csd_lock: Prepare more CSD lock debugging")

Signed-off-by: Paul E. McKenney 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Juergen Gross 
Link: https://lore.kernel.org/r/20230321005516.50558-3-paulmck@kernel.org

locking/csd_lock: Remove added data from CSD lock debugging

2023-03-24T10:01:25+00:00

The diagnostics added by this commit were extremely useful in one instance:

a5aabace5fb8 ("locking/csd_lock: Add more data to CSD lock debugging")

However, they have not seen much action since, and there have been some
concerns expressed that the complexity is not worth the benefit.

Therefore, manually revert this commit, but leave a comment telling
people where to find these diagnostics.

[ paulmck: Apply Juergen Gross feedback. ]

Signed-off-by: Paul E. McKenney 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Juergen Gross 
Link: https://lore.kernel.org/r/20230321005516.50558-2-paulmck@kernel.org

locking/csd_lock: Add Kconfig option for csd_debug default

2023-03-24T10:01:25+00:00

The csd_debug kernel parameter works well, but is inconvenient in cases
where it is more closely associated with boot loaders or automation than
with a particular kernel version or release.  Thererfore, provide a new
CSD_LOCK_WAIT_DEBUG_DEFAULT Kconfig option that defaults csd_debug to
1 when selected and 0 otherwise, with this latter being the default.

Signed-off-by: Paul E. McKenney 
Signed-off-by: Peter Zijlstra (Intel) 
Acked-by: Juergen Gross 
Link: https://lore.kernel.org/r/20230321005516.50558-1-paulmck@kernel.org