From aeec9d9ae6143db48a01d8acc64c940b4e65d762 Mon Sep 17 00:00:00 2001 From: Zenghui Yu Date: Sat, 11 Jul 2026 21:51:54 +0800 Subject: cpu/hotplug: Remove CPUHP_AP_ARM_CORESIGHT_CTI_STARTING Thestate CPUHP_AP_ARM_CORESIGHT_CTI_STARTING has become unused since commit 59213b4be5c1 ("coresight: cti: Remove CPU power management code"). Remove it as well. Signed-off-by: Zenghui Yu Signed-off-by: Thomas Gleixner Link: https://patch.msgid.link/20260711135154.94276-1-zenghui.yu@linux.dev --- include/linux/cpuhotplug.h | 1 - 1 file changed, 1 deletion(-) (limited to 'include/linux') diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 0fb3a2a62eb0..feb32949aeea 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -180,7 +180,6 @@ enum cpuhp_state { CPUHP_AP_DUMMY_TIMER_STARTING, CPUHP_AP_ARM_XEN_STARTING, CPUHP_AP_ARM_XEN_RUNSTATE_STARTING, - CPUHP_AP_ARM_CORESIGHT_CTI_STARTING, CPUHP_AP_ARM64_ISNDEP_STARTING, CPUHP_AP_SMPCFD_DYING, CPUHP_AP_HRTIMERS_DYING, -- cgit v1.2.3 From 0485235b6e0dcb8c70e5c947faa2b5031febd736 Mon Sep 17 00:00:00 2001 From: Chuyi Zhou Date: Thu, 9 Jul 2026 20:29:22 +0800 Subject: smp: Refactor remote CPU selection in smp_call_function_any() smp_call_function_any() disables preemption across the entire operation: selecting a target CPU, enqueueing the IPI, and synchronously waiting for the remote CPU. smp_call_function_single() already re-enables preemption before the synchronous csd_lock_wait(), so callers of smp_call_function_any() should benefit from the same shorter preemption-disabled section. Simply removing get_cpu() and put_cpu() from smp_call_function_any() would leave the preemption disablement entirely to smp_call_function_single(). That opens a preemption window between selecting the remote CPU, for example via sched_numa_find_nth_cpu(), and dispatching the IPI in smp_call_function_single(). If the selected CPU is fully offlined in that window, smp_call_function_single() fails its cpu_online() check and returns -ENXIO to the caller, violating the guarantee that smp_call_function_any() executes on any online CPU in the mask. Move the remote CPU selection into a common __smp_call_function_single() helper. Keep the target CPU selection and IPI dispatch within the same preemption-disabled region, while still allowing the wait path to use the shorter preemption-disabled section provided by smp_call_function_single(). Signed-off-by: Chuyi Zhou Signed-off-by: Thomas Gleixner Tested-by: Paul E. McKenney Reviewed-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20260709122933.4021501-4-zhouchuyi@bytedance.com --- include/linux/smp.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'include/linux') diff --git a/include/linux/smp.h b/include/linux/smp.h index 6925d15ccaa7..11e36c7bc4d6 100644 --- a/include/linux/smp.h +++ b/include/linux/smp.h @@ -47,8 +47,7 @@ extern void __smp_call_single_queue(int cpu, struct llist_node *node); /* total number of cpus in this system (may exceed NR_CPUS) */ extern unsigned int total_cpus; -int smp_call_function_single(int cpuid, smp_call_func_t func, void *info, - int wait); +int smp_call_function_single(int cpuid, smp_call_func_t func, void *info, bool wait); void on_each_cpu_cond_mask(smp_cond_func_t cond_func, smp_call_func_t func, void *info, bool wait, const struct cpumask *mask); -- cgit v1.2.3 From 9a560af15fd31b3b93235b1584a2509445a17963 Mon Sep 17 00:00:00 2001 From: Chuyi Zhou Date: Thu, 9 Jul 2026 20:29:23 +0800 Subject: smp: Use task-local IPI cpumask in smp_call_function_many_cond() smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list of remote CPUs to wait for. That is safe while the caller remains pinned to the current CPU for the whole operation, because another task cannot run on the same CPU and reuse the per-CPU mask. The synchronous wait is the long-latency part of the operation. To make that wait preemptible, the mask iterated by csd_lock_wait() must remain stable even if the task is preempted or migrates. If the wait used the per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled on the original CPU could enter smp_call_function_many_cond() and overwrite the mask while the first task is still iterating it. Give each task private IPI cpumask storage and use it as the wait mask in smp_call_function_many_cond(). Other cpumask storage choices do not fit this use case: - Per-CPU storage is the state that becomes unsafe once the wait is made preemptible. After the caller drops CPU pinning, another task scheduled on the original CPU can enter smp_call_function_many_cond() and reuse the same per-CPU mask. - Stack storage is not suitable for large NR_CPUS or CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to scale with cpumask_size(), and putting that storage on the stack is not acceptable on large systems. - Allocating the mask inside smp_call_function_many_cond() would put an allocation and a failure path in the generic IPI path. A sleeping allocation is not suitable because callers have historically only provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC would avoid sleeping, but a failure fallback would make the latency improvement opportunistic instead of guaranteed. The users are not limited to a small, pre-identifiable class of tasks. On x86, ordinary tasks can reach this path through TLB flushes during exit, unmap and reclaim, so allocating the mask only for a known subset of tasks is not straightforward. The memory cost is explicit: one word is added to task_struct. When cpumask_size() fits in that word, the mask is stored inline and no separate allocation is needed. Larger systems allocate cpumask_size() per task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context, x86 already carries several KiB of per-task architecture and FPU state, depending on the enabled features and configuration. That does not make the extra cpumask free, but it puts the large-NR_CPUS case in perspective. Signed-off-by: Chuyi Zhou Signed-off-by: Thomas Gleixner Tested-by: Paul E. McKenney Reviewed-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com --- include/linux/sched.h | 12 ++++++++++++ include/linux/smp.h | 12 ++++++++++++ 2 files changed, 24 insertions(+) (limited to 'include/linux') diff --git a/include/linux/sched.h b/include/linux/sched.h index 373bcc0598d1..5738c54eb85a 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -823,6 +823,17 @@ struct kmap_ctrl { #endif }; +#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPTION) +struct task_ipi_mask { + union { + cpumask_t *ipi_mask_ptr; + unsigned long ipi_mask_val; + }; +}; +#else +struct task_ipi_mask { }; +#endif + struct task_struct { #ifdef CONFIG_THREAD_INFO_IN_TASK /* @@ -1359,6 +1370,7 @@ struct task_struct { struct list_head perf_event_list; struct perf_ctx_data __rcu *perf_ctx_data; #endif + struct task_ipi_mask __private ipi_mask; #ifdef CONFIG_DEBUG_PREEMPT unsigned long preempt_disable_ip; #endif diff --git a/include/linux/smp.h b/include/linux/smp.h index 11e36c7bc4d6..2dfa7390717a 100644 --- a/include/linux/smp.h +++ b/include/linux/smp.h @@ -238,6 +238,18 @@ static inline int get_boot_cpu_id(void) #endif /* !SMP */ +#if defined(CONFIG_PREEMPTION) && defined(CONFIG_SMP) +int smp_task_ipi_mask_alloc(struct task_struct *task); +void smp_task_ipi_mask_free(struct task_struct *task); +#else +static inline int smp_task_ipi_mask_alloc(struct task_struct *task) +{ + return 0; +} + +static inline void smp_task_ipi_mask_free(struct task_struct *task) { } +#endif + /* * raw_smp_processor_id() - get the current (unstable) CPU id * -- cgit v1.2.3