linux-toradex.git/arch/x86/include/asm/percpu.h, branch v5.18

x86/percpu: Remove volatile from arch_raw_cpu_ptr().

2022-04-05T07:59:38+00:00

The volatile attribute in the inline assembly of arch_raw_cpu_ptr()
forces the compiler to always generate the code, even if the compiler
can decide upfront that its result is not needed.

For instance invoking __intel_pmu_disable_all(false) (like
intel_pmu_snapshot_arch_branch_stack() does) leads to loading the
address of &cpu_hw_events into the register while compiler knows that it
has no need for it. This ends up with code like:

|	movq	$cpu_hw_events, %rax			#, tcp_ptr__
|	add	%gs:this_cpu_off(%rip), %rax		# this_cpu_off, tcp_ptr__
|	xorl	%eax, %eax				# tmp93

It also creates additional code within local_lock() with !RT &&
!LOCKDEP which is not desired.

By removing the volatile attribute the compiler can place the
function freely and avoid it if it is not needed in the end.
By using the function twice the compiler properly caches only the
variable offset and always loads the CPU-offset.

this_cpu_ptr() also remains properly placed within a preempt_disable()
sections because
- arch_raw_cpu_ptr() assembly has a memory input ("m" (this_cpu_off))
- prempt_{dis,en}able() fundamentally has a 'barrier()' in it

Therefore this_cpu_ptr() is already properly serialized and does not
rely on the 'volatile' attribute.

Remove volatile from arch_raw_cpu_ptr().

[ bigeasy: Added Linus' explanation why this_cpu_ptr() is not moved out
  of a preempt_disable() section without the 'volatile' attribute. ]

Suggested-by: Linus Torvalds 
Signed-off-by: Sebastian Andrzej Siewior 
Signed-off-by: Peter Zijlstra (Intel) 
Link: https://lore.kernel.org/r/20220328145810.86783-2-bigeasy@linutronix.de

x86/percpu: Remove unused PER_CPU() macro

2020-07-23T09:46:43+00:00

Also remove now unused __percpu_mov_op.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Linus Torvalds 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-11-ndesaulniers@google.com

x86/percpu: Clean up percpu_stable_op()

2020-07-23T09:46:42+00:00

Use __pcpu_size_call_return() to simplify this_cpu_read_stable().
Also remove __bad_percpu_size() which is now unused.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-10-ndesaulniers@google.com

x86/percpu: Clean up percpu_cmpxchg_op()

2020-07-23T09:46:42+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-9-ndesaulniers@google.com

x86/percpu: Clean up percpu_xchg_op()

2020-07-23T09:46:41+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-8-ndesaulniers@google.com

x86/percpu: Clean up percpu_add_return_op()

2020-07-23T09:46:41+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-7-ndesaulniers@google.com

x86/percpu: Remove "e" constraint from XADD

2020-07-23T09:46:40+00:00

The "e" constraint represents a constant, but the XADD instruction doesn't
accept immediate operands.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-6-ndesaulniers@google.com

x86/percpu: Clean up percpu_add_op()

2020-07-23T09:46:40+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-5-ndesaulniers@google.com

x86/percpu: Clean up percpu_from_op()

2020-07-23T09:46:39+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-4-ndesaulniers@google.com

x86/percpu: Clean up percpu_to_op()

2020-07-23T09:46:39+00:00

The core percpu macros already have a switch on the data size, so the switch
in the x86 code is redundant and produces more dead code.

Also use appropriate types for the width of the instructions.  This avoids
errors when compiling with Clang.

Signed-off-by: Brian Gerst 
Signed-off-by: Nick Desaulniers 
Signed-off-by: Thomas Gleixner 
Tested-by: Nick Desaulniers 
Tested-by: Sedat Dilek 
Reviewed-by: Nick Desaulniers 
Acked-by: Linus Torvalds 
Acked-by: Peter Zijlstra (Intel) 
Acked-by: Dennis Zhou 
Link: https://lkml.kernel.org/r/20200720204925.3654302-3-ndesaulniers@google.com