linux-toradex.git/arch/powerpc/kernel/interrupt_64.S, branch v6.7

powerpc: Fix merge conflict between pcrel and copy_thread changes

2023-04-26T06:20:16+00:00

Fix a conflict between commit 4e991e3c16a35 ("powerpc: add CFUNC
assembly label annotation") and commit b504b6aade040 ("powerpc:
differentiate kthread from user kernel thread start").

Fixes: 4e991e3c16a35 ("powerpc: add CFUNC assembly label annotation")
Fixes: b504b6aade040 ("powerpc: differentiate kthread from user kernel thread start")
Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230426055848.402993-2-npiggin@gmail.com

powerpc: add CFUNC assembly label annotation

2023-04-20T02:54:24+00:00

This macro is to be used in assembly where C functions are called.
pcrel addressing mode requires branches to functions with a
localentry value of 1 to have either a trailing nop or @notoc.
This macro permits the latter without changing callers.

Signed-off-by: Nicholas Piggin 
[mpe: Add dummy definitions to fix selftests build]
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230408021752.862660-5-npiggin@gmail.com

powerpc: copy_thread don't set _TIF_RESTOREALL

2023-04-11T13:13:33+00:00

In the kernel user thread path, don't set _TIF_RESTOREALL because
the thread is required to call kernel_execve() before it returns,
which will set _TIF_RESTOREALL if necessary via start_thread().

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230325122904.2375060-8-npiggin@gmail.com

powerpc: differentiate kthread from user kernel thread start

2023-04-11T13:13:33+00:00

Kernel created user threads start similarly to kernel threads in that
they call a kernel function after first returning from _switch, so
they share ret_from_kernel_thread for this. Kernel threads never return
from that function though, whereas user threads often do (although some
don't, e.g., IO threads).

Split these startup functions in two, and catch kernel threads that
improperly return from their function. This is intended to make the
complicated code a little bit easier to understand.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230325122904.2375060-7-npiggin@gmail.com

powerpc/64: ret_from_fork avoid restoring regs twice

2023-04-11T13:13:32+00:00

If the system call return path always restores NVGPRs then there is no
need for ret_from_fork to do it. The HANDLER_RESTORE_NVGPRS does the
right thing for this.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230325122904.2375060-5-npiggin@gmail.com

powerpc: use switch frame for ret_from_kernel_thread parameters

2023-04-11T13:13:32+00:00

The kernel thread path in copy_thread creates a user interrupt frame on
stack and stores the function and arg parameters there, and
ret_from_kernel_thread loads them. This is a slightly confusing way to
overload that frame. Non-volatile registers are loaded from the switch
frame, so the parameters can be stored there. The user interrupt frame
is now only used by user threads when they return to user.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://msgid.link/20230325122904.2375060-4-npiggin@gmail.com

powerpc/64s: Zeroise gprs on interrupt routine entry on Book3S

2022-12-02T09:46:05+00:00

Zeroise user state in gprs (assign to zero) to reduce the influence of user
registers on speculation within kernel syscall handlers. Clears occur
at the very beginning of the sc and scv 0 interrupt handlers, with
restores occurring following the execution of the syscall handler.

Zeroise GPRS r0, r2-r11, r14-r31, on entry into the kernel for all
other interrupt sources. The remaining gprs are overwritten by
entry macros to interrupt handlers, irrespective of whether or not a
given handler consumes these register values. If an interrupt does not
select the IMSR_R12 IOption, zeroise r12.

Prior to this commit, r14-r31 are restored on a per-interrupt basis at
exit, but now they are always restored on 64bit Book3S. Remove explicit
REST_NVGPRS invocations on 64-bit Book3S. 32-bit systems do not clear
user registers on interrupt, and continue to depend on the return value
of interrupt_exit_user_prepare to determine whether or not to restore
non-volatiles.

The mmap_bench benchmark in selftests should rapidly invoke pagefaults.
See ~0.8% performance regression with this mitigation, but this
indicates the worst-case performance due to heavier-weight interrupt
handlers. This mitigation is able to be enabled/disabled through
CONFIG_INTERRUPT_SANITIZE_REGISTERS.

Reviewed-by: Nicholas Piggin 
Signed-off-by: Rohan McLure 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/20221201071019.1953023-5-rmclure@linux.ibm.com

powerpc/64: Sanitise common exit code for interrupts

2022-12-02T09:46:01+00:00

Interrupt code is shared between Book3E/S 64-bit systems for interrupt
handlers. Ensure that exit code correctly restores non-volatile gprs on
each system when CONFIG_INTERRUPT_SANITIZE_REGISTERS is enabled.

Also introduce macros for clearing/restoring registers on interrupt
entry for when this configuration option is either disabled or enabled.

Reviewed-by: Nicholas Piggin 
Signed-off-by: Rohan McLure 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/20221201071019.1953023-3-rmclure@linux.ibm.com

powerpc: add a definition for the marker offset within the interrupt frame

2022-12-02T06:54:08+00:00

Define a constant rather than open-code the offset for the
"regs" marker.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/20221127124942.1665522-9-npiggin@gmail.com

powerpc: add definition for pt_regs offset within an interrupt frame

2022-12-02T06:54:08+00:00

This is a common offset that currently uses the overloaded
STACK_FRAME_OVERHEAD constant. It's easier to read and more
flexible to use a specific regs offset for this.

Signed-off-by: Nicholas Piggin 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/20221127124942.1665522-8-npiggin@gmail.com