linux-toradex.git/arch/x86/kernel/relocate_kernel_64.S, branch v6.16

x86/asm: Replace "REP; NOP" with PAUSE mnemonic

2025-04-18T08:19:25+00:00

Current minimum required version of binutils is 2.25,
which supports PAUSE instruction mnemonic.

Replace "REP; NOP" with this proper mnemonic.

No functional change intended.

Reviewed-by: Nikolay Borisov 
Signed-off-by: Uros Bizjak 
Signed-off-by: Ingo Molnar 
Cc: H. Peter Anvin 
Link: https://lore.kernel.org/r/20250418080805.83679-2-ubizjak@gmail.com

x86/asm: Remove semicolon from "rep" prefixes

2025-04-18T07:33:33+00:00

Minimum version of binutils required to compile the kernel is 2.25.
This version correctly handles the "rep" prefixes, so it is possible
to remove the semicolon, which was used to support ancient versions
of GNU as.

Due to the semicolon, the compiler considers "rep; insn" (or its
alternate "rep\n\tinsn" form) as two separate instructions. Removing
the semicolon makes asm length calculations more accurate, consequently
making scheduling and inlining decisions of the compiler more accurate.

Removing the semicolon also enables assembler checks involving "rep"
prefixes. Trying to assemble e.g. "rep addl %eax, %ebx" results in:

  Error: invalid instruction `add' after `rep'

Signed-off-by: Uros Bizjak 
Signed-off-by: Ingo Molnar 
Cc: Andy Lutomirski 
Cc: Boris Ostrovsky 
Cc: Brian Gerst 
Cc: H. Peter Anvin 
Cc: Josh Poimboeuf 
Cc: Juergen Gross 
Cc: Kees Cook 
Cc: Linus Torvalds 
Cc: Pavel Machek 
Cc: Rafael J. Wysocki 
Link: https://lore.kernel.org/r/20250418071437.4144391-2-ubizjak@gmail.com

x86/kexec: Invalidate GDT/IDT from relocate_kernel() instead of earlier

2025-04-10T10:17:14+00:00

Reduce the window during which exceptions are unhandled, by leaving the
GDT/IDT in place all the way into the relocate_kernel() function, until
the moment that %cr3 gets replaced.

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Brian Gerst 
Cc: Juergen Gross 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Josh Poimboeuf 
Cc: Kees Cook 
Cc: Ard Biesheuvel 
Link: https://lore.kernel.org/r/20250326142404.256980-4-dwmw2@infradead.org

x86/kexec: Add 8250 MMIO serial port output

2025-04-10T10:17:14+00:00

This supports the same 32-bit MMIO-mapped 8250 as the early_printk code.

It's not clear why the early_printk code supports this form and only this
form; the actual runtime 8250_pci doesn't seem to support it. But having
hacked up QEMU to expose such a device, early_printk does work with it,
and now so does the kexec debug code.

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Andy Shevchenko 
Cc: Brian Gerst 
Cc: Juergen Gross 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Josh Poimboeuf 
Cc: Kees Cook 
Cc: Ard Biesheuvel 
Link: https://lore.kernel.org/r/20250326142404.256980-3-dwmw2@infradead.org

x86/kexec: Add 8250 serial port output

2025-04-10T10:17:13+00:00

If a serial port was configured for early_printk, use it for debug output
from the relocate_kernel exception handler too.

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Andy Shevchenko 
Cc: Brian Gerst 
Cc: Juergen Gross 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Josh Poimboeuf 
Cc: Kees Cook 
Cc: Ard Biesheuvel 
Link: https://lore.kernel.org/r/20250326142404.256980-2-dwmw2@infradead.org

x86/kexec: Debugging support: Dump registers on exception

2025-03-25T11:49:05+00:00

The actual serial output function is a no-op for now.

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Brian Gerst 
Cc: Juergen Gross 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Josh Poimboeuf 
Cc: Kees Cook 
Link: https://lore.kernel.org/r/20250314173226.3062535-3-dwmw2@infradead.org

x86/kexec: Debugging support: Load an IDT and basic exception entry points

2025-03-25T11:49:05+00:00

[ mingo: Minor readability edits ]

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Brian Gerst 
Cc: Juergen Gross 
Cc: H. Peter Anvin 
Cc: Linus Torvalds 
Cc: Josh Poimboeuf 
Cc: Kees Cook 
Link: https://lore.kernel.org/r/20250314173226.3062535-2-dwmw2@infradead.org

x86/kexec: Add relocate_kernel() debugging support: Load a GDT

2025-03-14T10:01:53+00:00

There are some failure modes which lead to triple-faults in the
relocate_kernel() function, which is fairly much undebuggable
for normal mortals.

Adding a GDT in the relocate_kernel() environment is step 1 towards
being able to catch faults and do something more useful.

Signed-off-by: David Woodhouse 
Signed-off-by: Ingo Molnar 
Cc: Andy Lutomirski 
Cc: Brian Gerst 
Cc: H. Peter Anvin 
Cc: Kees Cook 
Cc: Ard Biesheuvel 
Cc: Linus Torvalds 
Link: https://lore.kernel.org/r/20250312144257.2348250-2-dwmw2@infradead.org

Merge tag 'x86-boot-2025-01-21' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2025-01-24T13:54:26+00:00

Pull x86 boot updates from Ingo Molnar:

 - A large and involved preparatory series to pave the way to add
   exception handling for relocate_kernel - which will be a debugging
   facility that has aided in the field to debug an exceptionally hard
   to debug early boot bug. Plus assorted cleanups and fixes that were
   discovered along the way, by David Woodhouse:

      - Clean up and document register use in relocate_kernel_64.S
      - Use named labels in swap_pages in relocate_kernel_64.S
      - Only swap pages for ::preserve_context mode
      - Allocate PGD for x86_64 transition page tables separately
      - Copy control page into place in machine_kexec_prepare()
      - Invoke copy of relocate_kernel() instead of the original
      - Move relocate_kernel to kernel .data section
      - Add data section to relocate_kernel
      - Drop page_list argument from relocate_kernel()
      - Eliminate writes through kernel mapping of relocate_kernel page
      - Clean up register usage in relocate_kernel()
      - Mark relocate_kernel page as ROX instead of RWX
      - Disable global pages before writing to control page
      - Ensure preserve_context flag is set on return to kernel
      - Use correct swap page in swap_pages function
      - Fix stack and handling of re-entry point for ::preserve_context
      - Mark machine_kexec() with __nocfi
      - Cope with relocate_kernel() not being at the start of the page
      - Use typedef for relocate_kernel_fn function prototype
      - Fix location of relocate_kernel with -ffunction-sections (fix by Nathan Chancellor)

 - A series to remove the last remaining absolute symbol references from
   .head.text, and enforce this at build time, by Ard Biesheuvel:

      - Avoid WARN()s and panic()s in early boot code
      - Don't hang but terminate on failure to remap SVSM CA
      - Determine VA/PA offset before entering C code
      - Avoid intentional absolute symbol references in .head.text
      - Disable UBSAN in early boot code
      - Move ENTRY_TEXT to the start of the image
      - Move .head.text into its own output section
      - Reject absolute references in .head.text

 - The above build-time enforcement uncovered a handful of bugs of
   essentially non-working code, and a wrokaround for a toolchain bug,
   fixed by Ard Biesheuvel as well:

      - Fix spurious undefined reference when CONFIG_X86_5LEVEL=n, on GCC-12
      - Disable UBSAN on SEV code that may execute very early
      - Disable ftrace branch profiling in SEV startup code

 - And miscellaneous cleanups:

      - kexec_core: Add and update comments regarding the KEXEC_JUMP flow (Rafael J. Wysocki)
      - x86/sysfs: Constify 'struct bin_attribute' (Thomas Weißschuh)"

* tag 'x86-boot-2025-01-21' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (33 commits)
  x86/sev: Disable ftrace branch profiling in SEV startup code
  x86/kexec: Use typedef for relocate_kernel_fn function prototype
  x86/kexec: Cope with relocate_kernel() not being at the start of the page
  kexec_core: Add and update comments regarding the KEXEC_JUMP flow
  x86/kexec: Mark machine_kexec() with __nocfi
  x86/kexec: Fix location of relocate_kernel with -ffunction-sections
  x86/kexec: Fix stack and handling of re-entry point for ::preserve_context
  x86/kexec: Use correct swap page in swap_pages function
  x86/kexec: Ensure preserve_context flag is set on return to kernel
  x86/kexec: Disable global pages before writing to control page
  x86/sev: Don't hang but terminate on failure to remap SVSM CA
  x86/sev: Disable UBSAN on SEV code that may execute very early
  x86/boot/64: Fix spurious undefined reference when CONFIG_X86_5LEVEL=n, on GCC-12
  x86/sysfs: Constify 'struct bin_attribute'
  x86/kexec: Mark relocate_kernel page as ROX instead of RWX
  x86/kexec: Clean up register usage in relocate_kernel()
  x86/kexec: Eliminate writes through kernel mapping of relocate_kernel page
  x86/kexec: Drop page_list argument from relocate_kernel()
  x86/kexec: Add data section to relocate_kernel
  x86/kexec: Move relocate_kernel to kernel .data section
  ...

x86/kexec: Cope with relocate_kernel() not being at the start of the page

2025-01-14T12:05:14+00:00

A few places in the kexec control code page make the assumption that the first
instruction of relocate_kernel is at the very start of the page.

To allow for Clang CFI information to be added to relocate_kernel(), as well
as the general principle of removing unwarranted assumptions, fix them to use
the external __relocate_kernel_start symbol that the linker adds. This means
using a separate addq and subq for calculating offsets, as the assembler can
no longer calculate the delta directly for itself and relocations aren't that
versatile. But those values can at least be used relative to a local label to
avoid absolute relocations.

Turn the jump from relocate_kernel() to identity_mapped() into a real indirect
'jmp *%rsi' too, while touching it. There was no real reason for it to be
a push+ret in the first place, and adding Clang CFI info will also give
objtool enough visibility to start complaining 'return with modified stack
frame' about it.

  [ bp: Massage commit message. ]

Signed-off-by: David Woodhouse 
Signed-off-by: Borislav Petkov (AMD) 
Link: https://lore.kernel.org/r/20250109140757.2841269-9-dwmw2@infradead.org