commit bee25f97ad24 ("powerpc: Enable GENERIC_ENTRY feature") introduced
a bug that, when CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX is not
set, expand to two consecutive ELFv2 global entry points for the
same function body:
_GLOBAL_TOC_KASAN(memcpy) expands to:
_GLOBAL_TOC(memcpy);
_GLOBAL_TOC(__memcpy)
On powerpc64le this produces:
memcpy:
addis r2,r12,(.TOC.-0b)@ha <- global entry, sets up r2
addi r2,r2,(.TOC.-0b)@l
.localentry memcpy,.-memcpy <- local entry at memcpy+8
__memcpy:
addis r2,r12,(.TOC.-0b)@ha <- memcpy+8: __memcpy global entry
addi r2,r2,(.TOC.-0b)@l
.localentry __memcpy,.-__memcpy
The assembler sets memcpy's st_other field so its local entry offset is
8 bytes past the global entry point. The linker resolves same-TOC
calls to the local entry, so every same-TOC caller of memcpy() lands
at memcpy+8 — which is the beginning of the __memcpy global-entry TOC
prologue. That prologue uses r12 as the base to recompute r2, but r12
holds whatever the caller left in it (not the address of memcpy+8).
The result is r2 pointing at garbage on return. Same-TOC callers do
not reload r2 after the call, so the first TOC-relative access after
returning from memcpy() or memmove() crashes or silently corrupts data.
With GENERIC_ENTRY selected, mm/kasan/shadow.c no longer provides its
own memcpy()/memmove()/memset() wrappers, so on a toolchain without
-mllvm -asan-kernel-mem-intrinsic-prefix=1 (e.g. GCC 9) every
instrumented file resolves mem*() directly to the assembly symbols and
hits this corrupt-r2 path. This manifests as an early boot hang unique
to powerpc with GCC 9 and CONFIG_KASAN=y.
Fix this by removing the dual-entry macros entirely. The assembly
files in mem_64.S, memcpy_64.S and copy_32.S now use plain
_GLOBAL()/_GLOBAL_TOC() for memset/memcpy/memmove, emitting a single
correct ELFv2 entry point. The __mem* aliases required by mm/kasan
(used in mm/kasan/shadow.c and mm/kasan/generic.c as raw backends to
bypass KASAN checking) are provided entirely through the #define
redirections in asm/string.h, which already handles both the
CC_HAS_KASAN_MEMINTRINSIC_PREFIX and non-prefix cases in C.
As a consequence the _GLOBAL_KASAN(), _GLOBAL_TOC_KASAN() and
EXPORT_SYMBOL_KASAN() macros have no remaining users and are removed
from asm/kasan.h. The has_renamed_memintrinsics() branch in
prom_init_check.sh, which conditionally whitelisted __memcpy/__memset
when CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX was absent, is dead code
after this change and is removed; the whitelist now unconditionally
lists memcpy and memset.
Fixes: bee25f97ad24 ("powerpc: Enable GENERIC_ENTRY feature")
Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Closes: https://lore.kernel.org/all/96dae110-f79b-4e54-8a9b-514369019b5d@linux.ibm.com
Tested-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
[Maddy: Fixed commit message to avoid checkpatch error]
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20261003094006.3404731-1-mkchauras@gmail.com
When compiling with W=1 on 32-bit PowerPC, -Wunused-but-set-variable was
set, and alongside -Werror caused the build to fail with this error:
```
../arch/powerpc/lib/sstep.c: In function 'analyse_instr':
../arch/powerpc/lib/sstep.c:1350:28: error: variable 'suffix' set but not used [-Werror=unused-but-set-variable]
1350 | unsigned int word, suffix;
| ^~~~~~
../arch/powerpc/lib/sstep.c:1346:38: error: variable 'rc' set but not used [-Werror=unused-but-set-variable]
1346 | unsigned int opcode, ra, rb, rc, rd, spr, u;
| ^~
cc1: all warnings being treated as errors
```
Signed-off-by: Link Mauve <linkmauve@linkmauve.fr>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260730065007.964703-3-linkmauve@linkmauve.fr
Previously, both CONFIG_PPC64 and __powerpc64__ were being used to check whether we built a 32-bit or a 64-bit kernel, sometimes in a single function. I went with CONFIG_PPC64 because it’s the most used in the kernel, and the preferred way to check whether we’re building a 64-bit kernel, not just 64-bit code, as recommended by Christophe Leroy. Signed-off-by: Link Mauve <linkmauve@linkmauve.fr> Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org> Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com> Link: https://patch.msgid.link/20260730065007.964703-2-linkmauve@linkmauve.fr
Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
modes"), powerpc always has CONFIG_PREEMPTION because only
CONFIG_PREEMPT and CONFIG_PREEMPT_LAZY are possible, even in
dynamic preemption mode (see sched_dynamic_mode).
As a consequence, need_irq_preemption() is always true and can be
removed.
And because commit bee25f97ad24 ("powerpc: Enable GENERIC_ENTRY
feature") includes linux/irq-entry-common.h which already declares
sk_dynamic_irqentry_exit_cond_resched static key, asm/preempt.h
becauses useless and can be removed.
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/2bf10a0afffefb6aca44bf2f864cc17471a80e31.1781870889.git.chleroy@kernel.org
strncpy() has no remaining callers in the kernel[1]. Remove the powerpc-specific assembly implementation from both the kernel (arch/powerpc/lib/string.S) and the boot wrapper (arch/powerpc/boot/string.S), along with the __HAVE_ARCH_STRNCPY define and declaration, falling back to the generic version in lib/string.c. The boot wrapper's strncpy had no callers in arch/powerpc/boot/. Link: https://github.com/KSPP/linux/issues/90 [1] Signed-off-by: Kees Cook <kees@kernel.org>
Pull arm64 updates from Will Deacon:
"It feels like the new world of AI tooling has slowed us down a little
on the feature side when compared to the fixes side. The extra rounds
of Sashiko review have also pushed a few things out until next time.
Still, there's some good foundational stuff here for the fpsimd code
and hardening work towards removing the predictable linear alias of
the kernel image.
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more
CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the
core architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections
from the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking
secondary cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board"
* tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux: (73 commits)
arm64: errata: Mitigate TLBI errata on Microsoft Azure Cobalt 100 CPU
arm64: errata: Mitigate TLBI errata on NVIDIA Olympus CPU
arm64: errata: Mitigate TLBI errata on various Arm CPUs
arm64: cputype: Add C1-Premium definitions
arm64: cputype: Add C1-Ultra definitions
Revert "arm64: mm: Unmap kernel data/bss entirely from the linear map"
Revert "arm64: mm: Defer remap of linear alias of data/bss"
arm64: arch_timer: reuse arch_timer_read_cnt{p,v}ct_el0() helpers
arm64/mm: Rename ptdesc_t
arm64: mm: Defer remap of linear alias of data/bss
KVM: arm64: Omit tag sync on stage-2 mappings of the zero page
arm64: Avoid double evaluation of __ptep_get()
kasan: Move generic KASAN page tables out of BSS too
arm64: Rename page table BSS section to .bss..pgtbl
arm64: patching: replace min_t with min in __text_poke
perf/arm-cmn: Fix DVM node events
arm64: fpsimd: Remove <asm/fpsimdmacros.h>
arm64: fpsimd: Move SME save/restore inline
arm64: fpsimd: Move sve_flush_live() inline
arm64: fpsimd: Move SVE save/restore inline
...
The only remaining use of map_patch_area() is mapping the zero page, and immediately unmapping it again so that the intermediate page table levels are all guaranteed to be populated. The use of the zero page here is completely arbitrary, and not harmful per se, but currently, it creates a writable mapping, and does so in a manner that requires that the empty_zero_page[] symbol is not const-qualified. Given that this is about to change, and that map_patch_area() now never maps anything other than the zero page, let's simplify the code and - remove the helpers and call [un]map_kernel_page() directly - take the PA of empty_zero_page directly - create a read-only temporary mapping. This allows empty_zero_page[] to be repainted as const u8[] in a subsequent patch, without making substantial changes to this code patching logic. Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy (CS GROUP) <chleroy@kernel.org> Link: https://lore.kernel.org/all/20260520085423.485402-1-ardb@kernel.org/ Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
The kexec sequence invokes enter_vmx_ops() via copy_page() with the MMU
disabled. In this context, code must not rely on normal virtual address
translations or trigger page faults.
With KASAN enabled, functions get instrumented and may access shadow
memory using regular address translation. When executed with the MMU
off, this can lead to page faults (bad_page_fault) from which the
kernel cannot recover in the kexec path, resulting in a hang.
The kexec path sets preempt_count to HARDIRQ_OFFSET before entering
the MMU-off copy sequence.
current_thread_info()->preempt_count = HARDIRQ_OFFSET
kexec_sequence(..., copy_with_mmu_off = 1)
-> kexec_copy_flush(image)
copy_segments()
-> copy_page(dest, addr)
bl enter_vmx_ops()
if (in_interrupt())
return 0
beq .Lnonvmx_copy
Since kexec sets preempt_count to HARDIRQ_OFFSET, in_interrupt()
evaluates to true and enter_vmx_ops() returns early.
As in_interrupt() (and preempt_count()) are always inlined, mark
enter_vmx_ops() with __no_sanitize_address to avoid KASAN
instrumentation and shadow memory access with MMU disabled, helping
kexec boot fine with KASAN enabled.
Reported-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Reviewed-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260407124349.1698552-2-sourabhjain@linux.ibm.com
Pull non-MM updates from Andrew Morton:
- "pid: make sub-init creation retryable" (Oleg Nesterov)
Make creation of init in a new namespace more robust by clearing away
some historical cruft which is no longer needed. Also some
documentation fixups
- "selftests/fchmodat2: Error handling and general" (Mark Brown)
Fix and a cleanup for the fchmodat2() syscall selftest
- "lib: polynomial: Move to math/ and clean up" (Andy Shevchenko)
- "hung_task: Provide runtime reset interface for hung task detector"
(Aaron Tomlin)
Give administrators the ability to zero out
/proc/sys/kernel/hung_task_detect_count
- "tools/getdelays: use the static UAPI headers from
tools/include/uapi" (Thomas Weißschuh)
Teach getdelays to use the in-kernel UAPI headers rather than the
system-provided ones
- "watchdog/hardlockup: Improvements to hardlockup" (Mayank Rungta)
Several cleanups and fixups to the hardlockup detector code and its
documentation
- "lib/bch: fix undefined behavior from signed left-shifts" (Josh Law)
A couple of small/theoretical fixes in the bch code
- "ocfs2/dlm: fix two bugs in dlm_match_regions()" (Junrui Luo)
- "cleanup the RAID5 XOR library" (Christoph Hellwig)
A quite far-reaching cleanup to this code. I can't do better than to
quote Christoph:
"The XOR library used for the RAID5 parity is a bit of a mess right
now. The main file sits in crypto/ despite not being cryptography
and not using the crypto API, with the generic implementations
sitting in include/asm-generic and the arch implementations
sitting in an asm/ header in theory. The latter doesn't work for
many cases, so architectures often build the code directly into
the core kernel, or create another module for the architecture
code.
Change this to a single module in lib/ that also contains the
architecture optimizations, similar to the library work Eric
Biggers has done for the CRC and crypto libraries later. After
that it changes to better calling conventions that allow for
smarter architecture implementations (although none is contained
here yet), and uses static_call to avoid indirection function call
overhead"
- "lib/list_sort: Clean up list_sort() scheduling workarounds"
(Kuan-Wei Chiu)
Clean up this library code by removing a hacky thing which was added
for UBIFS, which UBIFS doesn't actually need
- "Fix bugs in extract_iter_to_sg()" (Christian Ehrhardt)
Fix a few bugs in the scatterlist code, add in-kernel tests for the
now-fixed bugs and fix a leak in the test itself
- "kdump: Enable LUKS-encrypted dump target support in ARM64 and
PowerPC" (Coiby Xu)
Enable support of the LUKS-encrypted device dump target on arm64 and
powerpc
- "ocfs2: consolidate extent list validation into block read callbacks"
(Joseph Qi)
Cleanup, simplify, and make more robust ocfs2's validation of extent
list fields (Kernel test robot loves mounting corrupted fs images!)
* tag 'mm-nonmm-stable-2026-04-15-04-20' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (127 commits)
ocfs2: validate group add input before caching
ocfs2: validate bg_bits during freefrag scan
ocfs2: fix listxattr handling when the buffer is full
doc: watchdog: fix typos etc
update Sean's email address
ocfs2: use get_random_u32() where appropriate
ocfs2: split transactions in dio completion to avoid credit exhaustion
ocfs2: remove redundant l_next_free_rec check in __ocfs2_find_path()
ocfs2: validate extent block list fields during block read
ocfs2: remove empty extent list check in ocfs2_dx_dir_lookup_rec()
ocfs2: validate dx_root extent list fields during block read
ocfs2: fix use-after-free in ocfs2_fault() when VM_FAULT_RETRY
ocfs2: handle invalid dinode in ocfs2_group_extend
.get_maintainer.ignore: add Askar
ocfs2: validate bg_list extent bounds in discontig groups
checkpatch: exclude forward declarations of const structs
tools/accounting: handle truncated taskstats netlink messages
taskstats: set version in TGID exit notifications
ocfs2/heartbeat: fix slot mapping rollback leaks on error paths
arm64,ppc64le/kdump: pass dm-crypt keys to kdump kernel
...
Pull powerpc updates from Madhavan Srinivasan: - powerpc support for huge pfnmaps - Cleanups to use masked user access - Rework pnv_ioda_pick_m64_pe() to use better bitmap API - Convert powerpc to AUDIT_ARCH_COMPAT_GENERIC - Backup region offset update to eflcorehdr - Fixes for wii/ps3 platform - Implement JIT support for private stack in powerpc - Implement JIT support for fsession in powerpc64 trampoline - Add support for instruction array and indirect jump in powerpc - Misc selftest fixes and cleanups Thanks to Abhishek Dubey, Aditya Gupta, Alex Williamson, Amit Machhiwal, Andrew Donnellan, Bartosz Golaszewski, Cédric Le Goater, Chen Ni, Christophe Leroy (CS GROUP), Hari Bathini, J. Neuschäfer, Mukesh Kumar Chaurasiya (IBM), Nam Cao, Nilay Shroff, Pavithra Prakash, Randy Dunlap, Ritesh Harjani (IBM), Shrikanth Hegde, Sourabh Jain, Vaibhav Jain, Venkat Rao Bagalkote, and Yury Norov (NVIDIA) * tag 'powerpc-7.1-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: (47 commits) mailmap: Add entry for Andrew Donnellan powerpc32/bpf: fix loading fsession func metadata using PPC_LI32 selftest/bpf: Enable gotox tests for powerpc64 powerpc64/bpf: Add support for indirect jump selftest/bpf: Enable instruction array test for powerpc powerpc/bpf: Add support for instruction array powerpc32/bpf: Add fsession support powerpc64/bpf: Implement fsession support selftests/bpf: Enable private stack tests for powerpc64 powerpc64/bpf: Implement JIT support for private stack powerpc: pci-ioda: Optimize pnv_ioda_pick_m64_pe() powerpc: pci-ioda: use bitmap_alloc() in pnv_ioda_pick_m64_pe() powerpc/net: Inline checksum wrappers and convert to scoped user access powerpc/sstep: Convert to scoped user access powerpc/align: Convert emulate_spe() to scoped user access powerpc/ptrace: Convert gpr32_set_common_user() to scoped user access powerpc/futex: Use masked user access powerpc/audit: Convert powerpc to AUDIT_ARCH_COMPAT_GENERIC cpuidle: powerpc: avoid double clear when breaking snooze powerpc/ps3: spu.c: fix enum and Return kernel-doc warnings ...