diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-22 08:05:19 -1000 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-22 08:05:19 -1000 |
| commit | bb2235894a8a604c08b3f93f4e1082b9df03265a (patch) | |
| tree | c6e58a900d8e77472ecd8d5c7837985f2023055b | |
| parent | 979fac3eaaa02fa887c7dfe6509248b21dca3872 (diff) | |
| parent | 129f5d395fd66b6587a5ac156b232cca060d6512 (diff) | |
Merge tag 'powerpc-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
Pull powerpc updates from Madhavan Srinivasan:
- powerpc/numa: Support coregroup on PowerNV
- powerpc/irq: Move __softirq_pending out of irq_stat
- powerpc/512x: Remove pdm360ng platform setup in favor of mpc512x_generic
- powerpc/ps3: Simplify control flow in ps3_{virq,irq_plug}_setup()
- KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
- powerpc/syscall: Avoid atomic test_and_clear_thread_flag on fast path
- pseries/plpks cleanup and fixes
- Misc fixes and cleanup
Thanks to Abhishek Dubey, Amit Machhiwal, Anushree Mathur, Bui Nguyen
Nhu Quynh, Christophe Leroy (CS GROUP), Dmitry Torokhov, Gaurav Batra,
Gautam Menghani, Geert Uytterhoeven, Greg Joyce, Hari Bathini, Harsh
Prateek Bora, Karl Mehltretter, Link Mauve, Michal Suchánek, Mukesh
Kumar Chaurasiya (IBM), Nayna Jain, Ritesh Harjani (IBM), R Nageswara
Sastry, Saket Kumar Bhaskar, Samir Mulani, Shivang Upadhyay, Shrikanth
Hegde, Sourabh Jain, Srikar Dronamraju, Srish Srinivasan, Thorsten Blum,
Vaibhav Jain (IBM), Venkat Rao Bagalkote, and Yeswanth Krishna Tellakula
* tag 'powerpc-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux: (44 commits)
powerpc/numa: Support coregroup on PowerNV
powerpc/numa: Allow cpu_to_coregroup_id without PPC_SPLPAR
powerpc/numa: Simplify find_primary_domain_index
powerpc/fadump: collect dump when CPU_STATE_DATA is less than reserved
powerpc/syscall: Avoid atomic test_and_clear_thread_flag on fast path
powerpc/kasan: fix r2 corruption in mem*() on GENERIC_ENTRY with old compilers
powerpc: fix NMI exit ordering: restore soft masks after irqentry_nmi_exit()
powerpc/pseries/iommu: Add TCEs for 16GB pages when RAM is pre-mapped
KVM: PPC: Book3S HV: Free the queue pages in error path
KVM: PPC: Book3S HV: Add missing mappings for tracing exits
powerpc/interrupt: Use early_radix_enabled() in NMI real-mode guard
KVM: PPC: Fix get_d_signext() 12-bit displacement for paired-single D-form
KVM: PPC: Book3S HV: Add preempt_disable() around virtual-mode HPTE bit-lock users
KVM: PPC: Book3S HV: Add SRCU protection for virtual-mode HPT hcalls
pseries/plpks: update PKS documentation and MAINTAINERS entry
pseries/plpks: rename the default wrapping key macro
pseries/plpks: make narrowing conversions explicit
pseries/plpks: hide wrapping_features when unsupported
pseries/plpks: fix self-reference in plpks_var initializer
pseries/plpks: clear SED key data after use
...
46 files changed, 544 insertions, 572 deletions
diff --git a/Documentation/ABI/testing/ppc-memtrace b/Documentation/ABI/testing/ppc-memtrace index 9606aed33137..2de9fd18715e 100644 --- a/Documentation/ABI/testing/ppc-memtrace +++ b/Documentation/ABI/testing/ppc-memtrace @@ -3,7 +3,7 @@ Date: Aug 2017 KernelVersion: 4.14 Contact: linuxppc-dev@lists.ozlabs.org Description: This folder contains the relevant debugfs files for the - hardware trace macro to use. CONFIG_PPC64_HARDWARE_TRACING + hardware trace macro to use. CONFIG_PPC_MEMTRACE must be set. What: /sys/kernel/debug/powerpc/memtrace/enable diff --git a/Documentation/arch/powerpc/papr_hcalls.rst b/Documentation/arch/powerpc/papr_hcalls.rst index 14e39f095a1c..44c9c8b32ae3 100644 --- a/Documentation/arch/powerpc/papr_hcalls.rst +++ b/Documentation/arch/powerpc/papr_hcalls.rst @@ -305,8 +305,8 @@ like core instruction, core LLAT and nest. | Input: authorization, objectlabel, objectlabellen, policy, out, outlen | Out: *Hypervisor Generated Key, or None when the wrapping key policy is set* | Return Value: *H_SUCCESS, H_Function, H_State, H_R_State, H_Parameter, H_P2, - H_P3, H_P4, H_P5, H_P6, H_Authority, H_Nomem, H_Busy, H_Resource, - H_Aborted* + H_P3, H_P4, H_P5, H_P6, H_Authority, H_Nomem, H_Busy, + H_Resource, H_Aborted* H_PKS_GEN_KEY is used to have the hypervisor generate a new random key. This key is stored as an object in the Power LPAR Platform KeyStore with @@ -321,8 +321,8 @@ the user. Generation of wrapping keys is supported only for a key size of | inlen, out, outlen, continue-token | Out: *continue-token, byte size of wrapped object, wrapped object* | Return Value: *H_SUCCESS, H_Function, H_State, H_R_State, H_Parameter, H_P2, - H_P3, H_P4, H_P5, H_P6, H_P7, H_P8, H_P9, H_Authority, H_Invalid_Key, - H_NOT_FOUND, H_Busy, H_LongBusy, H_Aborted* + H_P3, H_P4, H_P5, H_P6, H_P7, H_P8, H_P9, H_Authority, + H_Invalid_Key, H_NOT_FOUND, H_Busy, H_LongBusy, H_Aborted* H_PKS_WRAP_OBJECT is used to wrap an object using a wrapping key stored in the Power LPAR Platform KeyStore and return the wrapped object to the caller. The @@ -331,16 +331,16 @@ which must have been previously created with H_PKS_GEN_KEY. The provided object is then encrypted with the wrapping key and additional metadata and the result is returned to the caller. - **H_PKS_UNWRAP_OBJECT** | Input: authorization, objectwrapflags, in, inlen, out, outlen, continue-token | Out: *continue-token, byte size of unwrapped object, unwrapped object* | Return Value: *H_SUCCESS, H_Function, H_State, H_R_State, H_Parameter, H_P2, - H_P3, H_P4, H_P5, H_P6, H_P7, H_Authority, H_Unsupported, H_Bad_Data, - H_NOT_FOUND, H_Invalid_Key, H_Busy, H_LongBusy, H_Aborted* + H_P3, H_P4, H_P5, H_P6, H_P7, H_Authority, H_Unsupported, + H_Bad_Data, H_NOT_FOUND, H_Invalid_Key, H_Busy, H_LongBusy, + H_Aborted* -H_PKS_UNWRAP_OBJECT is used to unwrap an object that was previously warapped with +H_PKS_UNWRAP_OBJECT is used to unwrap an object that was previously wrapped with H_PKS_WRAP_OBJECT. References diff --git a/MAINTAINERS b/MAINTAINERS index b4bfb698fd1e..b4c1b751698d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14555,7 +14555,7 @@ S: Supported F: include/keys/trusted_dcp.h F: security/keys/trusted-keys/trusted_dcp.c -KEYS-TRUSTED-PLPKS +KEYS-TRUSTED-PKWM M: Srish Srinivasan <ssrish@linux.ibm.com> M: Nayna Jain <nayna@linux.ibm.com> L: linux-integrity@vger.kernel.org diff --git a/arch/powerpc/boot/dts/ac14xx.dts b/arch/powerpc/boot/dts/ac14xx.dts index 5d8877e1f4ad..05fcf08df1ca 100644 --- a/arch/powerpc/boot/dts/ac14xx.dts +++ b/arch/powerpc/boot/dts/ac14xx.dts @@ -148,19 +148,6 @@ compatible = "fsl,mpc5121rev2-clock", "fsl,mpc5121-clock"; }; - /* - * GPIO PIC: - * interrupts cell = <pin nr, sense> - * sense == 8: Level, low assertion - * sense == 2: Edge, high-to-low change - */ - gpio_pic: gpio@1100 { - gpio-controller; - #gpio-cells = <2>; - interrupt-controller; - #interrupt-cells = <2>; - }; - sdhc@1500 { cd-gpios = <&gpio_pic 23 0>; /* card detect */ wp-gpios = <&gpio_pic 24 0>; /* write protect */ diff --git a/arch/powerpc/boot/dts/mpc5121.dtsi b/arch/powerpc/boot/dts/mpc5121.dtsi index a278fb7b9e71..9c5dacb94b92 100644 --- a/arch/powerpc/boot/dts/mpc5121.dtsi +++ b/arch/powerpc/boot/dts/mpc5121.dtsi @@ -146,10 +146,14 @@ interrupts = <83 0x8>; }; - gpio@1100 { + gpio_pic: gpio@1100 { compatible = "fsl,mpc5121-gpio"; reg = <0x1100 0x100>; interrupts = <78 0x8>; + gpio-controller; + #gpio-cells = <2>; + interrupt-controller; + #interrupt-cells = <2>; }; can@1300 { diff --git a/arch/powerpc/boot/dts/pdm360ng.dts b/arch/powerpc/boot/dts/pdm360ng.dts index 67c3b9db75d7..05e93ca294ee 100644 --- a/arch/powerpc/boot/dts/pdm360ng.dts +++ b/arch/powerpc/boot/dts/pdm360ng.dts @@ -9,6 +9,7 @@ * Copyright 2008 Freescale Semiconductor Inc. */ +#include <dt-bindings/gpio/gpio.h> #include "mpc5121.dtsi" / { @@ -176,11 +177,12 @@ /* ADS7845 touch screen controller */ ts@0 { - compatible = "ti,ads7846"; + compatible = "ti,ads7845"; reg = <0x0>; spi-max-frequency = <3000000>; - /* pen irq is GPIO25 */ - interrupts = <78 0x8>; + interrupt-parent = <&gpio_pic>; + interrupts = <25 0x8>; + pendown-gpio = <&gpio_pic 25 GPIO_ACTIVE_LOW>; }; }; diff --git a/arch/powerpc/include/asm/hardirq.h b/arch/powerpc/include/asm/hardirq.h index f133b5930ae1..be6cd5aab016 100644 --- a/arch/powerpc/include/asm/hardirq.h +++ b/arch/powerpc/include/asm/hardirq.h @@ -5,24 +5,33 @@ #include <linux/threads.h> #include <linux/irq.h> -typedef struct { - unsigned int __softirq_pending; - unsigned int timer_irqs_event; - unsigned int broadcast_irqs_event; - unsigned int timer_irqs_others; - unsigned int pmu_irqs; - unsigned int mce_exceptions; - unsigned int spurious_irqs; - unsigned int sreset_irqs; +enum irq_stat_counts { + IRQ_COUNT_LOC_TIMER, + IRQ_COUNT_BCT_TIMER, + IRQ_COUNT_OTHER_TIMER, + IRQ_COUNT_SPURIOUS, + IRQ_COUNT_PMI, + IRQ_COUNT_MCE, + IRQ_COUNT_NMI_SRESET, #ifdef CONFIG_PPC_WATCHDOG - unsigned int soft_nmi_irqs; + IRQ_COUNT_WATCHDOG, #endif #ifdef CONFIG_PPC_DOORBELL - unsigned int doorbell_irqs; + IRQ_COUNT_DOORBELL, #endif + IRQ_COUNT_MAX, +}; + +typedef struct { + unsigned int counts[IRQ_COUNT_MAX]; } ____cacheline_aligned irq_cpustat_t; DECLARE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat); +DECLARE_PER_CPU(unsigned int, __softirq_pending); +#define local_softirq_pending_ref __softirq_pending + +#define inc_irq_stat(index) __this_cpu_inc(irq_stat.counts[IRQ_COUNT_##index]) +void inc_irq_stat_and_enable(enum irq_stat_counts which); #define __ARCH_IRQ_STAT #define __ARCH_IRQ_EXIT_IRQS_DISABLED diff --git a/arch/powerpc/include/asm/interrupt.h b/arch/powerpc/include/asm/interrupt.h index 1b45a49e9bed..c7d0e71eac83 100644 --- a/arch/powerpc/include/asm/interrupt.h +++ b/arch/powerpc/include/asm/interrupt.h @@ -70,6 +70,7 @@ #include <linux/irq-entry-common.h> #include <asm/kprobes.h> +#include <asm/mmu.h> #include <asm/runlatch.h> #ifdef CONFIG_PPC_IRQ_SOFT_MASK_DEBUG @@ -290,7 +291,7 @@ interrupt_handler long func(struct pt_regs *regs) \ state = irqentry_nmi_enter(regs); \ } else if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && \ firmware_has_feature(FW_FEATURE_LPAR) && \ - !radix_enabled()) { \ + !early_radix_enabled()) { \ /* no nmi_entry for a pseries hash guest \ * taking a real mode exception */ \ } else if (IS_ENABLED(CONFIG_KASAN)) { \ @@ -301,13 +302,12 @@ interrupt_handler long func(struct pt_regs *regs) \ state = irqentry_nmi_enter(regs); \ } \ ret = ____##func (regs); \ - arch_interrupt_nmi_exit_prepare(regs, &nmi_state); \ if (mfmsr() & MSR_DR) { \ /* nmi_exit if relocations are on */ \ irqentry_nmi_exit(regs, state); \ } else if (IS_ENABLED(CONFIG_PPC_BOOK3S_64) && \ firmware_has_feature(FW_FEATURE_LPAR) && \ - !radix_enabled()) { \ + !early_radix_enabled()) { \ /* no nmi_exit for a pseries hash guest \ * taking a real mode exception */ \ } else if (IS_ENABLED(CONFIG_KASAN)) { \ @@ -317,6 +317,7 @@ interrupt_handler long func(struct pt_regs *regs) \ } else { \ irqentry_nmi_exit(regs, state); \ } \ + arch_interrupt_nmi_exit_prepare(regs, &nmi_state); \ \ return ret; \ } \ diff --git a/arch/powerpc/include/asm/kasan.h b/arch/powerpc/include/asm/kasan.h index a690e7da53c2..599a9e02af02 100644 --- a/arch/powerpc/include/asm/kasan.h +++ b/arch/powerpc/include/asm/kasan.h @@ -2,21 +2,6 @@ #ifndef __ASM_KASAN_H #define __ASM_KASAN_H -#if defined(CONFIG_KASAN) && !defined(CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX) -#define _GLOBAL_KASAN(fn) \ - _GLOBAL(fn); \ - _GLOBAL(__##fn) -#define _GLOBAL_TOC_KASAN(fn) \ - _GLOBAL_TOC(fn); \ - _GLOBAL_TOC(__##fn) -#define EXPORT_SYMBOL_KASAN(fn) \ - EXPORT_SYMBOL(__##fn) -#else /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ -#define _GLOBAL_KASAN(fn) _GLOBAL(fn) -#define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(fn) -#define EXPORT_SYMBOL_KASAN(fn) -#endif /* CONFIG_KASAN && !CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ - #ifndef __ASSEMBLER__ #include <asm/page.h> diff --git a/arch/powerpc/include/asm/string.h b/arch/powerpc/include/asm/string.h index 1981bd4036b5..161b949683b3 100644 --- a/arch/powerpc/include/asm/string.h +++ b/arch/powerpc/include/asm/string.h @@ -29,29 +29,10 @@ extern void * memchr(const void *,int,__kernel_size_t); void memcpy_flushcache(void *dest, const void *src, size_t size); #ifdef CONFIG_KASAN -/* __mem variants are used by KASAN to implement instrumented meminstrinsics. */ -#ifdef CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX +/* Used by mm/kasan as raw backends to bypass KASAN checking. */ #define __memset memset #define __memcpy memcpy #define __memmove memmove -#else /* CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ -void *__memset(void *s, int c, __kernel_size_t count); -void *__memcpy(void *to, const void *from, __kernel_size_t n); -void *__memmove(void *to, const void *from, __kernel_size_t n); -#ifndef __SANITIZE_ADDRESS__ -/* - * For files that are not instrumented (e.g. mm/slub.c) we - * should use not instrumented version of mem* functions. - */ -#define memcpy(dst, src, len) __memcpy(dst, src, len) -#define memmove(dst, src, len) __memmove(dst, src, len) -#define memset(s, c, n) __memset(s, c, n) - -#ifndef __NO_FORTIFY -#define __NO_FORTIFY /* FORTIFY_SOURCE uses __builtin_memcpy, etc. */ -#endif -#endif /* !__SANITIZE_ADDRESS__ */ -#endif /* CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX */ #endif /* CONFIG_KASAN */ #ifdef CONFIG_PPC64 diff --git a/arch/powerpc/include/asm/topology.h b/arch/powerpc/include/asm/topology.h index 44ec416fa489..02c3c9ab3ea9 100644 --- a/arch/powerpc/include/asm/topology.h +++ b/arch/powerpc/include/asm/topology.h @@ -71,6 +71,7 @@ extern void map_cpu_to_node(int cpu, int node); extern void unmap_cpu_from_node(unsigned long cpu); #endif /* CONFIG_HOTPLUG_CPU */ +int cpu_to_coregroup_id(int cpu); #else static inline int early_cpu_to_node(int cpu) { return 0; } @@ -107,14 +108,6 @@ static inline void map_cpu_to_node(int cpu, int node) {} static inline void unmap_cpu_from_node(unsigned long cpu) {} #endif /* CONFIG_HOTPLUG_CPU */ #endif /* CONFIG_SMP */ - -#endif /* CONFIG_NUMA */ - -#if defined(CONFIG_NUMA) && defined(CONFIG_PPC_SPLPAR) -void find_and_update_cpu_nid(int cpu); -extern int cpu_to_coregroup_id(int cpu); -#else -static inline void find_and_update_cpu_nid(int cpu) {} static inline int cpu_to_coregroup_id(int cpu) { #ifdef CONFIG_SMP @@ -124,6 +117,12 @@ static inline int cpu_to_coregroup_id(int cpu) #endif } +#endif /* CONFIG_NUMA */ + +#if defined(CONFIG_NUMA) && defined(CONFIG_PPC_SPLPAR) +void find_and_update_cpu_nid(int cpu); +#else +static inline void find_and_update_cpu_nid(int cpu) {} #endif /* CONFIG_NUMA && CONFIG_PPC_SPLPAR */ #include <asm-generic/topology.h> diff --git a/arch/powerpc/kernel/cputable.c b/arch/powerpc/kernel/cputable.c index 6f6801da9dc1..4731e27bb7bc 100644 --- a/arch/powerpc/kernel/cputable.c +++ b/arch/powerpc/kernel/cputable.c @@ -35,12 +35,7 @@ void __init set_cur_cpu_spec(struct cpu_spec *s) struct cpu_spec *t = &the_cpu_spec; t = PTRRELOC(t); - /* - * use memcpy() instead of *t = *s so that GCC replaces it - * by __memcpy() when KASAN is active - */ - memcpy(t, s, sizeof(*t)); - + *t = *s; *PTRRELOC(&cur_cpu_spec) = &the_cpu_spec; } @@ -53,11 +48,8 @@ static struct cpu_spec * __init setup_cpu_spec(unsigned long offset, t = PTRRELOC(t); old = *t; - /* - * Copy everything, then do fixups. Use memcpy() instead of *t = *s - * so that GCC replaces it by __memcpy() when KASAN is active - */ - memcpy(t, s, sizeof(*t)); + /* Copy everything, then do fixups. */ + *t = *s; /* * If we are overriding a previous value derived from the real diff --git a/arch/powerpc/kernel/dbell.c b/arch/powerpc/kernel/dbell.c index 5712dd846263..f5e298a4c4c0 100644 --- a/arch/powerpc/kernel/dbell.c +++ b/arch/powerpc/kernel/dbell.c @@ -31,7 +31,7 @@ DEFINE_INTERRUPT_HANDLER_ASYNC(doorbell_exception) do_hard_irq_enable(); kvmppc_clear_host_ipi(smp_processor_id()); - __this_cpu_inc(irq_stat.doorbell_irqs); + inc_irq_stat(DOORBELL); smp_ipi_demux_relaxed(); /* already performed the barrier */ diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c index 15a3c3fd8e70..bce546931a13 100644 --- a/arch/powerpc/kernel/irq.c +++ b/arch/powerpc/kernel/irq.c @@ -73,6 +73,7 @@ DEFINE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat); EXPORT_PER_CPU_SYMBOL(irq_stat); +DEFINE_PER_CPU(unsigned int, __softirq_pending); #ifdef CONFIG_PPC32 atomic_t ppc_n_lost_interrupts; @@ -83,105 +84,105 @@ u32 tau_interrupts(unsigned long cpu); #endif #endif /* CONFIG_PPC32 */ +struct irq_stat_info { + const char *symbol; + const char *text; + int optional; +}; + +/* ISE - IRQ STAT ENABLED, ISC - IRQ STAT CONDITIONAL */ +#define ISE(idx, sym, txt)[IRQ_COUNT_##idx] = { .symbol = sym, .text = txt, .optional = 0} +#define ISC(idx, sym, txt)[IRQ_COUNT_##idx] = { .symbol = sym, .text = txt, .optional = 1} + + +static struct irq_stat_info irq_stat_info[IRQ_COUNT_MAX] __ro_after_init = { + ISE(LOC_TIMER, "LOC", " Local timer interrupts for timer event device\n"), + ISE(BCT_TIMER, "BCT", " Broadcast timer interrupts for timer event device\n"), + ISE(OTHER_TIMER, "LOC", " Local timer interrupts for others\n"), + ISE(SPURIOUS, "SPU", " Spurious interrupts\n"), + ISE(PMI, "PMI", " Performance monitoring interrupts\n"), + ISC(MCE, "MCE", " Machine check exceptions\n"), + ISC(NMI_SRESET, "NMI", " System Reset interrupts\n"), +#ifdef CONFIG_PPC_WATCHDOG + ISE(WATCHDOG, "WDG", " Watchdog soft-NMI interrupts\n"), +#endif +#ifdef CONFIG_PPC_DOORBELL + ISE(DOORBELL, "DBL", " Doorbell interrupts\n"), +#endif +}; + +/* + * Used for default disabled counters to increment the stats and to enable the + * entry for /proc/interrupts output. + */ +static DECLARE_BITMAP(irq_stat_count_show, IRQ_COUNT_MAX) __read_mostly; +void inc_irq_stat_and_enable(enum irq_stat_counts which) +{ + __this_cpu_inc(irq_stat.counts[which]); + set_bit(which, irq_stat_count_show); +} + int arch_show_interrupts(struct seq_file *p, int prec) { - int j; + const struct irq_stat_info *info = irq_stat_info; + + for (unsigned int i = 0; i < ARRAY_SIZE(irq_stat_info); i++, info++) { + if (info->optional && !test_bit(i, irq_stat_count_show)) + continue; + + seq_printf(p, "%*s:", prec, info->symbol); + irq_proc_emit_counts(p, &irq_stat.counts[i]); + seq_puts(p, info->text); + } #if defined(CONFIG_PPC32) && defined(CONFIG_TAU_INT) if (tau_initialized) { + int j; seq_printf(p, "%*s:", prec, "TAU"); for_each_online_cpu(j) seq_put_decimal_ull_width(p, " ", tau_interrupts(j), 10); seq_puts(p, " PowerPC Thermal Assist (cpu temp)\n"); } #endif /* CONFIG_PPC32 && CONFIG_TAU_INT */ - - seq_printf(p, "%*s:", prec, "LOC"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).timer_irqs_event, 10); - seq_printf(p, " Local timer interrupts for timer event device\n"); - - seq_printf(p, "%*s:", prec, "BCT"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).broadcast_irqs_event, 10); - seq_printf(p, " Broadcast timer interrupts for timer event device\n"); - - seq_printf(p, "%*s:", prec, "LOC"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).timer_irqs_others, 10); - seq_printf(p, " Local timer interrupts for others\n"); - - seq_printf(p, "%*s:", prec, "SPU"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).spurious_irqs, 10); - seq_printf(p, " Spurious interrupts\n"); - - seq_printf(p, "%*s:", prec, "PMI"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).pmu_irqs, 10); - seq_printf(p, " Performance monitoring interrupts\n"); - - seq_printf(p, "%*s:", prec, "MCE"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).mce_exceptions, 10); - seq_printf(p, " Machine check exceptions\n"); - #ifdef CONFIG_PPC_BOOK3S_64 if (cpu_has_feature(CPU_FTR_HVMODE)) { + int j; seq_printf(p, "%*s:", prec, "HMI"); for_each_online_cpu(j) seq_put_decimal_ull_width(p, " ", paca_ptrs[j]->hmi_irqs, 10); seq_printf(p, " Hypervisor Maintenance Interrupts\n"); } #endif + return 0; +} - seq_printf(p, "%*s:", prec, "NMI"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).sreset_irqs, 10); - seq_printf(p, " System Reset interrupts\n"); - -#ifdef CONFIG_PPC_WATCHDOG - seq_printf(p, "%*s:", prec, "WDG"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).soft_nmi_irqs, 10); - seq_printf(p, " Watchdog soft-NMI interrupts\n"); -#endif +static int __init irq_init_stats(void) +{ + struct irq_stat_info *info = irq_stat_info; -#ifdef CONFIG_PPC_DOORBELL - if (cpu_has_feature(CPU_FTR_DBELL)) { - seq_printf(p, "%*s:", prec, "DBL"); - for_each_online_cpu(j) - seq_put_decimal_ull_width(p, " ", per_cpu(irq_stat, j).doorbell_irqs, 10); - seq_printf(p, " Doorbell interrupts\n"); + for (unsigned int i = 0; i < ARRAY_SIZE(irq_stat_info); i++, info++) { + if (info->optional == 0) + set_bit(i, irq_stat_count_show); } -#endif return 0; } +late_initcall(irq_init_stats); /* * /proc/stat helpers */ u64 arch_irq_stat_cpu(unsigned int cpu) { - u64 sum = per_cpu(irq_stat, cpu).timer_irqs_event; + irq_cpustat_t *p = per_cpu_ptr(&irq_stat, cpu); + u64 sum = 0; + + for (unsigned int i = 0; i < ARRAY_SIZE(irq_stat_info); i++) + sum += p->counts[i]; - sum += per_cpu(irq_stat, cpu).broadcast_irqs_event; - sum += per_cpu(irq_stat, cpu).pmu_irqs; - sum += per_cpu(irq_stat, cpu).mce_exceptions; - sum += per_cpu(irq_stat, cpu).spurious_irqs; - sum += per_cpu(irq_stat, cpu).timer_irqs_others; #ifdef CONFIG_PPC_BOOK3S_64 sum += paca_ptrs[cpu]->hmi_irqs; #endif - sum += per_cpu(irq_stat, cpu).sreset_irqs; -#ifdef CONFIG_PPC_WATCHDOG - sum += per_cpu(irq_stat, cpu).soft_nmi_irqs; -#endif -#ifdef CONFIG_PPC_DOORBELL - sum += per_cpu(irq_stat, cpu).doorbell_irqs; -#endif - return sum; } @@ -251,7 +252,7 @@ static void __do_irq(struct pt_regs *regs, unsigned long oldsp) /* And finally process it */ if (unlikely(!irq)) - __this_cpu_inc(irq_stat.spurious_irqs); + inc_irq_stat(SPURIOUS); else generic_handle_irq(irq); diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c index eb9f556b0937..e8e071024da5 100644 --- a/arch/powerpc/kernel/prom_init.c +++ b/arch/powerpc/kernel/prom_init.c @@ -1362,14 +1362,8 @@ static void __init prom_check_platform_support(void) int prop_len = prom_getproplen(prom.chosen, "ibm,arch-vec-5-platform-support"); - /* - * First copy the architecture vec template - * - * use memcpy() instead of *vec = *vec_template so that GCC replaces it - * by __memcpy() when KASAN is active - */ - memcpy(&ibm_architecture_vec, &ibm_architecture_vec_template, - sizeof(ibm_architecture_vec)); + /* First copy the architecture vec template */ + ibm_architecture_vec = ibm_architecture_vec_template; prom_strscpy_pad(ibm_architecture_vec.vec7.os_id, linux_banner, 256); diff --git a/arch/powerpc/kernel/prom_init_check.sh b/arch/powerpc/kernel/prom_init_check.sh index 3090b97258ae..3155cc722e48 100644 --- a/arch/powerpc/kernel/prom_init_check.sh +++ b/arch/powerpc/kernel/prom_init_check.sh @@ -13,21 +13,8 @@ # If you really need to reference something from prom_init.o add # it to the list below: -has_renamed_memintrinsics() -{ - grep -q "^CONFIG_KASAN=y$" "${KCONFIG_CONFIG}" && \ - ! grep -q "^CONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX=y" "${KCONFIG_CONFIG}" -} - -if has_renamed_memintrinsics -then - MEM_FUNCS="__memcpy __memset" -else - MEM_FUNCS="memcpy memset" -fi - WHITELIST="add_reloc_offset __bss_start __bss_stop copy_and_flush -_end enter_prom $MEM_FUNCS reloc_offset __secondary_hold +_end enter_prom memcpy memset reloc_offset __secondary_hold __secondary_hold_acknowledge __secondary_hold_spinloop __start logo_linux_clut224 btext_prepare_BAT reloc_got2 kernstart_addr memstart_addr linux_banner _stext diff --git a/arch/powerpc/kernel/syscall.c b/arch/powerpc/kernel/syscall.c index fbefe1927b10..8529827d697f 100644 --- a/arch/powerpc/kernel/syscall.c +++ b/arch/powerpc/kernel/syscall.c @@ -18,14 +18,12 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0) long ret; syscall_fn f; - if (unlikely(!syscall_enter_from_user_mode_randomize_stack(regs, &r0))) { + if (unlikely(!syscall_enter_from_user_mode_randomize_stack(regs, &r0) || + test_thread_flag(TIF_SYSCALL_RET))) { clear_thread_flag(TIF_SYSCALL_RET); return syscall_get_error(current, regs); } - if (unlikely(test_and_clear_thread_flag(TIF_SYSCALL_RET))) - return syscall_get_error(current, regs); - if (unlikely(r0 >= NR_syscalls)) { if (unlikely(trap_is_unsupported_scv(regs))) { /* Unsupported scv vector */ diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c index 329c1690b5ed..2100c5e2c821 100644 --- a/arch/powerpc/kernel/sysfs.c +++ b/arch/powerpc/kernel/sysfs.c @@ -338,7 +338,7 @@ static ssize_t show_pw20_wait_time(struct device *dev, time = pw20_wt; } - return sysfs_emit(buf, "%llu\n", time > 0 ? time : 0); + return sysfs_emit(buf, "%llu\n", time); } static void set_pw20_wait_entry_bit(void *val) @@ -460,7 +460,7 @@ static ssize_t show_altivec_idle_wait_time(struct device *dev, time = altivec_idle_wt; } - return sysfs_emit(buf, "%llu\n", time > 0 ? time : 0); + return sysfs_emit(buf, "%llu\n", time); } static void set_altivec_idle_wait_entry_bit(void *val) diff --git a/arch/powerpc/kernel/time.c b/arch/powerpc/kernel/time.c index 11145c40183d..e00dcda8240b 100644 --- a/arch/powerpc/kernel/time.c +++ b/arch/powerpc/kernel/time.c @@ -615,13 +615,13 @@ DEFINE_INTERRUPT_HANDLER_ASYNC(timer_interrupt) now = get_tb(); if (now >= *next_tb) { evt->event_handler(evt); - __this_cpu_inc(irq_stat.timer_irqs_event); + inc_irq_stat(LOC_TIMER); } else { now = *next_tb - now; if (now > decrementer_max) now = decrementer_max; set_dec_or_work(now); - __this_cpu_inc(irq_stat.timer_irqs_others); + inc_irq_stat(OTHER_TIMER); } trace_timer_interrupt_exit(regs); @@ -634,7 +634,7 @@ EXPORT_SYMBOL(timer_interrupt); void timer_broadcast_interrupt(void) { tick_receive_broadcast(); - __this_cpu_inc(irq_stat.broadcast_irqs_event); + inc_irq_stat(BCT_TIMER); } #endif diff --git a/arch/powerpc/kernel/trace/ftrace_entry.S b/arch/powerpc/kernel/trace/ftrace_entry.S index 6599fe3c6234..5eb8eee32549 100644 --- a/arch/powerpc/kernel/trace/ftrace_entry.S +++ b/arch/powerpc/kernel/trace/ftrace_entry.S @@ -220,7 +220,13 @@ /* Restore gprs */ .if \allregs == 1 +#ifdef CONFIG_PPC64 + REST_GPRS(2, 12, r1) + /* Do not restore a stale PACA pointer if the task migrated */ + REST_GPRS(14, 31, r1) +#else REST_GPRS(2, 31, r1) +#endif .else REST_GPRS(3, 10, r1) #if defined(CONFIG_LIVEPATCH_64) || defined(CONFIG_PPC_FTRACE_OUT_OF_LINE) diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c index 629f2a2d4780..3a8f5416e3f6 100644 --- a/arch/powerpc/kernel/traps.c +++ b/arch/powerpc/kernel/traps.c @@ -459,8 +459,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(system_reset_exception) } hv_nmi_check_nonrecoverable(regs); - - __this_cpu_inc(irq_stat.sreset_irqs); + inc_irq_stat_and_enable(IRQ_COUNT_NMI_SRESET); /* See if any machine dependent calls */ if (ppc_md.system_reset_exception) { @@ -817,7 +816,7 @@ static void __machine_check_exception(struct pt_regs *regs) { int recover = 0; - __this_cpu_inc(irq_stat.mce_exceptions); + inc_irq_stat_and_enable(IRQ_COUNT_MCE); add_taint(TAINT_MACHINE_CHECK, LOCKDEP_NOW_UNRELIABLE); @@ -1932,8 +1931,7 @@ DEFINE_INTERRUPT_HANDLER(vsx_unavailable_tm) DECLARE_INTERRUPT_HANDLER_NMI(performance_monitor_exception_nmi); DEFINE_INTERRUPT_HANDLER_NMI(performance_monitor_exception_nmi) { - __this_cpu_inc(irq_stat.pmu_irqs); - + inc_irq_stat(PMI); perf_irq(regs); return 0; @@ -1943,8 +1941,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(performance_monitor_exception_nmi) DECLARE_INTERRUPT_HANDLER_ASYNC(performance_monitor_exception_async); DEFINE_INTERRUPT_HANDLER_ASYNC(performance_monitor_exception_async) { - __this_cpu_inc(irq_stat.pmu_irqs); - + inc_irq_stat(PMI); perf_irq(regs); } diff --git a/arch/powerpc/kernel/watchdog.c b/arch/powerpc/kernel/watchdog.c index c40c69368476..582ee3fd71c1 100644 --- a/arch/powerpc/kernel/watchdog.c +++ b/arch/powerpc/kernel/watchdog.c @@ -381,7 +381,7 @@ DEFINE_INTERRUPT_HANDLER_NMI(soft_nmi_interrupt) if (!cpumask_test_cpu(cpu, &wd_cpus_enabled)) return 0; - __this_cpu_inc(irq_stat.soft_nmi_irqs); + inc_irq_stat(WATCHDOG); tb = get_tb(); if (tb - per_cpu(wd_timer_tb, cpu) >= wd_panic_timeout_tb) { diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index de05d721edc4..140eadb76dfe 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1159,6 +1159,38 @@ static long kvmppc_h_rpt_invalidate(struct kvm_vcpu *vcpu, return H_SUCCESS; } +static long kvmppc_pseries_do_hpt_hcall(struct kvm_vcpu *vcpu, unsigned long req) +{ + switch (req) { + case H_REMOVE: + return kvmppc_h_remove(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6)); + case H_ENTER: + return kvmppc_h_enter(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6), + kvmppc_get_gpr(vcpu, 7)); + case H_READ: + return kvmppc_h_read(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + case H_CLEAR_MOD: + return kvmppc_h_clear_mod(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + case H_CLEAR_REF: + return kvmppc_h_clear_ref(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + case H_PROTECT: + return kvmppc_h_protect(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6)); + case H_BULK_REMOVE: + return kvmppc_h_bulk_remove(vcpu); + } + + return H_FUNCTION; +} + int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; @@ -1174,47 +1206,17 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) switch (req) { case H_REMOVE: - ret = kvmppc_h_remove(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5), - kvmppc_get_gpr(vcpu, 6)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_ENTER: - ret = kvmppc_h_enter(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5), - kvmppc_get_gpr(vcpu, 6), - kvmppc_get_gpr(vcpu, 7)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_READ: - ret = kvmppc_h_read(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_CLEAR_MOD: - ret = kvmppc_h_clear_mod(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_CLEAR_REF: - ret = kvmppc_h_clear_ref(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_PROTECT: - ret = kvmppc_h_protect(vcpu, kvmppc_get_gpr(vcpu, 4), - kvmppc_get_gpr(vcpu, 5), - kvmppc_get_gpr(vcpu, 6)); - if (ret == H_TOO_HARD) - return RESUME_HOST; - break; case H_BULK_REMOVE: - ret = kvmppc_h_bulk_remove(vcpu); + preempt_disable(); + idx = srcu_read_lock(&kvm->srcu); + ret = kvmppc_pseries_do_hpt_hcall(vcpu, req); + srcu_read_unlock(&kvm->srcu, idx); + preempt_enable(); if (ret == H_TOO_HARD) return RESUME_HOST; break; @@ -1834,8 +1836,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, else vsid = vcpu->arch.fault_gpa; + preempt_disable(); err = kvmppc_hpte_hv_fault(vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, true); + preempt_enable(); if (err == 0) { r = RESUME_GUEST; } else if (err == -1 || err == -2) { @@ -1881,8 +1885,10 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, else vsid = vcpu->arch.fault_gpa; + preempt_disable(); err = kvmppc_hpte_hv_fault(vcpu, vcpu->arch.fault_dar, vsid, vcpu->arch.fault_dsisr, false); + preempt_enable(); if (err == 0) { r = RESUME_GUEST; } else if (err == -1) { diff --git a/arch/powerpc/kvm/book3s_paired_singles.c b/arch/powerpc/kvm/book3s_paired_singles.c index bc39c76c9d9f..532f96293de0 100644 --- a/arch/powerpc/kvm/book3s_paired_singles.c +++ b/arch/powerpc/kvm/book3s_paired_singles.c @@ -479,12 +479,7 @@ static bool kvmppc_inst_is_paired_single(struct kvm_vcpu *vcpu, u32 inst) static int get_d_signext(u32 inst) { - int d = inst & 0x8ff; - - if (d & 0x800) - return -(d & 0x7ff); - - return (d & 0x7ff); + return sign_extend32(inst & 0xfff, 11); } static int kvmppc_ps_three_in(struct kvm_vcpu *vcpu, bool rc, diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index 1d67237783b7..d348365df72d 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -961,9 +961,11 @@ static int xive_provision_queue(struct kvm_vcpu *vcpu, u8 prio) */ rc = xive_native_configure_queue(xc->vp_id, q, prio, qpage, xive->q_order, true); - if (rc) + if (rc) { pr_err("Failed to configure queue %d for VCPU %d\n", prio, xc->server_num); + free_pages((unsigned long)qpage, xive->q_page_order); + } return rc; } diff --git a/arch/powerpc/kvm/guest-state-buffer.c b/arch/powerpc/kvm/guest-state-buffer.c index 42843eca6727..aeb586ba4523 100644 --- a/arch/powerpc/kvm/guest-state-buffer.c +++ b/arch/powerpc/kvm/guest-state-buffer.c @@ -598,7 +598,7 @@ EXPORT_SYMBOL_GPL(kvmppc_gsm_fill_info); int kvmppc_gsm_refresh_info(struct kvmppc_gs_msg *gsm, struct kvmppc_gs_buff *gsb) { - if (!gsm->ops->fill_info) + if (!gsm->ops->refresh_info) return -EINVAL; return gsm->ops->refresh_info(gsm, gsb); diff --git a/arch/powerpc/kvm/trace_book3s.h b/arch/powerpc/kvm/trace_book3s.h index 9260ddbd557f..4ee35c6d01aa 100644 --- a/arch/powerpc/kvm/trace_book3s.h +++ b/arch/powerpc/kvm/trace_book3s.h @@ -7,6 +7,7 @@ */ #define kvm_trace_symbol_exit \ + {0x0, "RETURN_TO_HOST"}, \ {0x100, "SYSTEM_RESET"}, \ {0x200, "MACHINE_CHECK"}, \ {0x300, "DATA_STORAGE"}, \ @@ -28,6 +29,7 @@ {0xea0, "H_VIRT"}, \ {0xf00, "PERFMON"}, \ {0xf20, "ALTIVEC"}, \ - {0xf40, "VSX"} + {0xf40, "VSX"}, \ + {0xf80, "H_FAC_UNAVAIL"} #endif diff --git a/arch/powerpc/lib/copy_32.S b/arch/powerpc/lib/copy_32.S index 933b685e7ab6..97eb9ca0cc23 100644 --- a/arch/powerpc/lib/copy_32.S +++ b/arch/powerpc/lib/copy_32.S @@ -10,7 +10,6 @@ #include <asm/errno.h> #include <asm/ppc_asm.h> #include <asm/code-patching-asm.h> -#include <asm/kasan.h> #define COPY_16_BYTES \ lwz r7,4(r4); \ @@ -87,7 +86,7 @@ EXPORT_SYMBOL(memset16) * We therefore skip the optimised bloc that uses dcbz. This jump is * replaced by a nop once cache is active. This is done in machine_init() */ -_GLOBAL_KASAN(memset) +_GLOBAL(memset) cmplwi 0,r5,4 blt 7f @@ -147,7 +146,6 @@ _GLOBAL_KASAN(memset) bdnz 9b blr EXPORT_SYMBOL(memset) -EXPORT_SYMBOL_KASAN(memset) /* * This version uses dcbz on the complete cache lines in the @@ -160,12 +158,12 @@ EXPORT_SYMBOL_KASAN(memset) * We therefore jump to generic_memcpy which doesn't use dcbz. This jump is * replaced by a nop once cache is active. This is done in machine_init() */ -_GLOBAL_KASAN(memmove) +_GLOBAL(memmove) cmplw 0,r3,r4 bgt backwards_memcpy /* fall through */ -_GLOBAL_KASAN(memcpy) +_GLOBAL(memcpy) 1: b generic_memcpy patch_site 1b, patch__memcpy_nocache @@ -241,8 +239,6 @@ _GLOBAL_KASAN(memcpy) 65: blr EXPORT_SYMBOL(memcpy) EXPORT_SYMBOL(memmove) -EXPORT_SYMBOL_KASAN(memcpy) -EXPORT_SYMBOL_KASAN(memmove) generic_memcpy: srwi. r7,r5,3 diff --git a/arch/powerpc/lib/mem_64.S b/arch/powerpc/lib/mem_64.S index 6fd06cd20faa..40eaedd31486 100644 --- a/arch/powerpc/lib/mem_64.S +++ b/arch/powerpc/lib/mem_64.S @@ -8,7 +8,6 @@ #include <asm/processor.h> #include <asm/errno.h> #include <asm/ppc_asm.h> -#include <asm/kasan.h> #ifndef CONFIG_KASAN _GLOBAL(__memset16) @@ -29,7 +28,7 @@ EXPORT_SYMBOL(__memset32) EXPORT_SYMBOL(__memset64) #endif -_GLOBAL_KASAN(memset) +_GLOBAL(memset) neg r0,r3 rlwimi r4,r4,8,16,23 andi. r0,r0,7 /* # bytes to be 8-byte aligned */ @@ -95,9 +94,8 @@ _GLOBAL_KASAN(memset) stb r4,0(r6) blr EXPORT_SYMBOL(memset) -EXPORT_SYMBOL_KASAN(memset) -_GLOBAL_TOC_KASAN(memmove) +_GLOBAL_TOC(memmove) cmplw 0,r3,r4 bgt backwards_memcpy b memcpy @@ -139,4 +137,3 @@ _GLOBAL(backwards_memcpy) mtctr r7 b 1b EXPORT_SYMBOL(memmove) -EXPORT_SYMBOL_KASAN(memmove) diff --git a/arch/powerpc/lib/memcpy_64.S b/arch/powerpc/lib/memcpy_64.S index b5a67e20143f..0cedd455231a 100644 --- a/arch/powerpc/lib/memcpy_64.S +++ b/arch/powerpc/lib/memcpy_64.S @@ -7,7 +7,6 @@ #include <asm/ppc_asm.h> #include <asm/asm-compat.h> #include <asm/feature-fixups.h> -#include <asm/kasan.h> #ifndef SELFTEST_CASE /* For big-endian, 0 == most CPUs, 1 == POWER6, 2 == Cell */ @@ -15,7 +14,7 @@ #endif .align 7 -_GLOBAL_TOC_KASAN(memcpy) +_GLOBAL_TOC(memcpy) BEGIN_FTR_SECTION #ifdef __LITTLE_ENDIAN__ cmpdi cr7,r5,0 @@ -227,4 +226,3 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_LD_STD) blr #endif EXPORT_SYMBOL(memcpy) -EXPORT_SYMBOL_KASAN(memcpy) diff --git a/arch/powerpc/lib/sstep.c b/arch/powerpc/lib/sstep.c index f0d6aa657c1a..d49b2ef5d763 100644 --- a/arch/powerpc/lib/sstep.c +++ b/arch/powerpc/lib/sstep.c @@ -47,7 +47,7 @@ extern void conv_sp_to_dp(const float *sp, double *dp); extern void conv_dp_to_sp(const double *dp, float *sp); #endif -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 /* * Functions in quad.S */ @@ -134,7 +134,7 @@ static nokprobe_inline unsigned long dform_ea(unsigned int instr, return ea; } -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 /* * Calculate effective address for a DS-form instruction */ @@ -168,7 +168,7 @@ static nokprobe_inline unsigned long dqform_ea(unsigned int instr, return ea; } -#endif /* __powerpc64 */ +#endif /* CONFIG_PPC64 */ /* * Calculate effective address for an X-form instruction @@ -250,7 +250,7 @@ static nokprobe_inline unsigned long byterev_4(unsigned long x) ((x & 0xff00) << 8) | ((x & 0xff) << 24); } -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 static nokprobe_inline unsigned long byterev_8(unsigned long x) { return (byterev_4(x) << 32) | byterev_4(x >> 32); @@ -266,7 +266,7 @@ static nokprobe_inline void do_byte_reverse(void *ptr, int nb) case 4: *(u32 *)ptr = byterev_4(*(u32 *)ptr); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: *(unsigned long *)ptr = byterev_8(*(unsigned long *)ptr); break; @@ -312,7 +312,7 @@ __read_mem_aligned(unsigned long *dest, unsigned long ea, int nb, struct pt_regs case 4: unsafe_get_user(x, (unsigned int __user *)ea, Efault); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: unsafe_get_user(x, (unsigned long __user *)ea, Efault); break; @@ -364,7 +364,7 @@ static __always_inline int __copy_mem_in(u8 *dest, unsigned long ea, int nb, str case 4: unsafe_get_user(*(u32 *)dest, (u32 __user *)ea, Efault); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: unsafe_get_user(*(u64 *)dest, (u64 __user *)ea, Efault); break; @@ -443,7 +443,7 @@ __write_mem_aligned(unsigned long val, unsigned long ea, int nb, struct pt_regs case 4: unsafe_put_user(val, (unsigned int __user *)ea, Efault); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: unsafe_put_user(val, (unsigned long __user *)ea, Efault); break; @@ -494,7 +494,7 @@ static __always_inline int __copy_mem_out(u8 *dest, unsigned long ea, int nb, st case 4: unsafe_put_user(*(u32 *)dest, (u32 __user *)ea, Efault); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: unsafe_put_user(*(u64 *)dest, (u64 __user *)ea, Efault); break; @@ -722,7 +722,7 @@ static nokprobe_inline int do_vec_store(int rn, unsigned long ea, } #endif /* CONFIG_ALTIVEC */ -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 static nokprobe_inline int emulate_lq(struct pt_regs *regs, unsigned long ea, int reg, bool cross_endian) { @@ -765,7 +765,7 @@ static nokprobe_inline int emulate_stq(struct pt_regs *regs, unsigned long ea, err = write_mem(vals[IS_BE], ea + 8, 8, regs); return err; } -#endif /* __powerpc64 */ +#endif /* CONFIG_PPC64 */ #ifdef CONFIG_VSX static nokprobe_inline void emulate_vsx_load(struct instruction_op *op, union vsx_reg *reg, @@ -1316,7 +1316,7 @@ static nokprobe_inline int trap_compare(long v1, long v2) */ #define MASK32(mb, me) ((0xffffffffUL >> (mb)) + \ ((signed long)-0x80000000L >> (me)) + ((me) >= (mb))) -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 #define MASK64_L(mb) (~0UL >> (mb)) #define MASK64_R(me) ((signed long)-0x8000000000000000L >> (me)) #define MASK64(mb, me) (MASK64_L(mb) + MASK64_R(me) + ((me) >= (mb))) @@ -1342,16 +1342,19 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, { #ifdef CONFIG_PPC64 unsigned int suffixopcode, prefixtype, prefix_r; + unsigned int rc, suffix; #endif - unsigned int opcode, ra, rb, rc, rd, spr, u; + unsigned int opcode, ra, rb, rd, spr, u; unsigned long int imm; unsigned long int val, val2; unsigned int mb, me, sh; - unsigned int word, suffix; + unsigned int word; long ival; word = ppc_inst_val(instr); +#ifdef CONFIG_PPC64 suffix = ppc_inst_suffix(instr); +#endif op->type = COMPUTE; @@ -1447,7 +1450,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, switch ((word >> 1) & 0x3ff) { case 598: /* sync */ op->type = BARRIER + BARRIER_SYNC; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 switch ((word >> 21) & 3) { case 1: /* lwsync */ op->type = BARRIER + BARRIER_LWSYNC; @@ -1469,10 +1472,12 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, rd = (word >> 21) & 0x1f; ra = (word >> 16) & 0x1f; rb = (word >> 11) & 0x1f; +#ifdef CONFIG_PPC64 rc = (word >> 6) & 0x1f; +#endif switch (opcode) { -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 1: if (!cpu_has_feature(CPU_FTR_ARCH_31)) goto unknown_opcode; @@ -1506,7 +1511,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, goto trap; return 1; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 4: /* * There are very many instructions with this primary opcode @@ -1555,7 +1560,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 10: /* cmpli */ imm = (unsigned short) word; val = regs->gpr[ra]; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if ((rd & 1) == 0) val = (unsigned int) val; #endif @@ -1565,7 +1570,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 11: /* cmpi */ imm = (short) word; val = regs->gpr[ra]; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if ((rd & 1) == 0) val = (int) val; #endif @@ -1662,7 +1667,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, set_cr0(regs, op); goto logical_done_nocc; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 30: /* rld* */ mb = ((word >> 6) & 0x1f) | (word & 0x20); val = regs->gpr[rd]; @@ -1720,7 +1725,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, (int)regs->gpr[rb]))) goto trap; return 1; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 68: /* td */ if (rd & trap_compare(regs->gpr[ra], regs->gpr[rb])) goto trap; @@ -1826,7 +1831,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 0: /* cmp */ val = regs->gpr[ra]; val2 = regs->gpr[rb]; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if ((rd & 1) == 0) { /* word (32-bit) compare */ val = (int) val; @@ -1839,7 +1844,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 32: /* cmpl */ val = regs->gpr[ra]; val2 = regs->gpr[rb]; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if ((rd & 1) == 0) { /* word (32-bit) compare */ val = (unsigned int) val; @@ -1860,7 +1865,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, add_with_carry(regs, op, rd, ~regs->gpr[ra], regs->gpr[rb], 1); goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 9: /* mulhdu */ asm("mulhdu %0,%1,%2" : "=r" (op->val) : "r" (regs->gpr[ra]), "r" (regs->gpr[rb])); @@ -1879,7 +1884,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 40: /* subf */ op->val = regs->gpr[rb] - regs->gpr[ra]; goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 73: /* mulhd */ asm("mulhd %0,%1,%2" : "=r" (op->val) : "r" (regs->gpr[ra]), "r" (regs->gpr[rb])); @@ -1918,7 +1923,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, add_with_carry(regs, op, rd, ~regs->gpr[ra], -1L, regs->xer & XER_CA); goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 233: /* mulld */ op->val = regs->gpr[ra] * regs->gpr[rb]; goto arith_done; @@ -1933,7 +1938,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, (int) regs->gpr[rb]; goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 265: /* modud */ if (!cpu_has_feature(CPU_FTR_ARCH_300)) goto unknown_opcode; @@ -1950,7 +1955,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->val = (unsigned int) regs->gpr[ra] % (unsigned int) regs->gpr[rb]; goto compute_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 457: /* divdu */ op->val = regs->gpr[ra] / regs->gpr[rb]; goto arith_done; @@ -1959,7 +1964,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->val = (unsigned int) regs->gpr[ra] / (unsigned int) regs->gpr[rb]; goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 489: /* divd */ op->val = (long int) regs->gpr[ra] / (long int) regs->gpr[rb]; @@ -1969,7 +1974,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->val = (int) regs->gpr[ra] / (int) regs->gpr[rb]; goto arith_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 425: /* divde[.] */ asm volatile(PPC_DIVDE(%0, %1, %2) : "=r" (op->val) : "r" (regs->gpr[ra]), @@ -2002,7 +2007,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, } goto unknown_opcode; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 777: /* modsd */ if (!cpu_has_feature(CPU_FTR_ARCH_300)) goto unknown_opcode; @@ -2025,7 +2030,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, val = (unsigned int) regs->gpr[rd]; op->val = ( val ? __builtin_clz(val) : 32 ); goto logical_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 58: /* cntlzd */ val = regs->gpr[rd]; op->val = ( val ? __builtin_clzl(val) : 64 ); @@ -2093,7 +2098,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, val = (unsigned int) regs->gpr[rd]; op->val = (val ? __builtin_ctz(val) : 32); goto logical_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 570: /* cnttzd */ if (!cpu_has_feature(CPU_FTR_ARCH_300)) goto unknown_opcode; @@ -2108,7 +2113,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 954: /* extsb */ op->val = (signed char) regs->gpr[rd]; goto logical_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 986: /* extsw */ op->val = (signed int) regs->gpr[rd]; goto logical_done; @@ -2159,7 +2164,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, set_ca32(op, op->xerval & XER_CA); goto logical_done; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 27: /* sld */ sh = regs->gpr[rb] & 0x7f; if (sh < 64) @@ -2216,7 +2221,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->val = val; goto logical_done; -#endif /* __powerpc64__ */ +#endif /* CONFIG_PPC64 */ /* * Cache instructions @@ -2296,7 +2301,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->type = MKOP(STCX, 0, 2); break; #endif -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 84: /* ldarx */ op->type = MKOP(LARX, 0, 8); break; @@ -2373,7 +2378,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, break; #endif /* CONFIG_ALTIVEC */ -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 21: /* ldx */ case 53: /* ldux */ op->type = MKOP(LOAD, u, 8); @@ -2400,7 +2405,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->type = MKOP(LOAD, u, 2); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 341: /* lwax */ case 373: /* lwaux */ op->type = MKOP(LOAD, SIGNEXT | u, 4); @@ -2417,7 +2422,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->type = MKOP(STORE, u, 2); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 532: /* ldbrx */ op->type = MKOP(LOAD, BYTEREV, 8); break; @@ -2459,7 +2464,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, op->type = MKOP(STORE_FP, u, 8); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 791: /* lfdpx */ op->type = MKOP(LOAD_FP, 0, 16); break; @@ -2479,10 +2484,10 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 983: /* stfiwx */ op->type = MKOP(STORE_FP, 0, 4); break; -#endif /* __powerpc64 */ +#endif /* CONFIG_PPC64 */ #endif /* CONFIG_PPC_FPU */ -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 660: /* stdbrx */ op->type = MKOP(STORE, BYTEREV, 8); op->val = byterev_8(regs->gpr[rd]); @@ -2820,7 +2825,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, break; #endif -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 56: /* lq */ if (!((rd & 1) || (rd == ra))) op->type = MKOP(LOAD, 0, 16); @@ -2857,7 +2862,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, break; #endif /* CONFIG_VSX */ -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 58: /* ld[u], lwa */ op->ea = dsform_ea(word, regs); switch (word & 3) { @@ -2946,7 +2951,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, break; #endif /* CONFIG_VSX */ -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 62: /* std[u] */ op->ea = dsform_ea(word, regs); switch (word & 3) { @@ -3099,7 +3104,7 @@ int analyse_instr(struct instruction_op *op, const struct pt_regs *regs, case 3: /* Type 11 Modified Register-to-Register */ break; } -#endif /* __powerpc64__ */ +#endif /* CONFIG_PPC64 */ } @@ -3200,7 +3205,7 @@ static nokprobe_inline void do_byterev(unsigned long *valp, int size) case 4: *valp = byterev_4(*valp); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: *valp = byterev_8(*valp); break; @@ -3341,7 +3346,7 @@ int emulate_loadstore(struct pt_regs *regs, struct instruction_op *op) case 4: __get_user_asmx(val, ea, err, "lwarx"); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: __get_user_asmx(val, ea, err, "ldarx"); break; @@ -3367,7 +3372,7 @@ int emulate_loadstore(struct pt_regs *regs, struct instruction_op *op) return -EFAULT; err = 0; switch (size) { -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 1: __put_user_asmx(op->val, ea, err, "stbcx.", cr); break; @@ -3378,7 +3383,7 @@ int emulate_loadstore(struct pt_regs *regs, struct instruction_op *op) case 4: __put_user_asmx(op->val, ea, err, "stwcx.", cr); break; -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 case 8: __put_user_asmx(op->val, ea, err, "stdcx.", cr); break; @@ -3399,7 +3404,7 @@ int emulate_loadstore(struct pt_regs *regs, struct instruction_op *op) break; case LOAD: -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if (size == 16) { err = emulate_lq(regs, ea, op->reg, cross_endian); break; @@ -3473,7 +3478,7 @@ int emulate_loadstore(struct pt_regs *regs, struct instruction_op *op) break; case STORE: -#ifdef __powerpc64__ +#ifdef CONFIG_PPC64 if (size == 16) { err = emulate_stq(regs, ea, op->reg, cross_endian); break; diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index f4cf3ae036de..e97b624203ea 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -432,7 +432,7 @@ static void __init initialize_form2_numa_distance_lookup_table(void) static int __init find_primary_domain_index(void) { - int index; + int index = -1; struct device_node *root; /* @@ -502,12 +502,9 @@ static int __init find_primary_domain_index(void) distance_ref_points_depth = MAX_DISTANCE_REF_POINTS; } - of_node_put(root); - return index; - err: of_node_put(root); - return -1; + return index; } static void __init get_n_mem_cells(int *n_addr_cells, int *n_size_cells) @@ -892,12 +889,32 @@ static int __init numa_setup_drmem_lmb(struct drmem_lmb *lmb, return 0; } +/* + * If hierarchy extends beyond primary_domain_index + 1, then next + * level corresponds to coregroup. + */ +static int detect_and_enable_coregroup(const __be32 *associativity, int index) +{ + if (!associativity || index == -1) + goto out; + + index = of_read_number(associativity, 1); + + if (index > primary_domain_index + 1) { + coregroup_enabled = 1; + return index; + } +out: + coregroup_enabled = 0; + return -1; +} + static int __init parse_numa_properties(void) { struct device_node *memory, *pci; - int default_nid = 0; - unsigned long i; + int default_nid = 0, index = 0; const __be32 *associativity; + unsigned long i; if (numa_enabled == 0) { pr_warn("disabled by user\n"); @@ -930,7 +947,6 @@ static int __init parse_numa_properties(void) */ for_each_present_cpu(i) { __be32 vphn_assoc[VPHN_ASSOC_BUFSIZE]; - struct device_node *cpu; int nid = NUMA_NO_NODE; memset(vphn_assoc, 0, VPHN_ASSOC_BUFSIZE * sizeof(__be32)); @@ -938,7 +954,9 @@ static int __init parse_numa_properties(void) if (__vphn_get_associativity(i, vphn_assoc) == 0) { nid = associativity_to_nid(vphn_assoc); initialize_form1_numa_distance(vphn_assoc); + index = detect_and_enable_coregroup(vphn_assoc, index); } else { + struct device_node *cpu; /* * Don't fall back to default_nid yet -- we will plug @@ -951,6 +969,7 @@ static int __init parse_numa_properties(void) associativity = of_get_associativity(cpu); if (associativity) { nid = associativity_to_nid(associativity); + index = detect_and_enable_coregroup(associativity, index); initialize_form1_numa_distance(associativity); } of_node_put(cpu); @@ -1431,9 +1450,26 @@ void find_and_update_cpu_nid(int cpu) pr_debug("%s:%d cpu %d nid %d\n", __func__, __LINE__, cpu, new_nid); } +static int topology_update_init(void) +{ + topology_inited = 1; + return 0; +} +device_initcall(topology_update_init); + +#else +static long vphn_get_associativity(unsigned long cpu, + __be32 *associativity) +{ + return -1; +} +#endif /* CONFIG_PPC_SPLPAR */ + int cpu_to_coregroup_id(int cpu) { - __be32 associativity[VPHN_ASSOC_BUFSIZE] = {0}; + int coregroup_id = cpu_to_core_id(cpu); + struct device_node *cpunode = NULL; + const __be32 *associativity; int index; if (cpu < 0 || cpu > nr_cpu_ids) @@ -1442,24 +1478,31 @@ int cpu_to_coregroup_id(int cpu) if (!coregroup_enabled) goto out; - if (!firmware_has_feature(FW_FEATURE_VPHN)) - goto out; + if (firmware_has_feature(FW_FEATURE_VPHN)) { + __be32 tmp[VPHN_ASSOC_BUFSIZE] = {0}; - if (vphn_get_associativity(cpu, associativity)) + if (vphn_get_associativity(cpu, tmp)) + goto out; + + associativity = tmp; + + } else { + cpunode = of_get_cpu_node(cpu, NULL); + if (!cpunode) + goto out; + + associativity = of_get_associativity(cpunode); + } + if (!associativity) goto out; index = of_read_number(associativity, 1); if (index > primary_domain_index + 1) - return of_read_number(&associativity[index - 1], 1); + coregroup_id = of_read_number(&associativity[index - 1], 1); out: - return cpu_to_core_id(cpu); -} + if (cpunode) + of_node_put(cpunode); -static int topology_update_init(void) -{ - topology_inited = 1; - return 0; + return coregroup_id; } -device_initcall(topology_update_init); -#endif /* CONFIG_PPC_SPLPAR */ diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index f32de8704d4d..4da8bde92e1e 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -14,6 +14,13 @@ #include <asm/ppc-opcode.h> #include <linux/build_bug.h> +/* + * We need at least 2 passes for proper code generation, and may need + * additional passes if code size changes between passes. + */ +#define CODEGEN_MIN_PASSES 2 +#define CODEGEN_MAX_PASSES 3 + #ifdef CONFIG_PPC64_ELF_ABI_V1 #define FUNCTION_DESCR_SIZE 24 #else @@ -188,6 +195,12 @@ struct codegen_context { #define bpf_to_ppc(r) (ctx->b2p[r]) +#ifdef CONFIG_PPC64 +#define PPC_RAW_CMPLLI(a, i) PPC_RAW_CMPLDI(a, i) +#else +#define PPC_RAW_CMPLLI(a, i) PPC_RAW_CMPLWI(a, i) +#endif + #ifdef CONFIG_PPC32 #define BPF_FIXUP_LEN 3 /* Three instructions => 12 bytes */ #else @@ -214,10 +227,11 @@ int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context * int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct codegen_context *ctx, u32 *addrs, int pass, bool extra_pass); void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx); -void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx); -void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx); +void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx); +void bpf_jit_build_fentry_stubs(u32 *image, u32 *fimage, struct codegen_context *ctx); void bpf_jit_realloc_regs(struct codegen_context *ctx); -int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr); +int bpf_jit_emit_exit_insn(u32 *image, u32 *fimage, struct codegen_context *ctx, int tmp_reg, + long exit_addr); void prepare_for_fsession_fentry(u32 *image, struct codegen_context *ctx, int cookie_cnt, int cookie_off, int retval_off); void store_func_meta(u32 *image, struct codegen_context *ctx, u64 func_meta, int func_meta_off); diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index 7a612688e7a2..cfe01c3544db 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -49,11 +49,35 @@ asm ( " .popsection ;" ); -void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx) +void bpf_jit_build_fentry_stubs(u32 *image, u32 *fimage, struct codegen_context *ctx) { int ool_stub_idx, long_branch_stub_idx; + int stub_sz; /* + * The dummy_tramp_addr field is placed at bottom of Long branch stub. + * Align the mis-aligned dummy_tramp_addr field in the fimage. + * The alignment NOP must appear before OOL stub, to make + * ool_stub_idx & long_branch_stub_idx constant from end. + * + * The fimage can be non 8-byte aligned, so final alignment depends + * on start of fimage and the stub's instruction count. The + * stubs block has 11 instructions (with CONFIG_PPC_FTRACE_OUT_OF_LINE) + * or 10 instructions (without) before dummy_tramp_addr field. Emit a + * NOP if the address of dummy_tramp_addr is non aligned. + * + * In pass=0 when image==NULL, conservatively account for space + * required to accommodate alignment NOP. In case final pass skips + * emitting alignment NOP, the image buffer have 4 spare bytes and + * jited_len signifies correct program size. + */ + + stub_sz = IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE) ? 44 : 40; + if (!image || !IS_ALIGNED((unsigned long)fimage + ctx->idx*4 + stub_sz, SZL)) + EMIT(PPC_RAW_NOP()); + + /* + * nop // optional, for alignment of dummy_tramp_addr * Out-of-line stub: * mflr r0 * [b|bl] tramp @@ -70,44 +94,46 @@ void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx) /* * Long branch stub: - * .long <dummy_tramp_addr> * mflr r11 * bcl 20,31,$+4 - * mflr r12 - * ld r12, -8-SZL(r12) + * mflr r12 // lr/r12 stores pc of current(this) inst. + * ld r12, 20(r12) // offset(dummy_tramp_addr) from prev inst. is 20 * mtctr r12 - * mtlr r11 // needed to retain ftrace ABI + * mtlr r11 // needed to retain ftrace ABI * bctr + * .long <dummy_tramp_addr> // SZL bytes aligned */ - if (image) - *((unsigned long *)&image[ctx->idx]) = (unsigned long)dummy_tramp; - ctx->idx += SZL / 4; long_branch_stub_idx = ctx->idx; EMIT(PPC_RAW_MFLR(_R11)); EMIT(PPC_RAW_BCL4()); EMIT(PPC_RAW_MFLR(_R12)); - EMIT(PPC_RAW_LL(_R12, _R12, -8-SZL)); + EMIT(PPC_RAW_LL(_R12, _R12, 20)); EMIT(PPC_RAW_MTCTR(_R12)); EMIT(PPC_RAW_MTLR(_R11)); EMIT(PPC_RAW_BCTR()); + if (image) + *((unsigned long *)&image[ctx->idx]) = (unsigned long)dummy_tramp; + + ctx->idx += SZL / 4; + if (!bpf_jit_ool_stub) { bpf_jit_ool_stub = (ctx->idx - ool_stub_idx) * 4; bpf_jit_long_branch_stub = (ctx->idx - long_branch_stub_idx) * 4; } } -int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr) +int bpf_jit_emit_exit_insn(u32 *image, u32 *fimage, struct codegen_context *ctx, + int tmp_reg, long exit_addr) { - if (!exit_addr || is_offset_in_branch_range(exit_addr - (ctx->idx * 4))) { + if (exit_addr && is_offset_in_branch_range(exit_addr - (long)(ctx->idx * 4))) { PPC_JMP(exit_addr); - } else if (ctx->alt_exit_addr) { - if (WARN_ON(!is_offset_in_branch_range((long)ctx->alt_exit_addr - (ctx->idx * 4)))) - return -1; + } else if (ctx->alt_exit_addr && is_offset_in_branch_range( + (long)(ctx->alt_exit_addr) - (long)(ctx->idx * 4))) { PPC_JMP(ctx->alt_exit_addr); } else { ctx->alt_exit_addr = ctx->idx * 4; - bpf_jit_build_epilogue(image, ctx); + bpf_jit_build_epilogue(image, fimage, ctx); } return 0; @@ -274,6 +300,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr */ if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) { cgctx.idx = 0; + cgctx.alt_exit_addr = 0; if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) goto out_err; } @@ -286,7 +313,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr */ bpf_jit_build_prologue(NULL, &cgctx); addrs[fp->len] = cgctx.idx * 4; - bpf_jit_build_epilogue(NULL, &cgctx); + bpf_jit_build_epilogue(NULL, NULL, &cgctx); fixup_len = fp->aux->num_exentries * BPF_FIXUP_LEN * 4; extable_len = fp->aux->num_exentries * sizeof(struct exception_table_entry); @@ -306,10 +333,13 @@ skip_init_ctx: code_base = (u32 *)(image + FUNCTION_DESCR_SIZE); fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE); - /* Code generation passes 1-2 */ - for (pass = 1; pass < 3; pass++) { + /* Code generation passes 1-2+, loop until program size converges. */ + for (pass = 1; pass <= CODEGEN_MAX_PASSES; pass++) { + u32 prev_proglen = proglen; + /* Now build the prologue, body code & epilogue for real. */ cgctx.idx = 0; + cgctx.exentry_idx = 0; cgctx.alt_exit_addr = 0; bpf_jit_build_prologue(code_base, &cgctx); if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass, @@ -318,11 +348,26 @@ skip_init_ctx: bpf_jit_binary_pack_free(fhdr, hdr); goto out_err; } - bpf_jit_build_epilogue(code_base, &cgctx); + addrs[fp->len] = cgctx.idx * 4; + bpf_jit_build_epilogue(code_base, fcode_base, &cgctx); + + proglen = cgctx.idx * 4; if (bpf_jit_enable > 1) pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass, - proglen - (cgctx.idx * 4), cgctx.seen); + prev_proglen - proglen, cgctx.seen); + + /* Check if program size has converged, but ensure minimum passes */ + if (pass >= CODEGEN_MIN_PASSES && proglen == prev_proglen) + break; + + if (pass == CODEGEN_MAX_PASSES && proglen != prev_proglen) { + pr_err("BPF JIT: Program did not converge after %d passes\n", + CODEGEN_MAX_PASSES); + bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size)); + bpf_jit_binary_pack_free(fhdr, hdr); + goto out_err; + } } if (bpf_jit_enable > 1) @@ -357,7 +402,7 @@ skip_init_ctx: (void *)fimage + FUNCTION_DESCR_SIZE); out_addrs: - if (!image && priv_stack_ptr) { + if (!fp->jited && priv_stack_ptr) { fp->aux->priv_stack_ptr = NULL; free_percpu(priv_stack_ptr); } @@ -399,7 +444,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass u32 *fixup; /* Populate extable entries only in the last pass */ - if (pass != 2) + if (pass < CODEGEN_MIN_PASSES) return 0; if (!fp->aux->extable || @@ -747,7 +792,7 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte * Setting the tail_call_info in trampoline's frame * depending on if previous frame had value or reference. */ - EMIT(PPC_RAW_CMPLWI(_R3, MAX_TAIL_CALL_CNT)); + EMIT(PPC_RAW_CMPLLI(_R3, MAX_TAIL_CALL_CNT)); PPC_BCC_CONST_SHORT(COND_GT, 8); EMIT(PPC_RAW_ADDI(_R3, _R4, -BPF_PPC_TAILCALL)); @@ -1266,6 +1311,7 @@ static void do_isync(void *info __maybe_unused) * bpf_func: * [nop|b] ool_stub * 2. Out-of-line stub: + * nop // optional nop for alignment * ool_stub: * mflr r0 * [b|bl] <bpf_prog>/<long_branch_stub> @@ -1273,14 +1319,14 @@ static void do_isync(void *info __maybe_unused) * b bpf_func + 4 * 3. Long branch stub: * long_branch_stub: - * .long <branch_addr>/<dummy_tramp> * mflr r11 * bcl 20,31,$+4 * mflr r12 - * ld r12, -16(r12) + * ld r12, 20(r12) * mtctr r12 * mtlr r11 // needed to retain ftrace ABI * bctr + * .long <branch_addr>/<dummy_tramp> * * dummy_tramp is used to reduce synchronization requirements. * @@ -1382,10 +1428,12 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t, * 1. Update the address in the long branch stub: * If new_addr is out of range, we will have to use the long branch stub, so patch new_addr * here. Otherwise, revert to dummy_tramp, but only if we had patched old_addr here. + * + * dummy_tramp_addr moved to bottom of long branch stub. */ if ((new_addr && !is_offset_in_branch_range(new_addr - ip)) || (old_addr && !is_offset_in_branch_range(old_addr - ip))) - ret = patch_ulong((void *)(bpf_func_end - bpf_jit_long_branch_stub - SZL), + ret = patch_ulong((void *)(bpf_func_end - SZL), /* SZL: dummy_tramp_addr offset */ (new_addr && !is_offset_in_branch_range(new_addr - ip)) ? (unsigned long)new_addr : (unsigned long)dummy_tramp); if (ret) diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c index bfdc50740da8..f5b9441cf46a 100644 --- a/arch/powerpc/net/bpf_jit_comp32.c +++ b/arch/powerpc/net/bpf_jit_comp32.c @@ -229,7 +229,7 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx } -void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx) +void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx) { EMIT(PPC_RAW_MR(_R3, bpf_to_ppc(BPF_REG_0))); @@ -237,7 +237,7 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx) EMIT(PPC_RAW_BLR()); - bpf_jit_build_fentry_stubs(image, ctx); + bpf_jit_build_fentry_stubs(image, fimage, ctx); } /* Relative offset needs to be calculated based on final image location */ @@ -1149,7 +1149,8 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code * we'll just fall through to the epilogue. */ if (i != flen - 1) { - ret = bpf_jit_emit_exit_insn(image, ctx, _R0, exit_addr); + ret = bpf_jit_emit_exit_insn(image, fimage, + ctx, _R0, exit_addr); if (ret) return ret; } diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index fc9db691e820..c1e86fe0bd79 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -276,7 +276,7 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) */ EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, 0)); EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_TAILCALL))); - EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); PPC_BCC_CONST_SHORT(COND_GT, 8); EMIT(PPC_RAW_ADDI(bpf_to_ppc(TMP_REG_1), bpf_to_ppc(TMP_REG_2), -(BPF_PPC_TAILCALL))); @@ -398,7 +398,7 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx } } -void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx) +void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx) { bpf_jit_emit_common_epilogue(image, ctx); @@ -407,7 +407,7 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx) EMIT(PPC_RAW_BLR()); - bpf_jit_build_fentry_stubs(image, ctx); + bpf_jit_build_fentry_stubs(image, fimage, ctx); } /* @@ -662,7 +662,7 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o PPC_BCC_SHORT(COND_GE, out); EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_1), _R1, bpf_jit_stack_tailcallinfo_offset(ctx))); - EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); PPC_BCC_CONST_SHORT(COND_LE, 8); /* dereference TMP_REG_1 */ @@ -672,7 +672,7 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o * if (tail_call_info == MAX_TAIL_CALL_CNT) * goto out; */ - EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_1), MAX_TAIL_CALL_CNT)); PPC_BCC_SHORT(COND_EQ, out); /* @@ -707,7 +707,7 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o * tail_call_info. */ EMIT(PPC_RAW_LD(bpf_to_ppc(TMP_REG_2), _R1, bpf_jit_stack_tailcallinfo_offset(ctx))); - EMIT(PPC_RAW_CMPLWI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); + EMIT(PPC_RAW_CMPLDI(bpf_to_ppc(TMP_REG_2), MAX_TAIL_CALL_CNT)); PPC_BCC_CONST_SHORT(COND_GT, 8); /* First get address of tail_call_info */ @@ -1737,7 +1737,8 @@ emit_clear: * we'll just fall through to the epilogue. */ if (i != flen - 1) { - ret = bpf_jit_emit_exit_insn(image, ctx, tmp1_reg, exit_addr); + ret = bpf_jit_emit_exit_insn(image, fimage, ctx, + tmp1_reg, exit_addr); if (ret) return ret; } diff --git a/arch/powerpc/platforms/512x/Kconfig b/arch/powerpc/platforms/512x/Kconfig index deecede78776..9d46d791af05 100644 --- a/arch/powerpc/platforms/512x/Kconfig +++ b/arch/powerpc/platforms/512x/Kconfig @@ -32,11 +32,12 @@ config MPC512x_GENERIC which do not need custom platform specific setup. Compatible boards include: Protonic LVT base boards (ZANMCU - and VICVT2), Freescale MPC5125 Tower system. + and VICVT2), Freescale MPC5125 Tower system, IFM PDM360NG. config PDM360NG bool "ifm PDM360NG board" depends on PPC_MPC512x + select MPC512x_GENERIC select DEFAULT_UIMAGE help This option enables support for the PDM360NG board. diff --git a/arch/powerpc/platforms/512x/Makefile b/arch/powerpc/platforms/512x/Makefile index 2daf22ee26a0..4da6f6923d8f 100644 --- a/arch/powerpc/platforms/512x/Makefile +++ b/arch/powerpc/platforms/512x/Makefile @@ -7,4 +7,3 @@ obj-y += mpc512x_shared.o obj-$(CONFIG_MPC5121_ADS) += mpc5121_ads.o mpc5121_ads_cpld.o obj-$(CONFIG_MPC512x_GENERIC) += mpc512x_generic.o obj-$(CONFIG_MPC512x_LPBFIFO) += mpc512x_lpbfifo.o -obj-$(CONFIG_PDM360NG) += pdm360ng.o diff --git a/arch/powerpc/platforms/512x/mpc512x_generic.c b/arch/powerpc/platforms/512x/mpc512x_generic.c index d4fa6c302ccf..d330d3d85887 100644 --- a/arch/powerpc/platforms/512x/mpc512x_generic.c +++ b/arch/powerpc/platforms/512x/mpc512x_generic.c @@ -24,6 +24,7 @@ static const char * const board[] __initconst = { "prt,prtlvt", "fsl,mpc5125ads", "ifm,ac14xx", + "ifm,pdm360ng", NULL }; diff --git a/arch/powerpc/platforms/512x/pdm360ng.c b/arch/powerpc/platforms/512x/pdm360ng.c deleted file mode 100644 index 8bbbf78bb42b..000000000000 --- a/arch/powerpc/platforms/512x/pdm360ng.c +++ /dev/null @@ -1,126 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* - * Copyright (C) 2010 DENX Software Engineering - * - * Anatolij Gustschin, <agust@denx.de> - * - * PDM360NG board setup - */ - -#include <linux/device.h> -#include <linux/kernel.h> -#include <linux/io.h> -#include <linux/of.h> -#include <linux/of_address.h> -#include <linux/of_fdt.h> - -#include <asm/machdep.h> -#include <asm/ipic.h> - -#include "mpc512x.h" - -#if defined(CONFIG_TOUCHSCREEN_ADS7846) || \ - defined(CONFIG_TOUCHSCREEN_ADS7846_MODULE) -#include <linux/interrupt.h> -#include <linux/spi/ads7846.h> -#include <linux/spi/spi.h> -#include <linux/notifier.h> - -static void *pdm360ng_gpio_base; - -static int pdm360ng_get_pendown_state(void) -{ - u32 reg; - - reg = in_be32(pdm360ng_gpio_base + 0xc); - if (reg & 0x40) - setbits32(pdm360ng_gpio_base + 0xc, 0x40); - - reg = in_be32(pdm360ng_gpio_base + 0x8); - - /* return 1 if pen is down */ - return (reg & 0x40) == 0; -} - -static struct ads7846_platform_data pdm360ng_ads7846_pdata = { - .model = 7845, - .get_pendown_state = pdm360ng_get_pendown_state, - .irq_flags = IRQF_TRIGGER_LOW, -}; - -static int __init pdm360ng_penirq_init(void) -{ - struct device_node *np; - - np = of_find_compatible_node(NULL, NULL, "fsl,mpc5121-gpio"); - if (!np) { - pr_err("%s: Can't find 'mpc5121-gpio' node\n", __func__); - return -ENODEV; - } - - pdm360ng_gpio_base = of_iomap(np, 0); - of_node_put(np); - if (!pdm360ng_gpio_base) { - pr_err("%s: Can't map gpio regs.\n", __func__); - return -ENODEV; - } - out_be32(pdm360ng_gpio_base + 0xc, 0xffffffff); - setbits32(pdm360ng_gpio_base + 0x18, 0x2000); - setbits32(pdm360ng_gpio_base + 0x10, 0x40); - - return 0; -} - -static int pdm360ng_touchscreen_notifier_call(struct notifier_block *nb, - unsigned long event, void *__dev) -{ - struct device *dev = __dev; - - if ((event == BUS_NOTIFY_ADD_DEVICE) && - of_device_is_compatible(dev->of_node, "ti,ads7846")) { - dev->platform_data = &pdm360ng_ads7846_pdata; - return NOTIFY_OK; - } - return NOTIFY_DONE; -} - -static struct notifier_block pdm360ng_touchscreen_nb = { - .notifier_call = pdm360ng_touchscreen_notifier_call, -}; - -static void __init pdm360ng_touchscreen_init(void) -{ - if (pdm360ng_penirq_init()) - return; - - bus_register_notifier(&spi_bus_type, &pdm360ng_touchscreen_nb); -} -#else -static inline void __init pdm360ng_touchscreen_init(void) -{ -} -#endif /* CONFIG_TOUCHSCREEN_ADS7846 */ - -static void __init pdm360ng_init(void) -{ - mpc512x_init(); - pdm360ng_touchscreen_init(); -} - -static int __init pdm360ng_probe(void) -{ - mpc512x_init_early(); - - return 1; -} - -define_machine(pdm360ng) { - .name = "PDM360NG", - .compatible = "ifm,pdm360ng", - .probe = pdm360ng_probe, - .setup_arch = mpc512x_setup_arch, - .init = pdm360ng_init, - .init_IRQ = mpc512x_init_IRQ, - .get_irq = ipic_get_irq, - .restart = mpc512x_restart, -}; diff --git a/arch/powerpc/platforms/ps3/interrupt.c b/arch/powerpc/platforms/ps3/interrupt.c index a4ad4b49eef7..358611638258 100644 --- a/arch/powerpc/platforms/ps3/interrupt.c +++ b/arch/powerpc/platforms/ps3/interrupt.c @@ -184,8 +184,7 @@ static int ps3_virq_setup(enum ps3_cpu_binding cpu, unsigned long outlet, if (!*virq) { FAIL("%s:%d: irq_create_mapping failed: outlet %lu\n", __func__, __LINE__, outlet); - result = -ENOMEM; - goto fail_create; + return -ENOMEM; } DBG("%s:%d: outlet %lu => cpu %u, virq %u\n", __func__, __LINE__, @@ -196,17 +195,13 @@ static int ps3_virq_setup(enum ps3_cpu_binding cpu, unsigned long outlet, if (result) { FAIL("%s:%d: irq_set_chip_data failed\n", __func__, __LINE__); - goto fail_set; + irq_dispose_mapping(*virq); + return result; } ps3_chip_mask(irq_get_irq_data(*virq)); return result; - -fail_set: - irq_dispose_mapping(*virq); -fail_create: - return result; } /** @@ -250,7 +245,7 @@ int ps3_irq_plug_setup(enum ps3_cpu_binding cpu, unsigned long outlet, if (result) { FAIL("%s:%d: ps3_virq_setup failed\n", __func__, __LINE__); - goto fail_setup; + return result; } pd = irq_get_chip_data(*virq); @@ -263,16 +258,11 @@ int ps3_irq_plug_setup(enum ps3_cpu_binding cpu, unsigned long outlet, if (result) { FAIL("%s:%d: lv1_connect_irq_plug_ext failed: %s\n", __func__, __LINE__, ps3_result(result)); - result = -EPERM; - goto fail_connect; + ps3_virq_destroy(*virq); + return -EPERM; } return result; - -fail_connect: - ps3_virq_destroy(*virq); -fail_setup: - return result; } EXPORT_SYMBOL_GPL(ps3_irq_plug_setup); diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 272e9aab66d4..32d414c2cb84 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -69,6 +69,8 @@ static struct iommu_table *iommu_pseries_alloc_table(int node) return tbl; } +static phys_addr_t pseries_ddw_max_ram __ro_after_init; + #ifdef CONFIG_IOMMU_API static struct iommu_table_group_ops spapr_tce_table_group_ops; #endif @@ -1284,15 +1286,19 @@ struct failed_ddw_pdn { static LIST_HEAD(failed_ddw_pdn_list); -static phys_addr_t ddw_memory_hotplug_max(void) +static phys_addr_t __init ddw_memory_hotplug_max(void) { - resource_size_t max_addr; + resource_size_t max_addr = memory_hotplug_max(); + struct device_node *memory; -#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG) - max_addr = hot_add_drconf_memory_max(); -#else - max_addr = memblock_end_of_DRAM(); -#endif + for_each_node_by_type(memory, "memory") { + struct resource res; + + if (of_address_to_resource(memory, 0, &res)) + continue; + + max_addr = max_t(resource_size_t, max_addr, res.end + 1); + } return max_addr; } @@ -1447,7 +1453,7 @@ static struct property *ddw_property_create(const char *propname, u32 liobn, u64 static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mask) { int len = 0, ret; - int max_ram_len = order_base_2(ddw_memory_hotplug_max()); + int max_ram_len = order_base_2(pseries_ddw_max_ram); struct ddw_query_response query; struct ddw_create_response create; int page_shift; @@ -1617,12 +1623,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas if (default_win_removed && pmem_present && !direct_mapping) { /* DDW is big enough to be split */ if ((1ULL << ddw_sz) >= - MIN_DDW_VPMEM_DMA_WINDOW + (1ULL << max_ram_len)) { + MIN_DDW_VPMEM_DMA_WINDOW + pseries_ddw_max_ram) { direct_mapping = true; /* offset of the Dynamic part of DDW */ - dynamic_offset = 1ULL << max_ram_len; + dynamic_offset = pseries_ddw_max_ram; } /* DDW will at least have dynamic allocation */ @@ -1669,7 +1675,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas if (direct_mapping) { /* DDW maps the whole partition, so enable direct DMA mapping */ - ret = walk_system_ram_range(0, ddw_memory_hotplug_max() >> PAGE_SHIFT, + ret = walk_system_ram_range(0, pseries_ddw_max_ram >> PAGE_SHIFT, win64->value, tce_setrange_multi_pSeriesLP_walk); if (ret) { dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n", @@ -1772,8 +1778,7 @@ out_unlock: /* For pre-mapped memory, set bus_dma_limit to the max RAM */ if (direct_mapping) - dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + - (1ULL << max_ram_len); + dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + pseries_ddw_max_ram; dev_info(&dev->dev, "lsa_required: %x, lsa_enabled: %x, direct mapping: %x\n", limited_addr_req, limited_addr_enabled, direct_mapping); @@ -2420,23 +2425,35 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action, { struct dma_win *window; struct memory_notify *arg = data; + unsigned long limit = arg->nr_pages; + unsigned long max_ram_pages = pseries_ddw_max_ram >> PAGE_SHIFT; int ret = 0; /* This notifier can get called when onlining persistent memory as well. * TCEs are not pre-mapped for persistent memory. Persistent memory will - * always be above ddw_memory_hotplug_max() + * always be above pseries_ddw_max_ram */ + if (arg->start_pfn >= max_ram_pages) + return NOTIFY_OK; + + /* RAM is being DLPAR'ed. The range should never exceed max ram. + * Just in case, clamp the range and throw a warning. + */ + if (arg->start_pfn + limit > max_ram_pages) { + limit = max_ram_pages - arg->start_pfn; + WARN_ONCE(1, "Limiting Page Range %lx - %lx to Max Mem Pages: %lx\n", + arg->start_pfn, arg->start_pfn + arg->nr_pages, + max_ram_pages); + } switch (action) { case MEM_GOING_ONLINE: spin_lock(&dma_win_list_lock); list_for_each_entry(window, &dma_win_list, list) { - if (window->direct && (arg->start_pfn << PAGE_SHIFT) < - ddw_memory_hotplug_max()) { + if (window->direct) { ret |= tce_setrange_multi_pSeriesLP(arg->start_pfn, - arg->nr_pages, window->prop); + limit, window->prop); } - /* XXX log error */ } spin_unlock(&dma_win_list_lock); break; @@ -2444,12 +2461,10 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action, case MEM_OFFLINE: spin_lock(&dma_win_list_lock); list_for_each_entry(window, &dma_win_list, list) { - if (window->direct && (arg->start_pfn << PAGE_SHIFT) < - ddw_memory_hotplug_max()) { + if (window->direct) { ret |= tce_clearrange_multi_pSeriesLP(arg->start_pfn, - arg->nr_pages, window->prop); + limit, window->prop); } - /* XXX log error */ } spin_unlock(&dma_win_list_lock); break; @@ -2533,6 +2548,14 @@ void __init iommu_init_early_pSeries(void) register_memory_notifier(&iommu_mem_nb); set_pci_dma_ops(&dma_iommu_ops); + + /* During init determine the max memory an LPAR can have and set it. This + * will be used for pre-mapping RAM in DDW. memblock_end_of_DRAM() can + * change during the running of LPAR - daxctl can add pmemory as + * "system-ram". This memory range should not be pre-mapped in DDW since + * the address of pmemory can be much higher than the DDW size. + */ + pseries_ddw_max_ram = ddw_memory_hotplug_max(); } static int __init disable_multitce(char *str) diff --git a/arch/powerpc/platforms/pseries/plpks-sysfs.c b/arch/powerpc/platforms/pseries/plpks-sysfs.c index c2ebcbb41ae3..f2436229f323 100644 --- a/arch/powerpc/platforms/pseries/plpks-sysfs.c +++ b/arch/powerpc/platforms/pseries/plpks-sysfs.c @@ -45,6 +45,16 @@ static const struct attribute *config_attrs[] = { static struct kobject *plpks_kobj, *plpks_config_kobj; +static umode_t plpks_config_attr_is_visible(struct kobject *kobj, + struct attribute *attr, int n) +{ + if (attr == &attr_wrapping_features.attr && + !plpks_wrapping_is_supported()) + return 0; + + return attr->mode; +} + int plpks_config_create_softlink(struct kobject *from) { if (!plpks_config_kobj) @@ -52,13 +62,13 @@ int plpks_config_create_softlink(struct kobject *from) return sysfs_create_link(from, plpks_config_kobj, "config"); } +static const struct attribute_group config_group = { + .attrs = (struct attribute **)config_attrs, + .is_visible = plpks_config_attr_is_visible, +}; + static __init int plpks_sysfs_config(struct kobject *kobj) { - struct attribute_group config_group = { - .name = NULL, - .attrs = (struct attribute **)config_attrs, - }; - return sysfs_create_group(kobj, &config_group); } diff --git a/arch/powerpc/platforms/pseries/plpks.c b/arch/powerpc/platforms/pseries/plpks.c index 23e4e2a922fc..240cd5632446 100644 --- a/arch/powerpc/platforms/pseries/plpks.c +++ b/arch/powerpc/platforms/pseries/plpks.c @@ -10,7 +10,7 @@ #define pr_fmt(fmt) "plpks: " fmt #define PLPKS_WRAPKEY_COMPONENT "PLPKSWR" -#define PLPKS_WRAPKEY_NAME "default-wrapping-key" +#define PLPKS_DEFAULT_WRAPKEY_LABEL "default-wrapping-key" /* * To 4K align the {input, output} buffers to the {UN}WRAP H_CALLs @@ -576,7 +576,7 @@ static int plpks_confirm_object_flushed(struct label *label, virt_to_phys(auth), virt_to_phys(label), label->size); - status = retbuf[0]; + status = (u8)retbuf[0]; if (rc) { timed_out = false; if (rc == H_NOT_FOUND && status == 1) @@ -637,6 +637,9 @@ int plpks_signed_update_var(struct plpks_var *var, u64 flags) u64 continuetoken = 0; u64 timeout = 0; + if (!var) + return -EINVAL; + if (!var->data || var->datalen <= 0 || var->namelen > PLPKS_MAX_NAME_SIZE) return -EINVAL; @@ -682,7 +685,7 @@ int plpks_signed_update_var(struct plpks_var *var, u64 flags) kfree(label); out: - kfree(auth); + kfree_sensitive(auth); return rc; } @@ -751,7 +754,7 @@ int plpks_write_var(struct plpks_var var) rc = pseries_status_to_err(rc); kfree(label); out: - kfree(auth); + kfree_sensitive(auth); return rc; } @@ -809,7 +812,7 @@ int plpks_remove_var(char *component, u8 varos, struct plpks_var_name vname) rc = pseries_status_to_err(rc); kfree(label); out: - kfree(auth); + kfree_sensitive(auth); return rc; } @@ -822,11 +825,14 @@ static int plpks_read_var(u8 consumer, struct plpks_var *var) u8 *output; int rc; + if (!var) + return -EINVAL; + if (var->namelen > PLPKS_MAX_NAME_SIZE) return -EINVAL; if (var->policy & PLPKS_WRAPPINGKEY) - return -EINVAL; + return -EPERM; auth = construct_auth(consumer); if (IS_ERR(auth)) @@ -856,28 +862,27 @@ static int plpks_read_var(u8 consumer, struct plpks_var *var) virt_to_phys(var->name), var->namelen, virt_to_phys(output), maxobjsize); - if (rc != H_SUCCESS) { rc = pseries_status_to_err(rc); - goto out_free_output; + if (rc != -EPERM || !retbuf[1]) + goto out_free_output; + goto out_copy_policy; } if (!var->data || var->datalen > retbuf[0]) - var->datalen = retbuf[0]; - - var->policy = retbuf[1]; + var->datalen = (u16)retbuf[0]; if (var->data) memcpy(var->data, output, var->datalen); - rc = 0; - +out_copy_policy: + var->policy = (u32)retbuf[1]; out_free_output: - kfree(output); + kfree_sensitive(output); out_free_label: kfree(label); out_free_auth: - kfree(auth); + kfree_sensitive(auth); return rc; } @@ -933,8 +938,8 @@ int plpks_gen_wrapping_key(void) struct label *label; int rc = 0, pseries_status = 0; struct plpks_var var = { - .name = PLPKS_WRAPKEY_NAME, - .namelen = strlen(var.name), + .name = PLPKS_DEFAULT_WRAPKEY_LABEL, + .namelen = sizeof(PLPKS_DEFAULT_WRAPKEY_LABEL) - 1, .policy = PLPKS_WRAPPINGKEY, .os = PLPKS_VAR_LINUX, .component = PLPKS_WRAPKEY_COMPONENT @@ -970,7 +975,7 @@ int plpks_gen_wrapping_key(void) kfree(label); out: - kfree(auth); + kfree_sensitive(auth); return rc; } EXPORT_SYMBOL_GPL(plpks_gen_wrapping_key); @@ -1028,8 +1033,8 @@ int plpks_wrap_object(u8 **input_buf, u32 input_len, u16 wrap_flags, bool sb_audit_or_enforce_bit = wrap_flags & BIT(0); bool sb_enforce_bit = wrap_flags & BIT(1); struct plpks_var var = { - .name = PLPKS_WRAPKEY_NAME, - .namelen = strlen(var.name), + .name = PLPKS_DEFAULT_WRAPKEY_LABEL, + .namelen = sizeof(PLPKS_DEFAULT_WRAPKEY_LABEL) - 1, .os = PLPKS_VAR_LINUX, .component = PLPKS_WRAPKEY_COMPONENT }; @@ -1095,7 +1100,7 @@ int plpks_wrap_object(u8 **input_buf, u32 input_len, u16 wrap_flags, out_free_label: kfree(label); out: - kfree(auth); + kfree_sensitive(auth); return rc; } EXPORT_SYMBOL_GPL(plpks_wrap_object); @@ -1172,14 +1177,14 @@ int plpks_unwrap_object(u8 **input_buf, u32 input_len, u8 **output_buf, if (rc) { pr_err("H_PKS_UNWRAP_OBJECT failed. pseries_status=%d, rc=%d", pseries_status, rc); - kfree(*output_buf); + kfree_sensitive(*output_buf); *output_buf = NULL; } else { *output_len = retbuf[1]; } out: - kfree(auth); + kfree_sensitive(auth); return rc; } EXPORT_SYMBOL_GPL(plpks_unwrap_object); diff --git a/arch/powerpc/platforms/pseries/plpks_sed_ops.c b/arch/powerpc/platforms/pseries/plpks_sed_ops.c index 7c873c9589ef..ac3ec2ca7fbe 100644 --- a/arch/powerpc/platforms/pseries/plpks_sed_ops.c +++ b/arch/powerpc/platforms/pseries/plpks_sed_ops.c @@ -84,14 +84,16 @@ int sed_read_key(char *keyname, char *key, u_int *keylen) ret = plpks_read_os_var(&var); if (ret != 0) - return ret; + goto out; len = min_t(u16, be32_to_cpu(data.key_len), var.datalen); memcpy(key, data.key, len); key[len] = '\0'; *keylen = len; - return 0; +out: + memzero_explicit(&data, sizeof(data)); + return ret; } /* @@ -102,6 +104,7 @@ int sed_write_key(char *keyname, char *key, u_int keylen) struct plpks_var var; struct plpks_sed_object_data data; struct plpks_var_name vname; + int ret; plpks_init_var(&var, keyname); @@ -127,5 +130,8 @@ int sed_write_key(char *keyname, char *key, u_int keylen) vname.name = var.name; plpks_remove_var(var.component, var.os, vname); - return plpks_write_var(var); + ret = plpks_write_var(var); + memzero_explicit(&data, sizeof(data)); + + return ret; } diff --git a/arch/powerpc/platforms/pseries/rtas-fadump.c b/arch/powerpc/platforms/pseries/rtas-fadump.c index 3bb4ac2ab6cc..9b9e09f3d900 100644 --- a/arch/powerpc/platforms/pseries/rtas-fadump.c +++ b/arch/powerpc/platforms/pseries/rtas-fadump.c @@ -459,6 +459,8 @@ static int __init rtas_fadump_process(struct fw_dump *fadump_conf) /* Check if the dump data is valid. */ for (int i = 0; i < be16_to_cpu(fdm_active->header.dump_num_sections); i++) { int type = be16_to_cpu(fdm_active->rgn[i].source_data_type); + uint64_t bytes_dumped = be64_to_cpu(fdm_active->rgn[i].bytes_dumped); + uint64_t source_len = be64_to_cpu(fdm_active->rgn[i].source_len); int rc = 0; switch (type) { @@ -469,10 +471,20 @@ static int __init rtas_fadump_process(struct fw_dump *fadump_conf) pr_err("Dump taken by platform is not valid (%d)\n", i); rc = -EINVAL; } - if (fdm_active->rgn[i].bytes_dumped != fdm_active->rgn[i].source_len) { - pr_err("Dump taken by platform is incomplete (%d)\n", i); + + /* + * Make sure that dump is collected for entire region. + * CPU_STATE_DATA region is allowed to dump less than allocated space. + */ + if (!(bytes_dumped == source_len || + (type == RTAS_FADUMP_CPU_STATE_DATA && bytes_dumped <= source_len))) { + pr_err("Dump taken by platform is %s (%d)\n", + (bytes_dumped < source_len ? "incomplete" : "corrupted"), i); + pr_debug("type -> %d, bytes_dumped -> %llx, source_len -> %llx\n", + type, bytes_dumped, source_len); rc = -EINVAL; } + if (rc) { pr_warn("Region type: %u src addr: 0x%llx dest addr: 0x%llx\n", be16_to_cpu(fdm_active->rgn[i].source_data_type), @@ -482,7 +494,7 @@ static int __init rtas_fadump_process(struct fw_dump *fadump_conf) } break; case RTAS_FADUMP_PARAM_AREA: - if (fdm_active->rgn[i].bytes_dumped != fdm_active->rgn[i].source_len || + if (bytes_dumped != source_len || fdm_active->rgn[i].error_flags != 0) { pr_warn("Failed to process additional parameters! Proceeding anyway..\n"); fadump_conf->param_area = 0; |