Percpu allocator honors alignment request upto PAGE_SIZE and both the percpu addresses in the percpu address space and the translated kernel addresses should be aligned accordingly. The calculation of the former depends on the alignment of percpu output section in the kernel image. The linker script macros PERCPU_VADDR() and PERCPU() are used to define this output section and the latter takes @align parameter. Several architectures are using @align smaller than PAGE_SIZE breaking percpu memory alignment. This patch removes @align parameter from PERCPU(), renames it to PERCPU_SECTION() and makes it always align to PAGE_SIZE. While at it, add PCPU_SETUP_BUG_ON() checks such that alignment problems are reliably detected and remove percpu alignment comment recently added in workqueue.c as the condition would trigger BUG way before reaching there. For um, this patch raises the alignment of percpu area. As the area is in .init, there shouldn't be any noticeable difference. This problem was discovered by David Howells while debugging boot failure on mn10300. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Mike Frysinger <vapier@gentoo.org> Cc: uclinux-dist-devel@blackfin.uclinux.org Cc: David Howells <dhowells@redhat.com> Cc: Jeff Dike <jdike@addtoit.com> Cc: user-mode-linux-devel@lists.sourceforge.net
* 'for-linus' of master.kernel.org:/home/rmk/linux-2.6-arm: (91 commits) ARM: 6806/1: irq: introduce entry and exit functions for chained handlers ARM: 6781/1: Thumb-2: Work around buggy Thumb-2 short branch relocations in gas ARM: 6747/1: P2V: Thumb2 support ARM: 6798/1: aout-core: zero thread debug registers in a.out core dump ARM: 6796/1: Footbridge: Fix I/O mappings for NOMMU mode ARM: 6784/1: errata: no automatic Store Buffer drain on Cortex-A9 ARM: 6772/1: errata: possible fault MMU translations following an ASID switch ARM: 6776/1: mach-ux500: activate fix for errata 753970 ARM: 6794/1: SPEAr: Append UL to device address macros. ARM: 6793/1: SPEAr: Remove unused *_SIZE macros from spear*.h files ARM: 6792/1: SPEAr: Replace SIZE macro's with SZ_4K macros ARM: 6791/1: SPEAr3xx: Declare device structures after shirq code ARM: 6790/1: SPEAr: Clock Framework: Rename usbd clock and align apb_clk entry ARM: 6789/1: SPEAr3xx: Rename sdio to sdhci ARM: 6788/1: SPEAr: Include mach/hardware.h instead of mach/spear.h ARM: 6787/1: SPEAr: Reorder #includes in .h & .c files. ARM: 6681/1: SPEAr: add debugfs support to clk API ARM: 6703/1: SPEAr: update clk API support ARM: 6679/1: SPEAr: make clk API functions more generic ARM: 6737/1: SPEAr: formalized timer support ...
Conflicts: arch/arm/kernel/module.c arch/arm/mach-s5pv210/sleep.S
* 'for-2.6.39' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu:
percpu, x86: Add arch-specific this_cpu_cmpxchg_double() support
percpu: Generic support for this_cpu_cmpxchg_double()
alpha: use L1_CACHE_BYTES for cacheline size in the linker script
percpu: align percpu readmostly subsection to cacheline
Fix up trivial conflict in arch/x86/kernel/vmlinux.lds.S due to the
percpu alignment having changed ("x86: Reduce back the alignment of the
per-CPU data section")
When SMP_ON_UP is used and the spinlocks are inlined, we end up with inline spinlocks in the exit code, with references from the SMP alternatives section to the exit sections. This causes link time errors. Avoid this by placing the exit sections in the init-discarded region. Cc: <stable@kernel.org> Tested-by: Dave Martin <dave.martin@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Commit 18991197b4b588255ccabf472ebc84db7b66a19c added --build-id linker option when toolchain supports it. ARM one does, but for some reason places the section at 0 when linker script doesn't mention it explicitly. The 1e621a8e3752367d4aae78a8ab00a18fb2793f34 worked around the problem removing this section from binary image with explicit objcopy options, but it still exists in vmlinux, confusing tools like debuggers and perf. This problem was discussed here: http://lists.infradead.org/pipermail/linux-arm-kernel/2010-May/015994.html http://lists.infradead.org/pipermail/linux-arm-kernel/2010-May/015994.html but the proposed changes to the linker script were substantial. This patch simply places NOTES (36 bytes long, at least when compiled with CodeSourcery toolchain) between data and bss, which seem to be the right place (and suggested by the sample linker script in include/asm-generic/vmlinux.lds.h). It is enough to place it correctly in vmlinux (so debuggers are happy): Section Headers: [11] .data PROGBITS c07ce000 7ce000 020fc0 00 WA 0 0 32 [12] .notes NOTE c07eefc0 7eefc0 000024 00 AX 0 0 4 [13] .bss NOBITS c07ef000 7eefe4 01e628 00 WA 0 0 32 Program Headers: LOAD 0x008000 0xc0008000 0xc0008000 0x7e6fe4 0x805628 RWE 0x8000 NOTE 0x7eefc0 0xc07eefc0 0xc07eefc0 0x00024 0x00024 R E 0x4 Section to Segment mapping: Segment Sections... 00 <...> .data .notes .bss 01 .notes and to get it exposed as /sys/kernel/notes used by perf tools. Signed-off-by: Pawel Moll <pawel.moll@arm.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This idea came from Nicolas, Eric Miao produced an initial version, which was then rewritten into this. Patch the physical to virtual translations at runtime. As we modify the code, this makes it incompatible with XIP kernels, but allows us to achieve this with minimal loss of performance. As many translations are of the form: physical = virtual + (PHYS_OFFSET - PAGE_OFFSET) virtual = physical - (PHYS_OFFSET - PAGE_OFFSET) we generate an 'add' instruction for __virt_to_phys(), and a 'sub' instruction for __phys_to_virt(). We calculate at run time (PHYS_OFFSET - PAGE_OFFSET) by comparing the address prior to MMU initialization with where it should be once the MMU has been initialized, and place this constant into the above add/sub instructions. Once we have (PHYS_OFFSET - PAGE_OFFSET), we can calculate the real PHYS_OFFSET as PAGE_OFFSET is a build-time constant, and save this for the C-mode PHYS_OFFSET variable definition to use. At present, we are unable to support Realview with Sparsemem enabled as this uses a complex mapping function, and MSM as this requires a constant which will not fit in our math instruction. Add a module version magic string for this feature to prevent incompatible modules being loaded. Tested-by: Tony Lindgren <tony@atomide.com> Reviewed-by: Nicolas Pitre <nicolas.pitre@linaro.org> Tested-by: Nicolas Pitre <nicolas.pitre@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Currently percpu readmostly subsection may share cachelines with other percpu subsections which may result in unnecessary cacheline bounce and performance degradation. This patch adds @cacheline parameter to PERCPU() and PERCPU_VADDR() linker macros, makes each arch linker scripts specify its cacheline size and use it to align percpu subsections. This is based on Shaohua's x86 only patch. Signed-off-by: Tejun Heo <tj@kernel.org> Cc: Shaohua Li <shaohua.li@intel.com>
Conflicts: arch/arm/Kconfig arch/arm/common/Makefile arch/arm/kernel/Makefile arch/arm/kernel/smp.c
As our SMP implementation uses MESI protocols. Grouping together data which is mostly only read together means that we avoid unnecessary cache line bouncing when this code shares a cache line with other data. In other words, cache lines associated with read-mostly data are expected to spend most of their time in shared state. Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>