The majority of SMP platforms handle their IPIs through do_IRQ()
which calls irq_{enter/exit}(). When a call function IPI is received,
smp_call_function_interrupt() is called which also calls
irq_{enter,exit}(), meaning irq_count is raised twice.

When tick broadcasting is used (which is implemented via a call
function IPI), this incorrectly causes all CPU idle time on the core
receiving broadcast ticks to be accounted as time spent servicing
IRQs, as account_process_tick() will account as such if irq_count is
greater than 1. This results in 100% CPU usage being reported on a
core which receives its ticks via broadcast.

This patch removes the SMP smp_call_function_interrupt() wrapper which
calls irq_{enter,exit}(). Platforms which handle their IPIs through
do_IRQ() now call generic_smp_call_function_interrupt() directly to
avoid incrementing irq_count a second time. Platforms which don't
(loongson, sgi-ip27, sibyte) call generic_smp_call_function_interrupt()
wrapped in irq_{enter,exit}().

Signed-off-by: Alex Smith <alex.smith@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10770/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

bmips_wr_vec() copies exception vector code from start to dst.

The call to dma_cache_wback() needs to flush (end-start) bytes,
starting at dst, from write-back cache to memory.

Signed-off-by: Petri Gynther <pgynther@google.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
Reviewed-by: Kevin Cernekee <cernekee@gmail.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10193/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

Thanks to spatch, plus manual removal of "&*".  Then a sweep for
for_each_cpu_mask => for_each_cpu.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Kevin Cernekee <cernekee@gmail.com>
Cc: Florian Fainelli <f.fainelli@gmail.com>
Cc: linux-mips@linux-mips.org

On some chips like bcm3384, "other stuff" gets wired up to CPU1's IE_IRQ1
input, generating spurious IRQs.  In this case we want the platform code
to be able to mask it off.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: jfraser@broadcom.com
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8163/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

The secondary CPU's reset vector needs to be set to KSEG1 for a cold
boot (release from reset), or KSEG0 for a warm restart.  On a cold boot
KSEG0 may be unavailable (BMIPS4380), and on a warm restart KSEG1 may
be unavailable (XKS01 mode on 4380 or 5000).

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: jfraser@broadcom.com
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8161/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

CPU interrupts need to be disabled on a cpu being taken down.
When a cpu is hot-plugged out of the system the following sequence occurs.

On the CPU where the hotplug sequence was initiated:
    cpu_down
        _cpu_down {
            __cpu_notify(CPU_DOWN_PREPARE
            __stop_machine(take_cpu_down
                wait for cpu to run disable code.
            __cpu_die
        }

On the CPU  being disabled:
    take_cpu_down
        __cpu_disable {
            mp_ops->cpu_disable
                bmips_cpu_disable
                    clear_c0_status(IE_IRQ5) (added)
            cpu_notify(CPU_DYING...
        }

Before the cpu_notifier is called with CPU_DYING, all interrupts on the
dying cpu must be disabled.  This guarantees that before tick_notify is
called with the CPU_DYING event and sets the clock device pointer to
NULL, there can not be any more clock interrupts.

When this wasn't done, an unfortunately-timed timer interrupt sometimes
caused hangs immediately prior to system suspend:

    Debug PM is not enabled. To enable partial suspend, rebuild kernel with CONFIG_PM_DEBUG
    Pass 1 out of 1,PM: Syncing filesystems ... mode=none, tp1=done.
    1, flags=5, cycle_tp=, sleep=
    Freezing user space processes ... (elapsed 0.01 seconds) done.
    Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
    PM: suspend of devices complete after 54.199 msecs
    PM: late suspend of devices complete after 0.172 msecs
    Disabling non-boot CPUs ...
    SMP: CPU1 is offline
    INFO: rcu_sched detected stalls on CPUs/tasks: { 3} (detected by 0, t=62537 jiffies)
    Call Trace:
    [<804baa78>] dump_stack+0x8/0x34
    [<8008a2d8>] __rcu_pending+0x4b8/0x55c
    [<8008adf4>] rcu_check_callbacks+0x78/0x180
    [<80037830>] update_process_times+0x40/0x6c
    [<80072fe4>] tick_sched_timer+0x74/0xe4
    [<80050180>] __run_hrtimer.clone.30+0x64/0x140
    [<80051150>] hrtimer_interrupt+0x19c/0x4bc
    [<8000cdb8>] c0_compare_interrupt+0x50/0x88
    [<80081b18>] handle_irq_event_percpu+0x5c/0x2f4
    [<80086490>] handle_percpu_irq+0x8c/0xc0
    [<800811b4>] generic_handle_irq+0x34/0x54
    [<800067dc>] do_IRQ+0x18/0x2c
    [<8000375c>] plat_irq_dispatch+0xd0/0x128
    [<80004a04>] ret_from_irq+0x0/0x4
    [<80004c40>] r4k_wait+0x20/0x40
    [<80006b6c>] cpu_idle+0x98/0xf0
    [<805d3988>] start_kernel+0x424/0x440

Signed-off-by: Jon Fraser <jfraser@broadcom.com>
Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8160/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

BMIPS3300 processors do not have the hardware to support SMP, but with a
small tweak, the SMP ebase relocation code allows BMIPS3300-based
platforms to reuse the S2/S3 power management code from BMIPS4380-based
chips.  Normally this is as simple as adding one line to prom_init():

    board_ebase_setup = &bmips_ebase_setup;

Signed-off-by: Jon Fraser <jfraser@broadcom.com>
Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8159/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

This will need to be called from a few different places, and the logic
is starting to get a bit hairy (with the need for IPIs, CPU bug
workarounds, and hazards).

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: jfraser@broadcom.com
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8158/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

On some older BMIPS5200 (dual core / quad thread) platforms, the
PROM code set up CPU2/CPU3 so they would be started through an NMI
instead of through the ACTION register.  But this was incompatible with
some power management features that were later added, so the scheme was
changed so that Linux is fully responsible for booting CPU2/CPU3.

Signed-off-by: Kevin Cernekee <cernekee@gmail.com>
Cc: f.fainelli@gmail.com
Cc: mbizon@freebox.fr
Cc: jogo@openwrt.org
Cc: jfraser@broadcom.com
Cc: linux-mips@linux-mips.org
Cc: devicetree@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/8157/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

__get_cpu_var() is used for multiple purposes in the kernel source. One of
them is address calculation via the form &__get_cpu_var(x).  This calculates
the address for the instance of the percpu variable of the current processor
based on an offset.

Other use cases are for storing and retrieving data from the current
processors percpu area.  __get_cpu_var() can be used as an lvalue when
writing data or on the right side of an assignment.

__get_cpu_var() is defined as :

#define __get_cpu_var(var) (*this_cpu_ptr(&(var)))

__get_cpu_var() always only does an address determination. However, store
and retrieve operations could use a segment prefix (or global register on
other platforms) to avoid the address calculation.

this_cpu_write() and this_cpu_read() can directly take an offset into a
percpu area and use optimized assembly code to read and write per cpu
variables.

This patch converts __get_cpu_var into either an explicit address
calculation using this_cpu_ptr() or into a use of this_cpu operations that
use the offset.  Thereby address calculations are avoided and less registers
are used when code is generated.

At the end of the patch set all uses of __get_cpu_var have been removed so
the macro is removed too.

The patch set includes passes over all arches as well. Once these operations
are used throughout then specialized macros can be defined in non -x86
arches as well in order to optimize per cpu access by f.e.  using a global
register that may be set to the per cpu base.

Transformations done to __get_cpu_var()

1. Determine the address of the percpu instance of the current processor.

	DEFINE_PER_CPU(int, y);
	int *x = &__get_cpu_var(y);

    Converts to

	int *x = this_cpu_ptr(&y);

2. Same as #1 but this time an array structure is involved.

	DEFINE_PER_CPU(int, y[20]);
	int *x = __get_cpu_var(y);

    Converts to

	int *x = this_cpu_ptr(y);

3. Retrieve the content of the current processors instance of a per cpu
variable.

	DEFINE_PER_CPU(int, y);
	int x = __get_cpu_var(y)

   Converts to

	int x = __this_cpu_read(y);

4. Retrieve the content of a percpu struct

	DEFINE_PER_CPU(struct mystruct, y);
	struct mystruct x = __get_cpu_var(y);

   Converts to

	memcpy(&x, this_cpu_ptr(&y), sizeof(x));

5. Assignment to a per cpu variable

	DEFINE_PER_CPU(int, y)
	__get_cpu_var(y) = x;

   Converts to

	__this_cpu_write(y, x);

6. Increment/Decrement etc of a per cpu variable

	DEFINE_PER_CPU(int, y);
	__get_cpu_var(y)++

   Converts to

	__this_cpu_inc(y)

Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Christoph Lameter <cl@linux.com>
Signed-off-by: Tejun Heo <tj@kernel.org>