linux-toradex.git/kernel/panic.c, branch v4.6-rc3

panic: change nmi_panic from macro to function

2016-03-22T22:36:02+00:00

Commit 1717f2096b54 ("panic, x86: Fix re-entrance problem due to panic
on NMI") and commit 58c5661f2144 ("panic, x86: Allow CPUs to save
registers even if looping in NMI context") introduced nmi_panic() which
prevents concurrent/recursive execution of panic().  It also saves
registers for the crash dump on x86.

However, there are some cases where NMI handlers still use panic().
This patch set partially replaces them with nmi_panic() in those cases.

Even this patchset is applied, some NMI or similar handlers (e.g.  MCE
handler) continue to use panic().  This is because I can't test them
well and actual problems won't happen.  For example, the possibility
that normal panic and panic on MCE happen simultaneously is very low.

This patch (of 3):

Convert nmi_panic() to a proper function and export it instead of
exporting internal implementation details to modules, for obvious
reasons.

Signed-off-by: Hidehiro Kawai 
Acked-by: Borislav Petkov 
Acked-by: Michal Nazarewicz 
Cc: Michal Hocko 
Cc: Rasmus Villemoes 
Cc: Nicolas Iooss 
Cc: Javi Merino 
Cc: Gobinda Charan Maji 
Cc: "Steven Rostedt (Red Hat)" 
Cc: Thomas Gleixner 
Cc: Vitaly Kuznetsov 
Cc: HATAYAMA Daisuke 
Cc: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

lib/bug.c: use common WARN helper

2016-03-17T22:09:34+00:00

The traceoff_on_warning option doesn't have any effect on s390, powerpc,
arm64, parisc, and sh because there are two different types of WARN
implementations:

1) The above mentioned architectures treat WARN() as a special case of a
   BUG() exception.  They handle warnings in report_bug() in lib/bug.c.

2) All other architectures just call warn_slowpath_*() directly.  Their
   warnings are handled in warn_slowpath_common() in kernel/panic.c.

Support traceoff_on_warning on all architectures and prevent any future
divergence by using a single common function to emit the warning.

Also remove the '()' from '%pS()', because the parentheses look funky:

  [   45.607629] WARNING: at /root/warn_mod/warn_mod.c:17 .init_dummy+0x20/0x40 [warn_mod]()

Reported-by: Chunyu Hu 
Signed-off-by: Josh Poimboeuf 
Acked-by: Heiko Carstens 
Tested-by: Prarit Bhargava 
Acked-by: Prarit Bhargava 
Acked-by: Steven Rostedt 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

printk: do cond_resched() between lines while outputting to consoles

2016-01-16T19:17:25+00:00

@console_may_schedule tracks whether console_sem was acquired through
lock or trylock.  If the former, we're inside a sleepable context and
console_conditional_schedule() performs cond_resched().  This allows
console drivers which use console_lock for synchronization to yield
while performing time-consuming operations such as scrolling.

However, the actual console outputting is performed while holding
irq-safe logbuf_lock, so console_unlock() clears @console_may_schedule
before starting outputting lines.  Also, only a few drivers call
console_conditional_schedule() to begin with.  This means that when a
lot of lines need to be output by console_unlock(), for example on a
console registration, the task doing console_unlock() may not yield for
a long time on a non-preemptible kernel.

If this happens with a slow console devices, for example a serial
console, the outputting task may occupy the cpu for a very long time.
Long enough to trigger softlockup and/or RCU stall warnings, which in
turn pile more messages, sometimes enough to trigger the next cycle of
warnings incapacitating the system.

Fix it by making console_unlock() insert cond_resched() between lines if
@console_may_schedule.

Signed-off-by: Tejun Heo 
Reported-by: Calvin Owens 
Acked-by: Jan Kara 
Cc: Dave Jones 
Cc: Kyle McMartin 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kexec: Fix race between panic() and crash_kexec()

2015-12-19T10:07:01+00:00

Currently, panic() and crash_kexec() can be called at the same time.
For example (x86 case):

CPU 0:
  oops_end()
    crash_kexec()
      mutex_trylock() // acquired
        nmi_shootdown_cpus() // stop other CPUs

CPU 1:
  panic()
    crash_kexec()
      mutex_trylock() // failed to acquire
    smp_send_stop() // stop other CPUs
    infinite loop

If CPU 1 calls smp_send_stop() before nmi_shootdown_cpus(), kdump
fails.

In another case:

CPU 0:
  oops_end()
    crash_kexec()
      mutex_trylock() // acquired
        
        io_check_error()
          panic()
            crash_kexec()
              mutex_trylock() // failed to acquire
            infinite loop

Clearly, this is an undesirable result.

To fix this problem, this patch changes crash_kexec() to exclude others
by using the panic_cpu atomic.

Signed-off-by: Hidehiro Kawai 
Acked-by: Michal Hocko 
Cc: Andrew Morton 
Cc: Baoquan He 
Cc: Dave Young 
Cc: "Eric W. Biederman" 
Cc: HATAYAMA Daisuke 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Jonathan Corbet 
Cc: kexec@lists.infradead.org
Cc: linux-doc@vger.kernel.org
Cc: Martin Schwidefsky 
Cc: Masami Hiramatsu 
Cc: Minfei Huang 
Cc: Peter Zijlstra 
Cc: Seth Jennings 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Cc: Vitaly Kuznetsov 
Cc: Vivek Goyal 
Cc: x86-ml 
Link: http://lkml.kernel.org/r/20151210014630.25437.94161.stgit@softrs
Signed-off-by: Borislav Petkov 
Signed-off-by: Thomas Gleixner

panic, x86: Allow CPUs to save registers even if looping in NMI context

2015-12-19T10:07:01+00:00

Currently, kdump_nmi_shootdown_cpus(), a subroutine of crash_kexec(),
sends an NMI IPI to CPUs which haven't called panic() to stop them,
save their register information and do some cleanups for crash dumping.
However, if such a CPU is infinitely looping in NMI context, we fail to
save its register information into the crash dump.

For example, this can happen when unknown NMIs are broadcast to all
CPUs as follows:

  CPU 0                             CPU 1
  ===========================       ==========================
  receive an unknown NMI
  unknown_nmi_error()
    panic()                         receive an unknown NMI
      spin_trylock(&panic_lock)     unknown_nmi_error()
      crash_kexec()                   panic()
                                        spin_trylock(&panic_lock)
                                        panic_smp_self_stop()
                                          infinite loop
        kdump_nmi_shootdown_cpus()
          issue NMI IPI -----------> blocked until IRET
                                          infinite loop...

Here, since CPU 1 is in NMI context, the second NMI from CPU 0 is
blocked until CPU 1 executes IRET. However, CPU 1 never executes IRET,
so the NMI is not handled and the callback function to save registers is
never called.

In practice, this can happen on some servers which broadcast NMIs to all
CPUs when the NMI button is pushed.

To save registers in this case, we need to:

  a) Return from NMI handler instead of looping infinitely
  or
  b) Call the callback function directly from the infinite loop

Inherently, a) is risky because NMI is also used to prevent corrupted
data from being propagated to devices.  So, we chose b).

This patch does the following:

1. Move the infinite looping of CPUs which haven't called panic() in NMI
   context (actually done by panic_smp_self_stop()) outside of panic() to
   enable us to refer pt_regs. Please note that panic_smp_self_stop() is
   still used for normal context.

2. Call a callback of kdump_nmi_shootdown_cpus() directly to save
   registers and do some cleanups after setting waiting_for_crash_ipi which
   is used for counting down the number of CPUs which handled the callback

Signed-off-by: Hidehiro Kawai 
Acked-by: Michal Hocko 
Cc: Aaron Tomlin 
Cc: Andrew Morton 
Cc: Andy Lutomirski 
Cc: Baoquan He 
Cc: Chris Metcalf 
Cc: Dave Young 
Cc: David Hildenbrand 
Cc: Don Zickus 
Cc: Eric Biederman 
Cc: Frederic Weisbecker 
Cc: Gobinda Charan Maji 
Cc: HATAYAMA Daisuke 
Cc: Hidehiro Kawai 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Javi Merino 
Cc: Jiang Liu 
Cc: Jonathan Corbet 
Cc: kexec@lists.infradead.org
Cc: linux-doc@vger.kernel.org
Cc: lkml 
Cc: Masami Hiramatsu 
Cc: Michal Nazarewicz 
Cc: Nicolas Iooss 
Cc: Oleg Nesterov 
Cc: Peter Zijlstra 
Cc: Prarit Bhargava 
Cc: Rasmus Villemoes 
Cc: Seth Jennings 
Cc: Stefan Lippers-Hollmann 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Cc: Ulrich Obergfell 
Cc: Vitaly Kuznetsov 
Cc: Vivek Goyal 
Cc: Yasuaki Ishimatsu 
Link: http://lkml.kernel.org/r/20151210014628.25437.75256.stgit@softrs
[ Cleanup comments, fixup formatting. ]
Signed-off-by: Borislav Petkov 
Signed-off-by: Thomas Gleixner

panic, x86: Fix re-entrance problem due to panic on NMI

2015-12-19T10:07:00+00:00

If panic on NMI happens just after panic() on the same CPU, panic() is
recursively called. Kernel stalls, as a result, after failing to acquire
panic_lock.

To avoid this problem, don't call panic() in NMI context if we've
already entered panic().

For that, introduce nmi_panic() macro to reduce code duplication. In
the case of panic on NMI, don't return from NMI handlers if another CPU
already panicked.

Signed-off-by: Hidehiro Kawai 
Acked-by: Michal Hocko 
Cc: Aaron Tomlin 
Cc: Andrew Morton 
Cc: Andy Lutomirski 
Cc: Baoquan He 
Cc: Chris Metcalf 
Cc: David Hildenbrand 
Cc: Don Zickus 
Cc: "Eric W. Biederman" 
Cc: Frederic Weisbecker 
Cc: Gobinda Charan Maji 
Cc: HATAYAMA Daisuke 
Cc: "H. Peter Anvin" 
Cc: Ingo Molnar 
Cc: Javi Merino 
Cc: Jonathan Corbet 
Cc: kexec@lists.infradead.org
Cc: linux-doc@vger.kernel.org
Cc: lkml 
Cc: Masami Hiramatsu 
Cc: Michal Nazarewicz 
Cc: Nicolas Iooss 
Cc: Peter Zijlstra 
Cc: Prarit Bhargava 
Cc: Rasmus Villemoes 
Cc: Rusty Russell 
Cc: Seth Jennings 
Cc: Steven Rostedt 
Cc: Thomas Gleixner 
Cc: Ulrich Obergfell 
Cc: Vitaly Kuznetsov 
Cc: Vivek Goyal 
Link: http://lkml.kernel.org/r/20151210014626.25437.13302.stgit@softrs
[ Cleanup comments, fixup formatting. ]
Signed-off-by: Borislav Petkov 
Signed-off-by: Thomas Gleixner

kernel/panic.c: turn off locks debug before releasing console lock

2015-11-21T00:17:32+00:00

Commit 08d78658f393 ("panic: release stale console lock to always get the
logbuf printed out") introduced an unwanted bad unlock balance report when
panic() is called directly and not from OOPS (e.g.  from out_of_memory()).
The difference is that in case of OOPS we disable locks debug in
oops_enter() and on direct panic call nobody does that.

Fixes: 08d78658f393 ("panic: release stale console lock to always get the logbuf printed out")
Reported-by: kernel test robot 
Signed-off-by: Vitaly Kuznetsov 
Cc: HATAYAMA Daisuke 
Cc: Masami Hiramatsu 
Cc: Jiri Kosina 
Cc: Baoquan He 
Cc: Prarit Bhargava 
Cc: Xie XiuQi 
Cc: Seth Jennings 
Cc: "K. Y. Srinivasan" 
Cc: Jan Kara 
Cc: Petr Mladek 
Cc: Yasuaki Ishimatsu 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

panic: release stale console lock to always get the logbuf printed out

2015-11-07T01:50:42+00:00

In some cases we may end up killing the CPU holding the console lock
while still having valuable data in logbuf. E.g. I'm observing the
following:

- A crash is happening on one CPU and console_unlock() is being called on
  some other.

- console_unlock() tries to print out the buffer before releasing the lock
  and on slow console it takes time.

- in the meanwhile crashing CPU does lots of printk()-s with valuable data
  (which go to the logbuf) and sends IPIs to all other CPUs.

- console_unlock() finishes printing previous chunk and enables interrupts
  before trying to print out the rest, the CPU catches the IPI and never
  releases console lock.

This is not the only possible case: in VT/fb subsystems we have many other
console_lock()/console_unlock() users.  Non-masked interrupts (or
receiving NMI in case of extreme slowness) will have the same result.
Getting the whole console buffer printed out on crash should be top
priority.

[akpm@linux-foundation.org: tweak comment text]
Signed-off-by: Vitaly Kuznetsov 
Cc: HATAYAMA Daisuke 
Cc: Masami Hiramatsu 
Cc: Jiri Kosina 
Cc: Baoquan He 
Cc: Prarit Bhargava 
Cc: Xie XiuQi 
Cc: Seth Jennings 
Cc: "K. Y. Srinivasan" 
Cc: Jan Kara 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kernel/panic/kexec: fix "crash_kexec_post_notifiers" option issue in oops path

2015-07-01T02:44:57+00:00

Commit f06e5153f4ae2e ("kernel/panic.c: add "crash_kexec_post_notifiers"
option for kdump after panic_notifers") introduced
"crash_kexec_post_notifiers" kernel boot option, which toggles wheather
panic() calls crash_kexec() before panic_notifiers and dump kmsg or after.

The problem is that the commit overlooks panic_on_oops kernel boot option.
 If it is enabled, crash_kexec() is called directly without going through
panic() in oops path.

To fix this issue, this patch adds a check to "crash_kexec_post_notifiers"
in the condition of kexec_should_crash().

Also, put a comment in kexec_should_crash() to explain not obvious things
on this patch.

Signed-off-by: HATAYAMA Daisuke 
Acked-by: Baoquan He 
Tested-by: Hidehiro Kawai 
Reviewed-by: Masami Hiramatsu 
Cc: Vivek Goyal 
Cc: Ingo Molnar 
Cc: Hidehiro Kawai 
Cc: Baoquan He 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kernel/panic: call the 2nd crash_kexec() only if crash_kexec_post_notifiers is enabled

2015-07-01T02:44:57+00:00

For compatibility with the behaviour before the commit f06e5153f4ae2e
("kernel/panic.c: add "crash_kexec_post_notifiers" option for kdump after
panic_notifers"), the 2nd crash_kexec() should be called only if
crash_kexec_post_notifiers is enabled.

Note that crash_kexec() returns immediately if kdump crash kernel is not
loaded, so in this case, this patch makes no functionality change, but the
point is to make it explicit, from the caller panic() side, that the 2nd
crash_kexec() does nothing.

Signed-off-by: HATAYAMA Daisuke 
Suggested-by: Ingo Molnar 
Cc: "Eric W. Biederman" 
Cc: Vivek Goyal 
Cc: Masami Hiramatsu 
Cc: Hidehiro Kawai 
Cc: Baoquan He 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds