linux-toradex.git/drivers/pci/pci-driver.c, branch v4.9.98

PCI/MSI: Stop disabling MSI/MSI-X in pci_device_shutdown()

2018-03-22T08:17:38+00:00

[ Upstream commit fda78d7a0ead144f4b2cdb582dcba47911f4952c ]

The pci_bus_type .shutdown method, pci_device_shutdown(), is called from
device_shutdown() in the kernel restart and shutdown paths.

Previously, pci_device_shutdown() called pci_msi_shutdown() and
pci_msix_shutdown().  This disables MSI and MSI-X, which causes the device
to fall back to raising interrupts via INTx.  But the driver is still bound
to the device, it doesn't know about this change, and it likely doesn't
have an INTx handler, so these INTx interrupts cause "nobody cared"
warnings like this:

  irq 16: nobody cared (try booting with the "irqpoll" option)
  CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.8.2-1.el7_UNSUPPORTED.x86_64 #1
  Hardware name: Hewlett-Packard HP Z820 Workstation/158B, BIOS J63 v03.90 06/
  ...

The MSI disabling code was added by d52877c7b1af ("pci/irq: let
pci_device_shutdown to call pci_msi_shutdown v2") because a driver left MSI
enabled and kdump failed because the kexeced kernel wasn't prepared to
receive the MSI interrupts.

Subsequent commits 1851617cd2da ("PCI/MSI: Disable MSI at enumeration even
if kernel doesn't support MSI") and  e80e7edc55ba ("PCI/MSI: Initialize MSI
capability for all architectures") changed the kexeced kernel to disable
all MSIs itself so it no longer depends on the crashed kernel to clean up
after itself.

Stop disabling MSI/MSI-X in pci_device_shutdown().  This resolves the
"nobody cared" unhandled IRQ issue above.  It also allows PCI serial
devices, which may rely on the MSI interrupts, to continue outputting
messages during reboot/shutdown.

[bhelgaas: changelog, drop pci_msi_shutdown() and pci_msix_shutdown() calls
altogether]
Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=187351
Signed-off-by: Prarit Bhargava 
Signed-off-by: Bjorn Helgaas 
CC: Alex Williamson 
CC: David Arcari 
CC: Myron Stowe 
CC: Lukas Wunner 
CC: Keith Busch 
CC: Mika Westerberg 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

PCI / PM: Force devices to D0 in pci_pm_thaw_noirq()

2017-12-29T16:42:59+00:00

commit 5839ee7389e893a31e4e3c9cf17b50d14103c902 upstream.

It is incorrect to call pci_restore_state() for devices in low-power
states (D1-D3), as that involves the restoration of MSI setup which
requires MMIO to be operational and that is only the case in D0.

However, pci_pm_thaw_noirq() may do that if the driver's "freeze"
callbacks put the device into a low-power state, so fix it by making
it force devices into D0 via pci_set_power_state() instead of trying
to "update" their power state which is pointless.

Fixes: e60514bd4485 (PCI/PM: Restore the status of PCI devices across hibernation)
Reported-by: Thomas Gleixner 
Reported-by: Maarten Lankhorst 
Tested-by: Thomas Gleixner 
Tested-by: Maarten Lankhorst 
Signed-off-by: Rafael J. Wysocki 
Acked-by: Bjorn Helgaas 
Signed-off-by: Greg Kroah-Hartman

PCI/PM: Restore the status of PCI devices across hibernation

2017-07-27T22:08:00+00:00

commit e60514bd4485c0c7c5a7cf779b200ce0b95c70d6 upstream.

Currently we saw a lot of "No irq handler" errors during hibernation, which
caused the system hang finally:

  ata4.00: qc timeout (cmd 0xec)
  ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4)
  ata4.00: revalidation failed (errno=-5)
  ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
  do_IRQ: 31.151 No irq handler for vector

According to above logs, there is an interrupt triggered and it is
dispatched to CPU31 with a vector number 151, but there is no handler for
it, thus this IRQ will not get acked and will cause an IRQ flood which
kills the system.  To be more specific, the 31.151 is an interrupt from the
AHCI host controller.

After some investigation, the reason why this issue is triggered is because
the thaw_noirq() function does not restore the MSI/MSI-X settings across
hibernation.

The scenario is illustrated below:

  1. Before hibernation, IRQ 34 is the handler for the AHCI device, which
     is bound to CPU31.

  2. Hibernation starts, the AHCI device is put into low power state.

  3. All the nonboot CPUs are put offline, so IRQ 34 has to be migrated to
     the last alive one - CPU0.

  4. After the snapshot has been created, all the nonboot CPUs are brought
     up again; IRQ 34 remains bound to CPU0.

  5. AHCI devices are put into D0.

  6. The snapshot is written to the disk.

The issue is triggered in step 6.  The AHCI interrupt should be delivered
to CPU0, however it is delivered to the original CPU31 instead, which
causes the "No irq handler" issue.

Ying Huang has provided a clue that, in step 3 it is possible that writing
to the register might not take effect as the PCI devices have been
suspended.

In step 3, the IRQ 34 affinity should be modified from CPU31 to CPU0, but
in fact it is not.  In __pci_write_msi_msg(), if the device is already in
low power state, the low level MSI message entry will not be updated but
cached.  During the device restore process after a normal suspend/resume,
pci_restore_msi_state() writes the cached MSI back to the hardware.

But this is not the case for hibernation.  pci_restore_msi_state() is not
currently called in pci_pm_thaw_noirq(), although pci_save_state() has
saved the necessary PCI cached information in pci_pm_freeze_noirq().

Restore the PCI status for the device during hibernation.  Otherwise the
status might be lost across hibernation (for example, settings for MSI,
MSI-X, ATS, ACS, IOV, etc.), which might cause problems during hibernation.

Suggested-by: Ying Huang 
Suggested-by: Rafael J. Wysocki 
Signed-off-by: Chen Yu 
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas 
Reviewed-by: Rafael J. Wysocki 
Cc: Len Brown 
Cc: Dan Williams 
Cc: Rui Zhang 
Cc: Ying Huang 
Signed-off-by: Greg Kroah-Hartman

Merge branch 'pci/pm' into next

2016-10-03T14:43:21+00:00

* pci/pm:
  PCI: Avoid unnecessary resume after direct-complete
  PCI: Recognize D3cold in pci_update_current_state()
  PCI: Query platform firmware for device power state
  PCI: Afford direct-complete to devices with non-standard PM

PCI: Avoid unnecessary resume after direct-complete

2016-09-28T16:49:21+00:00

Commit 58a1fbbb2ee8 ("PM / PCI / ACPI: Kick devices that might have been
reset by firmware") added a runtime resume for devices that were runtime
suspended when the system entered sleep.

The motivation was that devices might be in a reset-power-on state after
waking from system sleep, so their power state as perceived by Linux
(stored in pci_dev->current_state) would no longer reflect reality.  By
resuming such devices, we allow them to return to a low-power state via
autosuspend and also bring their current_state in sync with reality.

However for devices that are *not* in a reset-power-on state, doing an
unconditional resume wastes energy.  A more refined approach is called for
which issues a runtime resume only if the power state after direct-complete
is shallower than it was before. To achieve this, update the device's
current_state and compare it to its pre-sleep value.

Signed-off-by: Lukas Wunner 
Signed-off-by: Bjorn Helgaas 
Acked-by: Rafael J. Wysocki

PCI: Drop CONFIG_KEXEC_CORE ifdeffery

2016-08-09T16:56:04+00:00

Drop the CONFIG_KEXEC_CORE #ifdef around reference to "kexec_in_progress".

Commit 2b94ed245861 ("kexec: define kexec_in_progress in !CONFIG_KEXEC
case") has made this unnecessary.

Signed-off-by: Lukas Wunner 
Signed-off-by: Bjorn Helgaas

PCI: Put PCIe ports into D3 during suspend

2016-06-13T19:57:36+00:00

Currently the Linux PCI core does not touch power state of PCI bridges and
PCIe ports when system suspend is entered.  Leaving them in D0 consumes
power unnecessarily and may prevent the CPU from entering deeper C-states.

With recent PCIe hardware we can power down the ports to save power given
that we take into account few restrictions:

  - The PCIe port hardware is recent enough, starting from 2015.

  - Devices connected to PCIe ports are effectively in D3cold once the port
    is transitioned to D3 (the config space is not accessible anymore and
    the link may be powered down).

  - Devices behind the PCIe port need to be allowed to transition to D3cold
    and back.  There is a way both drivers and userspace can forbid this.

  - If the device behind the PCIe port is capable of waking the system it
    needs to be able to do so from D3cold.

This patch adds a new flag to struct pci_device called 'bridge_d3'.  This
flag is set and cleared by the PCI core whenever there is a change in power
management state of any of the devices behind the PCIe port.  When system
later on is suspended we only need to check this flag and if it is true
transition the port to D3 otherwise we leave it in D0.

Also provide override mechanism via command line parameter
"pcie_port_pm=[off|force]" that can be used to disable or enable the
feature regardless of the BIOS manufacturing date.

Tested-by: Lukas Wunner 
Signed-off-by: Mika Westerberg 
Signed-off-by: Bjorn Helgaas 
Acked-by: Rafael J. Wysocki

PCI / PM: Tune down retryable runtime suspend error messages

2015-12-02T14:24:21+00:00

The runtime PM core doesn't treat EBUSY and EAGAIN retvals from the driver
suspend hooks as errors, but they still show up as errors in dmesg. Tune
them down. See rpm_suspend() for details of handling these return values.

Note that we use dev_dbg() for the retryable retvals, so after this
change you'll need either CONFIG_DYNAMIC_DEBUG or CONFIG_PCI_DEBUG
for them to show up in the log.

One problem caused by this was noticed by Daniel: the i915 driver
returns EAGAIN to signal a temporary failure to suspend and as a request
towards the RPM core for scheduling a suspend again. This is a normal
event, but the resulting error message flags a breakage during the
driver's automated testing which parses dmesg and picks up the error.

Reported-by: Daniel Vetter 
Link: https://bugs.freedesktop.org/show_bug.cgi?id=92992
Signed-off-by: Imre Deak 
Acked-by: Bjorn Helgaas 
Signed-off-by: Rafael J. Wysocki

Merge tag 'pci-v4.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

2015-11-06T19:29:53+00:00

Pull PCI updates from Bjorn Helgaas:
 "Resource management:
   - Add support for Enhanced Allocation devices (Sean O. Stalley)
   - Add Enhanced Allocation register entries (Sean O. Stalley)
   - Handle IORESOURCE_PCI_FIXED when sizing resources (David Daney)
   - Handle IORESOURCE_PCI_FIXED when assigning resources (David Daney)
   - Handle Enhanced Allocation capability for SR-IOV devices (David Daney)
   - Clear IORESOURCE_UNSET when reverting to firmware-assigned address (Bjorn Helgaas)
   - Make Enhanced Allocation bitmasks more obvious (Bjorn Helgaas)
   - Expand Enhanced Allocation BAR output (Bjorn Helgaas)
   - Add of_pci_check_probe_only to parse "linux,pci-probe-only" (Marc Zyngier)
   - Fix lookup of linux,pci-probe-only property (Marc Zyngier)
   - Add sparc mem64 resource parsing for root bus (Yinghai Lu)

  PCI device hotplug:
   - pciehp: Queue power work requests in dedicated function (Guenter Roeck)

  Driver binding:
   - Add builtin_pci_driver() to avoid registration boilerplate (Paul Gortmaker)

  Virtualization:
   - Set SR-IOV NumVFs to zero after enumeration (Alexander Duyck)
   - Remove redundant validation of SR-IOV offset/stride registers (Alexander Duyck)
   - Remove VFs in reverse order if virtfn_add() fails (Alexander Duyck)
   - Reorder pcibios_sriov_disable() (Alexander Duyck)
   - Wait 1 second between disabling VFs and clearing NumVFs (Alexander Duyck)
   - Fix sriov_enable() error path for pcibios_enable_sriov() failures (Alexander Duyck)
   - Enable SR-IOV ARI Capable Hierarchy before reading TotalVFs (Ben Shelton)
   - Don't try to restore VF BARs (Wei Yang)

  MSI:
   - Don't alloc pcibios-irq when MSI is enabled (Joerg Roedel)
   - Add msi_controller setup_irqs() method for special multivector setup (Lucas Stach)
   - Export all remapped MSIs to sysfs attributes (Romain Bezut)
   - Disable MSI on SiS 761 (Ondrej Zary)

  AER:
   - Clear error status registers during enumeration and restore (Taku Izumi)

  Generic host bridge driver:
   - Fix lookup of linux,pci-probe-only property (Marc Zyngier)
   - Allow multiple hosts with different map_bus() methods (David Daney)
   - Pass starting bus number to pci_scan_root_bus() (David Daney)
   - Fix address window calculation for non-zero starting bus (David Daney)

  Altera host bridge driver:
   - Add msi.h to ARM Kbuild (Ley Foon Tan)
   - Add Altera PCIe host controller driver (Ley Foon Tan)
   - Add Altera PCIe MSI driver (Ley Foon Tan)

  APM X-Gene host bridge driver:
   - Remove msi_controller assignment (Duc Dang)

  Broadcom iProc host bridge driver:
   - Fix header comment "Corporation" misspelling (Florian Fainelli)
   - Fix code comment to match code (Ray Jui)
   - Remove unused struct iproc_pcie.irqs[] (Ray Jui)
   - Call pci_fixup_irqs() for ARM64 as well as ARM (Ray Jui)
   - Fix PCIe reset logic (Ray Jui)
   - Improve link detection logic (Ray Jui)
   - Update PCIe device tree bindings (Ray Jui)
   - Add outbound mapping support (Ray Jui)

  Freescale i.MX6 host bridge driver:
   - Return real error code from imx6_add_pcie_port() (Fabio Estevam)
   - Add PCIE_PHY_RX_ASIC_OUT_VALID definition (Fabio Estevam)

  Freescale Layerscape host bridge driver:
   - Remove ls_pcie_establish_link() (Minghuan Lian)
   - Ignore PCIe controllers in Endpoint mode (Minghuan Lian)
   - Factor out SCFG related function (Minghuan Lian)
   - Update ls_add_pcie_port() (Minghuan Lian)
   - Remove unused fields from struct ls_pcie (Minghuan Lian)
   - Add support for LS1043a and LS2080a (Minghuan Lian)
   - Add ls_pcie_msi_host_init() (Minghuan Lian)

  HiSilicon host bridge driver:
   - Add HiSilicon SoC Hip05 PCIe driver (Zhou Wang)

  Marvell MVEBU host bridge driver:
   - Return zero for reserved or unimplemented config space (Russell King)
   - Use exact config access size; don't read/modify/write (Russell King)
   - Use of_get_available_child_count() (Russell King)
   - Use for_each_available_child_of_node() to walk child nodes (Russell King)
   - Report full node name when reporting a DT error (Russell King)
   - Use port->name rather than "PCIe%d.%d" (Russell King)
   - Move port parsing and resource claiming to  separate function (Russell King)
   - Fix memory leaks and refcount leaks (Russell King)
   - Split port parsing and resource claiming from  port setup (Russell King)
   - Use gpio_set_value_cansleep() (Russell King)
   - Use devm_kcalloc() to allocate an array (Russell King)
   - Use gpio_desc to carry around gpio (Russell King)
   - Improve clock/reset handling (Russell King)
   - Add PCI Express root complex capability block (Russell King)
   - Remove code restricting accesses to slot 0 (Russell King)

  NVIDIA Tegra host bridge driver:
   - Wrap static pgprot_t initializer with __pgprot() (Ard Biesheuvel)

  Renesas R-Car host bridge driver:
   - Build pci-rcar-gen2.c only on ARM (Geert Uytterhoeven)
   - Build pcie-rcar.c only on ARM (Geert Uytterhoeven)
   - Make PCI aware of the I/O resources (Phil Edworthy)
   - Remove dependency on ARM-specific struct hw_pci (Phil Edworthy)
   - Set root bus nr to that provided in DT (Phil Edworthy)
   - Fix I/O offset for multiple host bridges (Phil Edworthy)

  ST Microelectronics SPEAr13xx host bridge driver:
   - Fix dw_pcie_cfg_read/write() usage (Gabriele Paoloni)

  Synopsys DesignWare host bridge driver:
   - Make "clocks" and "clock-names" optional DT properties (Bhupesh Sharma)
   - Use exact access size in dw_pcie_cfg_read() (Gabriele Paoloni)
   - Simplify dw_pcie_cfg_read/write() interfaces (Gabriele Paoloni)
   - Require config accesses to be naturally aligned (Gabriele Paoloni)
   - Make "num-lanes" an optional DT property (Gabriele Paoloni)
   - Move calculation of bus addresses to DRA7xx (Gabriele Paoloni)
   - Replace ARM pci_sys_data->align_resource with global function pointer (Gabriele Paoloni)
   - Factor out MSI msg setup (Lucas Stach)
   - Implement multivector MSI IRQ setup (Lucas Stach)
   - Make get_msi_addr() return phys_addr_t, not u32 (Lucas Stach)
   - Set up high part of MSI target address (Lucas Stach)
   - Fix PORT_LOGIC_LINK_WIDTH_MASK (Zhou Wang)
   - Revert "PCI: designware: Program ATU with untranslated address" (Zhou Wang)
   - Use of_pci_get_host_bridge_resources() to parse DT (Zhou Wang)
   - Make driver arch-agnostic (Zhou Wang)

  Miscellaneous:
   - Make x86 pci_subsys_init() static (Alexander Kuleshov)
   - Turn off Request Attributes to avoid Chelsio T5 Completion erratum (Hariprasad Shenai)"

* tag 'pci-v4.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (94 commits)
  PCI: altera: Add Altera PCIe MSI driver
  PCI: hisi: Add HiSilicon SoC Hip05 PCIe driver
  PCI: layerscape: Add ls_pcie_msi_host_init()
  PCI: layerscape: Add support for LS1043a and LS2080a
  PCI: layerscape: Remove unused fields from struct ls_pcie
  PCI: layerscape: Update ls_add_pcie_port()
  PCI: layerscape: Factor out SCFG related function
  PCI: layerscape: Ignore PCIe controllers in Endpoint mode
  PCI: layerscape: Remove ls_pcie_establish_link()
  PCI: designware: Make "clocks" and "clock-names" optional DT properties
  PCI: designware: Make driver arch-agnostic
  ARM/PCI: Replace pci_sys_data->align_resource with global function pointer
  PCI: designware: Use of_pci_get_host_bridge_resources() to parse DT
  Revert "PCI: designware: Program ATU with untranslated address"
  PCI: designware: Move calculation of bus addresses to DRA7xx
  PCI: designware: Make "num-lanes" an optional DT property
  PCI: designware: Require config accesses to be naturally aligned
  PCI: designware: Simplify dw_pcie_cfg_read/write() interfaces
  PCI: designware: Use exact access size in dw_pcie_cfg_read()
  PCI: spear: Fix dw_pcie_cfg_read/write() usage
  ...

PM / PCI / ACPI: Kick devices that might have been reset by firmware

2015-10-14T00:17:34+00:00

There is a concern that if the platform firmware was involved in
the system resume that's being completed,  some devices might have
been reset by it and if those devices had the power.direct_complete
flag set during the preceding suspend transition, they may stay
in a reset-power-on state indefinitely (until they are runtime-resumed
and then suspended again).  That may not be a big deal from the
individual device's perspective, but if the system is an SoC, it may
be prevented from entering deep SoC-wide low-power states on idle
because of that.

The devices that are most likely to be affected by this issue are
PCI devices and ACPI-enumerated devices using the general ACPI PM
domain, so to prevent it from happening for those devices, force a
runtime resume for them if they have their power.direct_complete
flags set and the platform firmware was involved in the resume
transition currently in progress.

Signed-off-by: Rafael J. Wysocki