linux-toradex.git/drivers/pci, branch v4.4.73

PCI: Freeze PME scan before suspending devices

2017-05-25T12:30:17+00:00

commit ea00353f36b64375518662a8ad15e39218a1f324 upstream.

Laurent Pinchart reported that the Renesas R-Car H2 Lager board (r8a7790)
crashes during suspend tests.  Geert Uytterhoeven managed to reproduce the
issue on an M2-W Koelsch board (r8a7791):

  It occurs when the PME scan runs, once per second.  During PME scan, the
  PCI host bridge (rcar-pci) registers are accessed while its module clock
  has already been disabled, leading to the crash.

One reproducer is to configure s2ram to use "s2idle" instead of "deep"
suspend:

  # echo 0 > /sys/module/printk/parameters/console_suspend
  # echo s2idle > /sys/power/mem_sleep
  # echo mem > /sys/power/state

Another reproducer is to write either "platform" or "processors" to
/sys/power/pm_test.  It does not (or is less likely) to happen during full
system suspend ("core" or "none") because system suspend also disables
timers, and thus the workqueue handling PME scans no longer runs.  Geert
believes the issue may still happen in the small window between disabling
module clocks and disabling timers:

  # echo 0 > /sys/module/printk/parameters/console_suspend
  # echo platform > /sys/power/pm_test    # Or "processors"
  # echo mem > /sys/power/state

(Make sure CONFIG_PCI_RCAR_GEN2 and CONFIG_USB_OHCI_HCD_PCI are enabled.)

Rafael Wysocki agrees that PME scans should be suspended before the host
bridge registers become inaccessible.  To that end, queue the task on a
workqueue that gets frozen before devices suspend.

Rafael notes however that as a result, some wakeup events may be missed if
they are delivered via PME from a device without working IRQ (which hence
must be polled) and occur after the workqueue has been frozen.  If that
turns out to be an issue in practice, it may be possible to solve it by
calling pci_pme_list_scan() once directly from one of the host bridge's
pm_ops callbacks.

Stacktrace for posterity:

  PM: Syncing filesystems ... [   38.566237] done.
  PM: Preparing system for sleep (mem)
  Freezing user space processes ... [   38.579813] (elapsed 0.001 seconds) done.
  Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
  PM: Suspending system (mem)
  PM: suspend of devices complete after 152.456 msecs
  PM: late suspend of devices complete after 2.809 msecs
  PM: noirq suspend of devices complete after 29.863 msecs
  suspend debug: Waiting for 5 second(s).
  Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
  pgd = c0003000
  [00000000] *pgd=80000040004003, *pmd=00000000
  Internal error: : 1211 [#1] SMP ARM
  Modules linked in:
  CPU: 1 PID: 20 Comm: kworker/1:1 Not tainted
  4.9.0-rc1-koelsch-00011-g68db9bc814362e7f #3383
  Hardware name: Generic R8A7791 (Flattened Device Tree)
  Workqueue: events pci_pme_list_scan
  task: eb56e140 task.stack: eb58e000
  PC is at pci_generic_config_read+0x64/0x6c
  LR is at rcar_pci_cfg_base+0x64/0x84
  pc : []    lr : []    psr: 600d0093
  sp : eb58fe98  ip : c041d750  fp : 00000008
  r10: c0e2283c  r9 : 00000000  r8 : 600d0013
  r7 : 00000008  r6 : eb58fed6  r5 : 00000002  r4 : eb58feb4
  r3 : 00000000  r2 : 00000044  r1 : 00000008  r0 : 00000000
  Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
  Control: 30c5387d  Table: 6a9f6c80  DAC: 55555555
  Process kworker/1:1 (pid: 20, stack limit = 0xeb58e210)
  Stack: (0xeb58fe98 to 0xeb590000)
  fe80:                                                       00000002 00000044
  fea0: eb6f5800 c041d9b0 eb58feb4 00000008 00000044 00000000 eb78a000 eb78a000
  fec0: 00000044 00000000 eb9aff00 c0424bf0 eb78a000 00000000 eb78a000 c0e22830
  fee0: ea8a6fc0 c0424c5c eaae79c0 c0424ce0 eb55f380 c0e22838 eb9a9800 c0235fbc
  ff00: eb55f380 c0e22838 eb55f380 eb9a9800 eb9a9800 eb58e000 eb9a9824 c0e02100
  ff20: eb55f398 c02366c4 eb56e140 eb5631c0 00000000 eb55f380 c023641c 00000000
  ff40: 00000000 00000000 00000000 c023a928 cd105598 00000000 40506a34 eb55f380
  ff60: 00000000 00000000 dead4ead ffffffff ffffffff eb58ff74 eb58ff74 00000000
  ff80: 00000000 dead4ead ffffffff ffffffff eb58ff90 eb58ff90 eb58ffac eb5631c0
  ffa0: c023a844 00000000 00000000 c0206d68 00000000 00000000 00000000 00000000
  ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
  ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 3a81336c 10ccd1dd
  [] (pci_generic_config_read) from []
  (pci_bus_read_config_word+0x58/0x80)
  [] (pci_bus_read_config_word) from []
  (pci_check_pme_status+0x34/0x78)
  [] (pci_check_pme_status) from [] (pci_pme_wakeup+0x28/0x54)
  [] (pci_pme_wakeup) from [] (pci_pme_list_scan+0x58/0xb4)
  [] (pci_pme_list_scan) from []
  (process_one_work+0x1bc/0x308)
  [] (process_one_work) from [] (worker_thread+0x2a8/0x3e0)
  [] (worker_thread) from [] (kthread+0xe4/0xfc)
  [] (kthread) from [] (ret_from_fork+0x14/0x2c)
  Code: ea000000 e5903000 f57ff04f e3a00000 (e5843000)
  ---[ end trace 667d43ba3aa9e589 ]---

Fixes: df17e62e5bff ("PCI: Add support for polling PME state on suspended legacy PCI devices")
Reported-and-tested-by: Laurent Pinchart 
Reported-and-tested-by: Geert Uytterhoeven 
Signed-off-by: Lukas Wunner 
Signed-off-by: Bjorn Helgaas 
Reviewed-by: Laurent Pinchart 
Acked-by: Rafael J. Wysocki 
Cc: Mika Westerberg 
Cc: Niklas Söderlund 
Cc: Simon Horman 
Cc: Yinghai Lu 
Cc: Matthew Garrett 
Signed-off-by: Greg Kroah-Hartman

PCI: Fix pci_mmap_fits() for HAVE_PCI_RESOURCE_TO_USER platforms

2017-05-25T12:30:17+00:00

commit 6bccc7f426abd640f08d8c75fb22f99483f201b4 upstream.

In the PCI_MMAP_PROCFS case when the address being passed by the user is a
'user visible' resource address based on the bus window, and not the actual
contents of the resource, that's what we need to be checking it against.

Signed-off-by: David Woodhouse 
Signed-off-by: Bjorn Helgaas 
Signed-off-by: Greg Kroah-Hartman

PCI: Do any VF BAR updates before enabling the BARs

2017-03-30T07:35:20+00:00

From: Gavin Shan 

[ Upstream commit f40ec3c748c6912f6266c56a7f7992de61b255ed ]

Previously we enabled VFs and enable their memory space before calling
pcibios_sriov_enable().  But pcibios_sriov_enable() may update the VF BARs:
for example, on PPC PowerNV we may change them to manage the association of
VFs to PEs.

Because 64-bit BARs cannot be updated atomically, it's unsafe to update
them while they're enabled.  The half-updated state may conflict with other
devices in the system.

Call pcibios_sriov_enable() before enabling the VFs so any BAR updates
happen while the VF BARs are disabled.

[bhelgaas: changelog]
Tested-by: Carol Soto 
Signed-off-by: Gavin Shan 
Signed-off-by: Bjorn Helgaas 

Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Ignore BAR updates on virtual functions

2017-03-30T07:35:20+00:00

From: Bjorn Helgaas 

[ Upstream commit 63880b230a4af502c56dde3d4588634c70c66006 ]

VF BARs are read-only zero, so updating VF BARs will not have any effect.
See the SR-IOV spec r1.1, sec 3.4.1.11.

We already ignore these updates because of 70675e0b6a1a ("PCI: Don't try to
restore VF BARs"); this merely restructures it slightly to make it easier
to split updates for standard and SR-IOV BARs.

Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Update BARs using property bits appropriate for type

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 45d004f4afefdd8d79916ee6d97a9ecd94bb1ffe ]

The BAR property bits (0-3 for memory BARs, 0-1 for I/O BARs) are supposed
to be read-only, but we do save them in res->flags and include them when
updating the BAR.

Mask the I/O property bits with ~PCI_BASE_ADDRESS_IO_MASK (0x3) instead of
PCI_REGION_FLAG_MASK (0xf) to make it obvious that we can't corrupt bits
2-3 of I/O addresses.

Use PCI_ROM_ADDRESS_MASK for ROM BARs.  This means we'll only check the top
21 bits (instead of the 28 bits we used to check) of a ROM BAR to see if
the update was successful.

Signed-off-by: Bjorn Helgaas 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Don't update VF BARs while VF memory space is enabled

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 546ba9f8f22f71b0202b6ba8967be5cc6dae4e21 ]

If we update a VF BAR while it's enabled, there are two potential problems:

  1) Any driver that's using the VF has a cached BAR value that is stale
     after the update, and

  2) We can't update 64-bit BARs atomically, so the intermediate state
     (new lower dword with old upper dword) may conflict with another
     device, and an access by a driver unrelated to the VF may cause a bus
     error.

Warn about attempts to update VF BARs while they are enabled.  This is a
programming error, so use dev_WARN() to get a backtrace.

Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Decouple IORESOURCE_ROM_ENABLE and PCI_ROM_ADDRESS_ENABLE

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 7a6d312b50e63f598f5b5914c4fd21878ac2b595 ]

Remove the assumption that IORESOURCE_ROM_ENABLE == PCI_ROM_ADDRESS_ENABLE.
PCI_ROM_ADDRESS_ENABLE is the ROM enable bit defined by the PCI spec, so if
we're reading or writing a BAR register value, that's what we should use.
IORESOURCE_ROM_ENABLE is a corresponding bit in struct resource flags.

Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Add comments about ROM BAR updating

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 0b457dde3cf8b7c76a60f8e960f21bbd4abdc416 ]

pci_update_resource() updates a hardware BAR so its address matches the
kernel's struct resource UNLESS it's a disabled ROM BAR.  We only update
those when we enable the ROM.

It's not obvious from the code why ROM BARs should be handled specially.
Apparently there are Matrox devices with defective ROM BARs that read as
zero when disabled.  That means that if pci_enable_rom() reads the disabled
BAR, sets PCI_ROM_ADDRESS_ENABLE (without re-inserting the address), and
writes it back, it would enable the ROM at address zero.

Add comments and references to explain why we can't make the code look more
rational.

The code changes are from 755528c860b0 ("Ignore disabled ROM resources at
setup") and 8085ce084c0f ("[PATCH] Fix PCI ROM mapping").

Link: https://lkml.org/lkml/2005/8/30/138
Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
 [sumits: minor fixup in rom.c for 4.4.y]
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Remove pci_resource_bar() and pci_iov_resource_bar()

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 286c2378aaccc7343ebf17ec6cd86567659caf70 ]

pci_std_update_resource() only deals with standard BARs, so we don't have
to worry about the complications of VF BARs in an SR-IOV capability.

Compute the BAR address inline and remove pci_resource_bar().  That makes
pci_iov_resource_bar() unused, so remove that as well.

Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman

PCI: Separate VF BAR updates from standard BAR updates

2017-03-30T07:35:19+00:00

From: Bjorn Helgaas 

[ Upstream commit 6ffa2489c51da77564a0881a73765ea2169f955d ]

Previously pci_update_resource() used the same code path for updating
standard BARs and VF BARs in SR-IOV capabilities.

Split the VF BAR update into a new pci_iov_update_resource() internal
interface, which makes it simpler to compute the BAR address (we can get
rid of pci_resource_bar() and pci_iov_resource_bar()).

This patch:

  - Renames pci_update_resource() to pci_std_update_resource(),
  - Adds pci_iov_update_resource(),
  - Makes pci_update_resource() a wrapper that calls the appropriate one,

No functional change intended.

Signed-off-by: Bjorn Helgaas 
Reviewed-by: Gavin Shan 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman 
Signed-off-by: Sumit Semwal 
Signed-off-by: Greg Kroah-Hartman