diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-23 15:34:47 -1000 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-23 15:34:47 -1000 |
| commit | 17bb0e2f6c8043f70a693a5f3797471cbed354c8 (patch) | |
| tree | d330ca8bd3d9515c7639dacd8d45458224349c04 /drivers/pci/controller | |
| parent | 4da9dedf3fc59137fd86d664693776aa38cb97f5 (diff) | |
| parent | f4319f548f7d8cb595ac1c91bfd459876399eeb3 (diff) | |
Merge tag 'pci-v7.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci
Pull PCI updates from Bjorn Helgaas:
"Enumeration:
- Set supported_speeds in set_pcie_port_type() so sparc64 and
powerpc, which call it via of_create_pci_dev() instead of the usual
pci_setup_device(), correctly set the supported PCIe link speeds
(Stian Halseth)
- When pcie_set_mps() changes MPS, update the value in the saved
state so a subsequent restore doesn't restore an invalid value
(Niklas Cassel)
- Skip Target Speed quirk on clamped ports with no link (Maciej W.
Rozycki)
Driver binding:
- Assume control of portdrv-related features only when portdrv is
enabled, since we can't use them without portdrv (Kuppuswamy
Sathyanarayanan)
- Centralize feature ownership checking so code only needs to check
'host->native_X' and doesn't need to also check 'pcie_ports_native'
(Kuppuswamy Sathyanarayanan)
Error handling:
- Fix leaks when ACPI APEI/GHES error events overflow the kfifo or
mention a device that no longer exists (Priyank Rathod)
- Fix aer_process_err_devices() leak when a device is recorded as an
error source purely by being mentioned in the Root Port Error
Source and the device is inaccessible or logged only masked errors
(Priyank Rathod)
ASPM:
- Disable ASPM when drivers request that, even if the OS doesn't own
ASPM; otherwise drivers do it on their own, which is worse (Ilpo
Järvinen)
Power management:
- Align hibernate poweroff flow with suspend flow for bridges so they
can be powered down at hibernate as they are for suspend (Mario
Limonciello)
- Restore BARs after powering up during enumeration to fix a
regression on devices that don't preserve state in D3hot (Keng-Yu
Lin)
- Avoid D3 on PLX PEX 8532/8533/8548 downstream ports, where putting
them in D3hot takes down the whole switch (Imre Kaloz)
Power control:
- Add GPIO driver for Toshiba TC9563 embedded GPIO controller (Alex
Elder)
- Add DT documentation and pwrctrl driver support for TC9563 embedded
GPIO controller (Lorenzo Bianconi)
- Switch the tc9563 per-port reset to use the GPIO descriptor API
instead of direct regmap accesses (Lorenzo Bianconi)
- Only destroy devices created by pwrctrl core, so that when removing
a PCIe controller driver that uses pwrctrl, we don't remove devices
populated by another mechanism (Shawn Lin)
- Power off child devices on power-on failure (Lorenzo Bianconi)
PCIe native device hotplug:
- Make the 'pciehp_poll_mode' parameter read-only to prevent changes
via sysfs, which can cause errors when unbinding (Runyu Xiao)
Virtualization:
- Read SR-IOV VF Revision ID and Subsystem ID from VF instead of
assuming they're the same as VF 0; they may differ (Logan Odell)
- Add ACS quirk for Pericom PI7C9X2G304 switches (Hans de Goede)
Peer-to-peer DMA:
- Add Zhaoxin KX-6000, KH-40000, KX-6000G, KX-7000, and KH-50000 host
bridges to allowlist (LeoLiu-oc)
- Add Alibaba T-HEAD Yitian 710 host bridge to allowlist (Feng Tang)
- Add Intel Haswell client host bridge to allowlist (Scott Lee)
- Intel Xeon E3/E5/E7 0x6f00 host bridge to allowlist (kai zhang)
- Fix handling of errors in pci_p2pdma_add_resource() to avoid (a)
unmapping a BAR if registering another fails and (b) use after free
of p2p_pgmap (Leon Romanovsky)
- Wait for RCU readers before freeing state to avoid use-after-free
of pdev->p2pdma (Leon Romanovsky)
- Restrict pci_p2pmem_find_many() to providers backed by a pool (Leon
Romanovsky)
- Safely terminate ACS redirect lists to avoid out-of-bounds access
to the buffer (Leon Romanovsky)
- Silence 'Host bridge not in P2PDMA whitelist' warnings unless
caller, e.g., pci_p2pmem_find_many(), requests 'verbose' (Leon
Romanovsky)
Sysfs:
- Return -ENODEV or -EIO sysfs 'label' or 'acpi_index' attributes
when Device Name _DSM fails, not -1, which is interpreted as -EPERM
(Krzysztof Wilczyński)
- Return valid sysfs 'acpi_index' attribute even if the Device Name
_DSM returned an invalid 'label' (Krzysztof Wilczyński)
- Expose sysfs 'acpi_index' and 'label' based on whether the Device
Name _DSM returns valid data, not merely whether the _DSM is
implemented (Krzysztof Wilczyński)
- Avoid out-of-bounds reads of the label returned by the Device Name
_DSM, which is in UTF-16 characters, not bytes (Krzysztof
Wilczyński)
- Reject unaligned resource I/O port accesses, which can oops on
arm64 (Deepanshu Kartikey)
Endpoint framework:
- Pass the correct physical and virtual endpoint function numbers
when setting translation of inbound memory windows (Koichiro Den)
- Make embedded doorbell IRQ non-shared, since currently only the
first EPF can allocate doorbells (Koichiro Den)
- MSI doorbells currently target the first EPF attached to an EPC;
let other EPFs try the embedded doorbell instead of failing when
they can't use the MSI doorbell (Koichiro Den)
- Serialize scan of vNTB virtual PCI bus with other PCI topology
changes (Koichiro Den)
- Manage vNTB lifetimes to avoid leaking virtual devices and buses
and allow removal (Koichiro Den)
- Track vNTB link state from both sides so both see the same
effective link state (Koichiro Den)
- Configure vNTB MSI-X when supported to avoid IOMMU faults and
failure to deliver doorbell interrupts on RC (Koichiro Den)
APM X-Gene PCIe controller driver:
- Use devm for clocks so they're automatically disabled, unprepared,
and released if errors happen during probe (Yuho Choi)
ASPEED PCIe controller driver:
- Fix clk and PHY leaks in aspeed_pcie_port_init() error paths (Liu
Zhenlong)
Broadcom iProc PCIe controller driver:
- Expose devm_pci_alloc_host_bridge_release() so drivers can share a
devm release function even if they don't need the device tree
parsing done by devm_pci_alloc_host_bridge() (Bjorn Helgaas)
- Request only the memory window from BCMA Enumeration ROM and warn
if it doesn't match the one in DT, which fixes an iproc-bcma probe
failure (Semih Baskan)
Broadcom STB PCIe controller driver:
- Replace bool properties for each SoC configuration with a u32
bitvector and a BFLAG() helper (Jim Quinlan)
- Use a similar BQUIRK() helper to test various SoC quirks (Jim
Quinlan)
Cadence PCIe controller driver:
- Remove device links to PHY, which are redundant because the PHY
core itself adds them (Vladimir Oltean)
- Return error (not success) from cdns_plat_pcie_probe() if
pm_runtime_get_sync() or cdns_pcie_ep_setup() failed (Vladimir
Oltean)
- Correct the driver data pointer usage; previously .probe() stored
'struct cdns_plat_pcie *', but .shutdown() used the pointer as
'struct cdns_pcie *', resulting in accessing garbage (Linmao Li)
ESWIN PCIe controller driver:
- Publish the platform driver MODULE_DEVICE_TABLE so the driver can
be autoloaded when built as a module (Pengpeng Hou)
Freescale i.MX6 PCIe controller driver:
- Fix resource and pwrctrl device leaks .probe() (Zhijian Han)
- Improve i.MX95 Gen3 link stability by changing MPLLB_BANDWIDTH from
default 120 to 140 (Richard Zhu)
- Use dev_err_probe() in host init error path to silence
-EPROBE_DEFER messages (Alexander Stein)
- Avoid dereferencing NULL clock names when 'clocks' includes more
phandles than 'clock-names' has names (Rudi Heitbaum)
- Add suspend_poweroff flag so platforms like i.MX that power off RC
controllers during suspend can enter L2 so endpoints can get
PME_Turn_Off notifications before power loss (Richard Zhu)
Freescale Layerscape PCIe controller driver:
- Enable PROBE_PREFER_ASYNCHRONOUS to reduce probe time of the four
controllers on LX2160A (Vincent Jardin)
Intel VMD host bridge driver:
- Flush DMA writes before handling MSI on Meteor Lake: MSI may arrive
before write has completed, resulting in an nvme 30 second stall
before a QID timeout and possibly other similar problems (Rickey
Bartlett)
- Flush DMA writes before handling MSI on Arrow Lake (Vladimir
Nedoshivin)
Marvell Aardvark PCIe controller driver:
- Disable PHY on probe failures (Myeonghun Pak)
Marvell MVEBU PCIe controller driver:
- Refcount child OF nodes during .probe() to make a concurrent OF
overlay removal safe (Krzysztof Kozlowski)
MediaTek PCIe controller driver:
- Find INTx DT property by 'interrupt-controller' name instead of
assuming it's the first child node (Zhengping Zhang)
MediaTek PCIe Gen3 controller driver:
- Fix a 64-bit type truncation in mtk_pcie_set_trans_table() to avoid
issues with windows above or larger than 4GB (Sreeraj S Kurup)
NVIDIA Tegra264 PCIe controller driver:
- Fix the Link Capabilities offset so we read the correct register
(Bjorn Helgaas)
Qualcomm PCIe controller driver:
- Only collect reset GPIOs from PCI bridge DT nodes to avoid stealing
resets used by downstream devices like USB controllers (Christopher
Obbard)
- Block accesses to downstream devices on link down to reduce a storm
of Completion Timeout AER interrupts (Qiang Yu)
- Skip system suspend/resume for firmware-managed PCIe, where
firmware manages clocks, regulators, resets, etc (Mrinmay Sarkar)
- Honor IOMMU provider's #iommu-cells in qcom_pcie_config_sid_1_9_0()
(Manivannan Sadhasivam)
- Deinitialize endpoint on remove (Guangshuo Li)
- Treat Root Port error recovery as successful even if link doesn't
come up after reset because the device was hot-unplugged (Konrad
Dybcio)
- Skip unnecessary iMSI-RX initialization if DT 'msi-map' is present,
indicating an external MSI controller (Ayiluri Naga Rashmi)
Renesas R-Car PCIe controller driver:
- Limit Max_Read_Request_Size to 256 to avoid hardware erratum (Marek
Vasut)
- Support PERST# via reset subsystem as well as the existing GPIO
support to prepare for R-Car Gen5 SoCs (Marek Vasut)
- Add DT and driver support for the R-Car Gen 5 X5H (R8A78000) PCIe
Gen 4 core (Marek Vasut)
- Add Application/Local register reset DT description and driver
support for both host and endpoint mode (Marek Vasut)
- Add PM ops so controller works after suspend/resume (Marek Vasut)
Renesas RZ/G3S PCIe controller driver:
- Disable refclk on probe failure (Lad Prabhakar)
- Propagate platform_get_irq_byname() errors so probe deferral works
correctly (Lad Prabhakar)
- Tear down INTx handlers if subsequent initialization fails (Lad
Prabhakar)
- Add RZ/G3L (R9A08G046) DT and driver support (Biju Das)
Rockchip PCIe controller driver:
- Limit host and endpoint controllers to 2.5 GT/s since 5.0 GT/s is
unsafe per Rockchip (Geraldo Nascimento)
Synopsys DesignWare PCIe controller driver:
- Set DMA_BYPASS bit in outbound IATU CTRL2 register to avoid data
corruption if the iATU translates DMA engine transactions (Sumit
Kumar)
- Use FIELD_PREP() to remove need for *_SHIFT definitions (Hans
Zhang)
TI DRA7xx PCIe controller driver:
- Remove device links to PHY, which are redundant because the PHY
core itself adds them (Vladimir Oltean)
- Fix optional clock leak when .probe() fails by using
devm_clk_get_optional_enabled() (Felix Gu)
TI Keystone PCIe controller driver:
- Remove device links to PHY, which are redundant because the PHY
core itself adds them (Vladimir Oltean)
UltraRISC PCIe controller driver:
- Allow DWC drivers to skip outbound iATU setup if they don't need
translation between CPU and PCI addresses (Xincheng Zhang)
- Skip MEM outbound iATU setup on DP1000 (Xincheng Zhang)
Xilinx Versal CPM PCIe controller driver:
- Add DT and driver support for optional PERST# for use in warm reset
scenarios (Sai Krishna Musham)
Xilinx XDMA PCIe controller driver:
- Fix device node reference leak in xilinx_pl_dma_pcie_init_irq_domain()
(Wentao Liang)"
* tag 'pci-v7.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci: (139 commits)
PCI/pwrctrl: Power off child devices on power-on failure
PCI/pwrctrl: Only destroy devices created by pwrctrl core
PCI/pwrctrl: generic: Fix leaking array from of_regulator_bulk_get_all()
PCI/pwrctrl: tc9563: Switch per-port reset to GPIO descriptor API
PCI/pwrctrl: tc9563: Add GPIO auxiliary device support
dt-bindings: PCI: toshiba,tc9563: Document embedded GPIO controller
gpio: tc9563: Add support for the embedded GPIO controller
PCI: qcom: Skip DWC MSI host initialization when DT 'msi-map' is present
PCI: qcom: Ignore -ENODEV after Root Port reset
PCI: qcom-ep: Deinitialize endpoint on remove
PCI/sysfs: Reject unaligned resource I/O port accesses
PCI: rcar-gen4: Add missing PM ops
PCI: endpoint: pci-epf-vntb: Honor MSI-X selection
PCI: endpoint: pci-epf-vntb: Use a local epf pointer in interrupt setup
PCI: dwc: imx6: Use regmap_assign_bits() for conditional set/clear
PCI: dwc: Describe DWC PCIe fields with full masks and FIELD_PREP()
PCI: dwc: Rename pp->irq_domain to pp->msi_irq_domain
PCI: dwc: Rename dw_pcie_allocate_domains() to dw_pcie_allocate_msi_domains()
PCI: dwc: Correct PCIE_DW_PLAT_EP reference in Kconfig help
PCI/ASPM: Disable ASPM when driver requests it
...
Diffstat (limited to 'drivers/pci/controller')
45 files changed, 1370 insertions, 743 deletions
diff --git a/drivers/pci/controller/cadence/pcie-cadence-debugfs.c b/drivers/pci/controller/cadence/pcie-cadence-debugfs.c index 0a308f95e9f6..7b8354052aea 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-debugfs.c +++ b/drivers/pci/controller/cadence/pcie-cadence-debugfs.c @@ -210,24 +210,21 @@ static const char *cdns_pcie_hpa_ltssm_status_string(enum cdns_pcie_hpa_ltssm lt static int ltssm_status_show(struct seq_file *s, void *v) { struct cdns_pcie *pci = s->private; - enum cdns_pcie_lga_ltssm lga_ltssm; - enum cdns_pcie_hpa_ltssm hpa_ltssm; const char *str_ltssm; - u32 val; + u32 val, ltssm_val; if (pci->is_hpa) { val = cdns_pcie_hpa_readl(pci, REG_BANK_IP_REG, CDNS_PCIE_HPA_PHY_DBG_STS_REG0); - hpa_ltssm = FIELD_GET(CDNS_PCIE_HPA_LTSSM_STATUS_MASK, val); - str_ltssm = cdns_pcie_hpa_ltssm_status_string(hpa_ltssm); + ltssm_val = FIELD_GET(CDNS_PCIE_HPA_LTSSM_STATUS_MASK, val); + str_ltssm = cdns_pcie_hpa_ltssm_status_string(ltssm_val); } else { val = cdns_pcie_readl(pci, CDNS_PCIE_LM_BASE); - lga_ltssm = FIELD_GET(CDNS_PCIE_LGA_LTSSM_STATUS_MASK, val); - str_ltssm = cdns_pcie_lga_ltssm_status_string(lga_ltssm); + ltssm_val = FIELD_GET(CDNS_PCIE_LGA_LTSSM_STATUS_MASK, val); + str_ltssm = cdns_pcie_lga_ltssm_status_string(ltssm_val); } - seq_printf(s, "%s (0x%02x)\n", str_ltssm, - pci->is_hpa ? hpa_ltssm : lga_ltssm); + seq_printf(s, "%s (0x%02x)\n", str_ltssm, ltssm_val); return 0; } diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c index a1ea24fc3b63..4a24302cc5b2 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-plat.c +++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c @@ -14,14 +14,6 @@ #define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF -/** - * struct cdns_plat_pcie - private data for this PCIe platform driver - * @pcie: Cadence PCIe controller - */ -struct cdns_plat_pcie { - struct cdns_pcie *pcie; -}; - static const struct of_device_id cdns_plat_pcie_of_match[]; static u64 cdns_plat_cpu_addr_fixup(struct cdns_pcie *pcie, u64 cpu_addr) @@ -36,12 +28,11 @@ static const struct cdns_pcie_ops cdns_plat_ops = { static int cdns_plat_pcie_probe(struct platform_device *pdev) { const struct cdns_plat_pcie_of_data *data; - struct cdns_plat_pcie *cdns_plat_pcie; struct device *dev = &pdev->dev; struct pci_host_bridge *bridge; + struct cdns_pcie *pcie; struct cdns_pcie_ep *ep; struct cdns_pcie_rc *rc; - int phy_count; bool is_rc; int ret; @@ -52,11 +43,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) is_rc = data->is_rc; pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc); - cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL); - if (!cdns_plat_pcie) - return -ENOMEM; - - platform_set_drvdata(pdev, cdns_plat_pcie); if (is_rc) { if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST)) return -ENODEV; @@ -68,9 +54,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) rc = pci_host_bridge_priv(bridge); rc->pcie.dev = dev; rc->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &rc->pcie; + pcie = &rc->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -95,9 +81,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) ep->pcie.dev = dev; ep->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &ep->pcie; + pcie = &ep->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -115,18 +101,17 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) goto err_init; } + platform_set_drvdata(pdev, pcie); + return 0; err_init: err_get_sync: pm_runtime_put_sync(dev); pm_runtime_disable(dev); - cdns_pcie_disable_phy(cdns_plat_pcie->pcie); - phy_count = cdns_plat_pcie->pcie->phy_count; - while (phy_count--) - device_link_del(cdns_plat_pcie->pcie->link[phy_count]); + cdns_pcie_disable_phy(pcie); - return 0; + return ret; } static void cdns_plat_pcie_shutdown(struct platform_device *pdev) diff --git a/drivers/pci/controller/cadence/pcie-cadence.c b/drivers/pci/controller/cadence/pcie-cadence.c index a1eada56edba..0ac980249941 100644 --- a/drivers/pci/controller/cadence/pcie-cadence.c +++ b/drivers/pci/controller/cadence/pcie-cadence.c @@ -222,7 +222,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) struct device_node *np = dev->of_node; int phy_count; struct phy **phy; - struct device_link **link; int i; int ret; const char *name; @@ -238,10 +237,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) if (!phy) return -ENOMEM; - link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL); - if (!link) - return -ENOMEM; - for (i = 0; i < phy_count; i++) { of_property_read_string_index(np, "phy-names", i, &name); phy[i] = devm_phy_get(dev, name); @@ -249,17 +244,10 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) ret = PTR_ERR(phy[i]); goto err_phy; } - link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS); - if (!link[i]) { - devm_phy_put(dev, phy[i]); - ret = -EINVAL; - goto err_phy; - } } pcie->phy_count = phy_count; pcie->phy = phy; - pcie->link = link; ret = cdns_pcie_enable_phy(pcie); if (ret) @@ -268,10 +256,8 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie) return 0; err_phy: - while (--i >= 0) { - device_link_del(link[i]); + while (--i >= 0) devm_phy_put(dev, phy[i]); - } return ret; } diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h index b4d51adefea5..4a96e1869e53 100644 --- a/drivers/pci/controller/cadence/pcie-cadence.h +++ b/drivers/pci/controller/cadence/pcie-cadence.h @@ -260,7 +260,6 @@ struct cdns_plat_pcie_of_data { * @is_hpa: indicates if the architecture is HPA * @phy_count: number of supported PHY devices * @phy: list of pointers to specific PHY control blocks - * @link: list of pointers to corresponding device link representations * @ops: Platform-specific ops to control various inputs from Cadence PCIe * wrapper * @cdns_pcie_reg_offsets: Register bank offsets for different SoC @@ -276,7 +275,6 @@ struct cdns_pcie { bool is_hpa; int phy_count; struct phy **phy; - struct device_link **link; const struct cdns_pcie_ops *ops; const struct cdns_plat_pcie_of_data *cdns_pcie_reg_offsets; int max_link_speed; diff --git a/drivers/pci/controller/dwc/Kconfig b/drivers/pci/controller/dwc/Kconfig index dcfbe7e229fd..0a0d56fcf0ac 100644 --- a/drivers/pci/controller/dwc/Kconfig +++ b/drivers/pci/controller/dwc/Kconfig @@ -295,7 +295,7 @@ config PCIE_DW_PLAT_HOST Designware IP. This controller can work either as EP or RC. In order to enable host-specific features PCIE_DW_PLAT_HOST must be selected and in - order to enable device-specific features PCI_DW_PLAT_EP must be + order to enable device-specific features PCIE_DW_PLAT_EP must be selected. config PCIE_DW_PLAT_EP @@ -310,7 +310,7 @@ config PCIE_DW_PLAT_EP in Designware IP. This controller can work either as EP or RC. In order to enable host-specific features PCIE_DW_PLAT_HOST must be selected and in - order to enable device-specific features PCI_DW_PLAT_EP must be + order to enable device-specific features PCIE_DW_PLAT_EP must be selected. config PCIE_QCOM_COMMON diff --git a/drivers/pci/controller/dwc/pci-dra7xx.c b/drivers/pci/controller/dwc/pci-dra7xx.c index 6ae5b27e27b3..0d56fd5a19ea 100644 --- a/drivers/pci/controller/dwc/pci-dra7xx.c +++ b/drivers/pci/controller/dwc/pci-dra7xx.c @@ -9,7 +9,6 @@ #include <linux/clk.h> #include <linux/delay.h> -#include <linux/device.h> #include <linux/err.h> #include <linux/interrupt.h> #include <linux/irq.h> @@ -214,7 +213,7 @@ static int dra7xx_pcie_handle_msi(struct dw_pcie_rp *pp, int index) pos = find_first_bit(&val, MAX_MSI_IRQS_PER_CTRL); while (pos != MAX_MSI_IRQS_PER_CTRL) { - generic_handle_domain_irq(pp->irq_domain, + generic_handle_domain_irq(pp->msi_irq_domain, (index * MAX_MSI_IRQS_PER_CTRL) + pos); pos++; pos = find_next_bit(&val, MAX_MSI_IRQS_PER_CTRL, pos); @@ -680,7 +679,6 @@ static int dra7xx_pcie_probe(struct platform_device *pdev) int i; int phy_count; struct phy **phy; - struct device_link **link; void __iomem *base; struct dw_pcie *pci; struct dra7xx_pcie *dra7xx; @@ -728,30 +726,16 @@ static int dra7xx_pcie_probe(struct platform_device *pdev) if (!phy) return -ENOMEM; - link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL); - if (!link) - return -ENOMEM; - - dra7xx->clk = devm_clk_get_optional(dev, NULL); + dra7xx->clk = devm_clk_get_optional_enabled(dev, NULL); if (IS_ERR(dra7xx->clk)) return dev_err_probe(dev, PTR_ERR(dra7xx->clk), "clock request failed"); - ret = clk_prepare_enable(dra7xx->clk); - if (ret) - return ret; - for (i = 0; i < phy_count; i++) { snprintf(name, sizeof(name), "pcie-phy%d", i); phy[i] = devm_phy_get(dev, name); if (IS_ERR(phy[i])) return PTR_ERR(phy[i]); - - link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS); - if (!link[i]) { - ret = -EINVAL; - goto err_link; - } } dra7xx->base = base; @@ -853,10 +837,6 @@ err_get_sync: pm_runtime_disable(dev); dra7xx_pcie_disable_phy(dra7xx); -err_link: - while (--i >= 0) - device_link_del(link[i]); - return ret; } diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c index f7a2eb257c16..83fbd998551d 100644 --- a/drivers/pci/controller/dwc/pci-imx6.c +++ b/drivers/pci/controller/dwc/pci-imx6.c @@ -80,6 +80,18 @@ #define IMX95_SID_MASK GENMASK(5, 0) #define IMX95_MAX_LUT 32 +#define IMX95_PCIE_PHY_REG_ADDR 0x3008 +#define IMX95_PCIE_PHY_REG_EN BIT(31) +#define IMX95_PCIE_PHY_REG_ADDR_MASK GENMASK(15, 0) + +#define IMX95_PCIE_PHY_REG_DATA 0x300c + +#define IMX95_PCIE_PHY_MPLLB_OVRD_IN 0x2004 +#define IMX95_PCIE_PHY_MPLLB_OVRD_BW_EN BIT(10) + +#define IMX95_PCIE_PHY_MPLLB_BW_IN 0x2005 +#define IMX95_PCIE_PHY_MPLLB_BW_VAL 0x8c + #define IMX95_PCIE_RST_CTRL 0x3010 #define IMX95_PCIE_COLD_RST BIT(0) @@ -180,8 +192,12 @@ struct imx_pcie { struct imx_lut_data luts[IMX95_MAX_LUT]; /* power domain for pcie */ struct device *pd_pcie; + /* device link for pcie power domain */ + struct device_link *pd_link; /* power domain for pcie phy */ struct device *pd_pcie_phy; + /* device link for pcie phy power domain */ + struct device_link *pd_phy_link; struct phy *phy; const struct imx_pcie_drvdata *drvdata; @@ -270,8 +286,19 @@ static int imx95_pcie_select_ref_clk_src(struct imx_pcie *imx_pcie) return 0; } +static void imx95_pcie_phy_write(struct imx_pcie *imx_pcie, int addr, u16 data) +{ + regmap_update_bits(imx_pcie->iomuxc_gpr, IMX95_PCIE_PHY_REG_ADDR, + IMX95_PCIE_PHY_REG_EN, IMX95_PCIE_PHY_REG_EN); + regmap_update_bits(imx_pcie->iomuxc_gpr, IMX95_PCIE_PHY_REG_ADDR, + IMX95_PCIE_PHY_REG_ADDR_MASK, addr); + regmap_write(imx_pcie->iomuxc_gpr, IMX95_PCIE_PHY_REG_DATA, data); +} + static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie) { + u32 val; + /* * ERR051624: The Controller Without Vaux Cannot Exit L23 Ready * Through Beacon or PERST# De-assertion @@ -290,6 +317,21 @@ static int imx95_pcie_init_phy(struct imx_pcie *imx_pcie) IMX95_PCIE_PHY_CR_PARA_SEL, IMX95_PCIE_PHY_CR_PARA_SEL); + /* Flush the IMX95_PCIE_PHY_CR_PARA_SEL update */ + regmap_read(imx_pcie->iomuxc_gpr, IMX95_PCIE_SS_RW_REG_0, &val); + + /* + * A delay is required between the assertion of + * IMX95_PCIE_PHY_CR_PARA_SEL and subsequent PHY register write + * operation to ensure values are successfully written. + */ + udelay(200); + + imx95_pcie_phy_write(imx_pcie, IMX95_PCIE_PHY_MPLLB_BW_IN, + IMX95_PCIE_PHY_MPLLB_BW_VAL); + imx95_pcie_phy_write(imx_pcie, IMX95_PCIE_PHY_MPLLB_OVRD_IN, + IMX95_PCIE_PHY_MPLLB_OVRD_BW_EN); + return 0; } @@ -542,7 +584,7 @@ static int imx_setup_phy_mpll(struct imx_pcie *imx_pcie) return 0; for (i = 0; i < imx_pcie->num_clks; i++) - if (strncmp(clks[i].id, "pcie_phy", 8) == 0) + if (clks[i].id && strncmp(clks[i].id, "pcie_phy", 8) == 0) phy_rate = clk_get_rate(clks[i].clk); switch (phy_rate) { @@ -639,10 +681,34 @@ static int imx6q_pcie_abort_handler(unsigned long addr, } #endif +static void imx_pcie_detach_pd_action(void *data) +{ + struct imx_pcie *imx_pcie = data; + + if (!IS_ERR_OR_NULL(imx_pcie->pd_phy_link)) { + device_link_del(imx_pcie->pd_phy_link); + imx_pcie->pd_phy_link = NULL; + } + + if (!IS_ERR_OR_NULL(imx_pcie->pd_link)) { + device_link_del(imx_pcie->pd_link); + imx_pcie->pd_link = NULL; + } + + if (!IS_ERR_OR_NULL(imx_pcie->pd_pcie_phy)) { + dev_pm_domain_detach(imx_pcie->pd_pcie_phy, true); + imx_pcie->pd_pcie_phy = NULL; + } + + if (!IS_ERR_OR_NULL(imx_pcie->pd_pcie)) { + dev_pm_domain_detach(imx_pcie->pd_pcie, true); + imx_pcie->pd_pcie = NULL; + } +} + static int imx_pcie_attach_pd(struct device *dev) { struct imx_pcie *imx_pcie = dev_get_drvdata(dev); - struct device_link *link; /* Do nothing when in a single power domain */ if (dev->pm_domain) @@ -654,11 +720,11 @@ static int imx_pcie_attach_pd(struct device *dev) /* Do nothing when power domain missing */ if (!imx_pcie->pd_pcie) return 0; - link = device_link_add(dev, imx_pcie->pd_pcie, - DL_FLAG_STATELESS | - DL_FLAG_PM_RUNTIME | - DL_FLAG_RPM_ACTIVE); - if (!link) { + imx_pcie->pd_link = device_link_add(dev, imx_pcie->pd_pcie, + DL_FLAG_STATELESS | + DL_FLAG_PM_RUNTIME | + DL_FLAG_RPM_ACTIVE); + if (!imx_pcie->pd_link) { dev_err(dev, "Failed to add device_link to pcie pd\n"); return -EINVAL; } @@ -667,11 +733,11 @@ static int imx_pcie_attach_pd(struct device *dev) if (IS_ERR(imx_pcie->pd_pcie_phy)) return PTR_ERR(imx_pcie->pd_pcie_phy); - link = device_link_add(dev, imx_pcie->pd_pcie_phy, - DL_FLAG_STATELESS | - DL_FLAG_PM_RUNTIME | - DL_FLAG_RPM_ACTIVE); - if (!link) { + imx_pcie->pd_phy_link = device_link_add(dev, imx_pcie->pd_pcie_phy, + DL_FLAG_STATELESS | + DL_FLAG_PM_RUNTIME | + DL_FLAG_RPM_ACTIVE); + if (!imx_pcie->pd_phy_link) { dev_err(dev, "Failed to add device_link to pcie_phy pd\n"); return -EINVAL; } @@ -681,12 +747,8 @@ static int imx_pcie_attach_pd(struct device *dev) static int imx6q_pcie_enable_ref_clk(struct imx_pcie *imx_pcie, bool enable) { - if (enable) - regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_REF_CLK_EN); - else - regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_REF_CLK_EN); + regmap_assign_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, + IMX6Q_GPR1_PCIE_REF_CLK_EN, enable); return 0; } @@ -802,12 +864,8 @@ static void imx_pcie_clk_disable(struct imx_pcie *imx_pcie) static int imx6sx_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) { - if (assert) - regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, - IMX6SX_GPR12_PCIE_TEST_POWERDOWN); - else - regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, - IMX6SX_GPR12_PCIE_TEST_POWERDOWN); + regmap_assign_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR12, + IMX6SX_GPR12_PCIE_TEST_POWERDOWN, assert); /* Force PCIe PHY reset */ regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR5, IMX6SX_GPR5_PCIE_BTNRST_RESET, @@ -817,12 +875,8 @@ static int imx6sx_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) static int imx6qp_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) { - if (assert) - regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_TEST_PD); - else - regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_TEST_PD); + regmap_assign_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, + IMX6Q_GPR1_PCIE_TEST_PD, assert); regmap_update_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_SW_RST, assert ? IMX6Q_GPR1_PCIE_SW_RST : 0); @@ -835,12 +889,8 @@ static int imx6qp_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) static int imx6q_pcie_core_reset(struct imx_pcie *imx_pcie, bool assert) { - if (assert) - regmap_set_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_TEST_PD); - else - regmap_clear_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, - IMX6Q_GPR1_PCIE_TEST_PD); + regmap_assign_bits(imx_pcie->iomuxc_gpr, IOMUXC_GPR1, + IMX6Q_GPR1_PCIE_TEST_PD, assert); if (!assert) usleep_range(200, 500); @@ -955,9 +1005,9 @@ static int imx_pcie_wait_for_speed_change(struct imx_pcie *imx_pcie) unsigned int retries; for (retries = 0; retries < 200; retries++) { - tmp = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); + tmp = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); /* Test if the speed change finished. */ - if (!(tmp & PORT_LOGIC_SPEED_CHANGE)) + if (!(tmp & SPEED_CHANGE)) return 0; usleep_range(100, 1000); } @@ -1040,9 +1090,9 @@ static int imx_pcie_start_link(struct dw_pcie *pci) * Start Directed Speed Change so the best possible * speed both link partners support can be negotiated. */ - tmp = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - tmp |= PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, tmp); + tmp = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + tmp |= SPEED_CHANGE; + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, tmp); dw_pcie_dbi_ro_wr_dis(pci); ret = imx_pcie_wait_for_speed_change(imx_pcie); @@ -1058,8 +1108,8 @@ static int imx_pcie_start_link(struct dw_pcie *pci) err_reset_phy: dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", - dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), - dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); + dw_pcie_readl_dbi(pci, PORT_LINK_DEBUG0), + dw_pcie_readl_dbi(pci, PORT_LINK_DEBUG1)); imx_pcie_reset_phy(imx_pcie); return 0; } @@ -1389,7 +1439,7 @@ static int imx_pcie_host_init(struct dw_pcie_rp *pp) if (!pp->skip_pwrctrl_off) { ret = pci_pwrctrl_power_on_devices(dev); if (ret) { - dev_err(dev, "failed to power on pwrctrl devices\n"); + dev_err_probe(dev, ret, "failed to power on pwrctrl devices\n"); goto err_reg_disable; } } @@ -1498,18 +1548,18 @@ static void imx_pcie_host_post_init(struct dw_pcie_rp *pp) /* * ERR051586: Compliance with 8GT/s Receiver Impedance ECN * - * The default value of GEN3_RELATED_OFF[GEN3_ZRXDC_NONCOMPL] + * The default value of GEN3_CTRL[GEN3_ZRXDC_NONCOMPL] * is 1 which makes receiver non-compliant with the ZRX-DC * parameter for 2.5 GT/s when operating at 8 GT/s or higher. * It causes unnecessary timeout in L1. * - * Workaround: Program GEN3_RELATED_OFF[GEN3_ZRXDC_NONCOMPL] + * Workaround: Program GEN3_CTRL[GEN3_ZRXDC_NONCOMPL] * to 0. */ dw_pcie_dbi_ro_wr_en(pci); - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_GEN3_ZRXDC_NONCOMPL; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); dw_pcie_dbi_ro_wr_dis(pci); } @@ -1521,7 +1571,7 @@ static void imx_pcie_host_post_init(struct dw_pcie_rp *pp) } /* - * In old DWC implementations, PCIE_ATU_INHIBIT_PAYLOAD in iATU Ctrl2 + * In old DWC implementations, ATU_INHIBIT_PAYLOAD in iATU Ctrl2 * register is reserved, so the generic DWC implementation of sending the * PME_Turn_Off message using a dummy MMIO write cannot be used. */ @@ -1836,7 +1886,8 @@ static int imx_pcie_probe(struct platform_device *pdev) return dev_err_probe(dev, imx_pcie->num_clks, "failed to get clocks\n"); for (i = 0; i < imx_pcie->num_clks; i++) - if (strncmp(imx_pcie->clks[i].id, "extref", 6) == 0) + if (imx_pcie->clks[i].id && + strncmp(imx_pcie->clks[i].id, "extref", 6) == 0) imx_pcie->enable_ext_refclk = true; if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_PHYDRV)) { @@ -1951,6 +2002,10 @@ static int imx_pcie_probe(struct platform_device *pdev) platform_set_drvdata(pdev, imx_pcie); + ret = devm_add_action_or_reset(dev, imx_pcie_detach_pd_action, imx_pcie); + if (ret) + return ret; + ret = imx_pcie_attach_pd(dev); if (ret) return ret; @@ -1975,9 +2030,11 @@ static int imx_pcie_probe(struct platform_device *pdev) pm_runtime_no_callbacks(dev); ret = devm_pm_runtime_set_active_enabled(dev); if (ret < 0) - return ret; + goto err_pwrctrl_destroy; } + pci->pp.suspend_poweroff = true; + if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_SKIP_L23_READY)) pci->pp.skip_l23_ready = true; if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_KEEP_MSI_CAP)) diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c index 602516239a57..edfbc32904cd 100644 --- a/drivers/pci/controller/dwc/pci-keystone.c +++ b/drivers/pci/controller/dwc/pci-keystone.c @@ -24,6 +24,7 @@ #include <linux/of_pci.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> +#include <linux/pm_runtime.h> #include <linux/regmap.h> #include <linux/resource.h> #include <linux/signal.h> @@ -129,7 +130,6 @@ struct keystone_pcie { int num_lanes; u32 num_viewport; struct phy **phy; - struct device_link **link; struct device_node *msi_intc_np; struct irq_domain *intx_irq_domain; struct device_node *np; @@ -309,7 +309,7 @@ static int ks_pcie_msi_host_init(struct dw_pcie_rp *pp) dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, ks_pcie->app.start); pp->msi_irq_chip = &ks_pcie_msi_irq_chip; - return dw_pcie_allocate_domains(pp); + return dw_pcie_allocate_msi_domains(pp); } static void ks_pcie_handle_intx_irq(struct keystone_pcie *ks_pcie, @@ -497,8 +497,8 @@ static bool ks_pcie_link_up(struct dw_pcie *pci) { u32 val; - val = dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0); - return (val & PORT_LOGIC_LTSSM_STATE_MASK) == PORT_LOGIC_LTSSM_STATE_L0; + val = dw_pcie_readl_dbi(pci, PORT_LINK_DEBUG0); + return (val & PORT_LINK_DEBUG0_LTSSM_MASK) == DW_PCIE_LTSSM_L0; } static void ks_pcie_stop_link(struct dw_pcie *pci) @@ -633,7 +633,7 @@ static void ks_pcie_msi_irq_handler(struct irq_desc *desc) vector = offset + (pos << 3); dev_dbg(dev, "irq: bit %d, vector %d\n", pos, vector); - generic_handle_domain_irq(pp->irq_domain, vector); + generic_handle_domain_irq(pp->msi_irq_domain, vector); } chained_irq_exit(chip, desc); @@ -1131,7 +1131,6 @@ static int ks_pcie_probe(struct platform_device *pdev) enum dw_pcie_device_mode mode; struct dw_pcie *pci; struct keystone_pcie *ks_pcie; - struct device_link **link; struct gpio_desc *gpiod; struct resource *res; void __iomem *base; @@ -1202,31 +1201,17 @@ static int ks_pcie_probe(struct platform_device *pdev) if (!phy) return -ENOMEM; - link = devm_kcalloc(dev, num_lanes, sizeof(*link), GFP_KERNEL); - if (!link) - return -ENOMEM; - for (i = 0; i < num_lanes; i++) { snprintf(name, sizeof(name), "pcie-phy%d", i); phy[i] = devm_phy_optional_get(dev, name); if (IS_ERR(phy[i])) { ret = PTR_ERR(phy[i]); - goto err_link; - } - - if (!phy[i]) - continue; - - link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS); - if (!link[i]) { - ret = -EINVAL; - goto err_link; + goto err; } } ks_pcie->np = np; ks_pcie->pci = pci; - ks_pcie->link = link; ks_pcie->num_lanes = num_lanes; ks_pcie->phy = phy; @@ -1236,7 +1221,7 @@ static int ks_pcie_probe(struct platform_device *pdev) ret = PTR_ERR(gpiod); if (ret != -EPROBE_DEFER) dev_err(dev, "Failed to get reset GPIO\n"); - goto err_link; + goto err; } /* Obtain references to the PHYs */ @@ -1251,7 +1236,7 @@ static int ks_pcie_probe(struct platform_device *pdev) if (ret) { dev_err(dev, "failed to enable phy\n"); - goto err_link; + goto err; } platform_set_drvdata(pdev, ks_pcie); @@ -1338,25 +1323,18 @@ err_get_sync: pm_runtime_disable(dev); ks_pcie_disable_phy(ks_pcie); -err_link: - while (--i >= 0 && link[i]) - device_link_del(link[i]); - +err: return ret; } static void ks_pcie_remove(struct platform_device *pdev) { struct keystone_pcie *ks_pcie = platform_get_drvdata(pdev); - struct device_link **link = ks_pcie->link; - int num_lanes = ks_pcie->num_lanes; struct device *dev = &pdev->dev; pm_runtime_put(dev); pm_runtime_disable(dev); ks_pcie_disable_phy(ks_pcie); - while (num_lanes--) - device_link_del(link[num_lanes]); } static struct platform_driver ks_pcie_driver = { diff --git a/drivers/pci/controller/dwc/pci-layerscape.c b/drivers/pci/controller/dwc/pci-layerscape.c index 14d6ac4fc53f..7f37a0537159 100644 --- a/drivers/pci/controller/dwc/pci-layerscape.c +++ b/drivers/pci/controller/dwc/pci-layerscape.c @@ -419,6 +419,7 @@ static struct platform_driver ls_pcie_driver = { .of_match_table = ls_pcie_of_match, .suppress_bind_attrs = true, .pm = &ls_pcie_pm_ops, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, }; module_platform_driver(ls_pcie_driver); diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index de8ee3db4360..630ba172280c 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -163,7 +163,7 @@ static void dw_pcie_ep_clear_ib_maps(struct dw_pcie_ep *ep, u8 func_no, enum pci /* Tear down the BAR Match Mode mapping, if any. */ if (ep_func->bar_to_atu[bar]) { atu_index = ep_func->bar_to_atu[bar] - 1; - dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, atu_index); + dw_pcie_disable_atu(pci, ATU_REGION_DIR_IB, atu_index); clear_bit(atu_index, ep->ib_window_map); ep_func->bar_to_atu[bar] = 0; return; @@ -177,7 +177,7 @@ static void dw_pcie_ep_clear_ib_maps(struct dw_pcie_ep *ep, u8 func_no, enum pci if (!indexes) return; for (i = 0; i < num; i++) { - dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, indexes[i]); + dw_pcie_disable_atu(pci, ATU_REGION_DIR_IB, indexes[i]); clear_bit(indexes[i], ep->ib_window_map); } devm_kfree(dev, indexes); @@ -647,7 +647,7 @@ static void dw_pcie_ep_unmap_addr(struct pci_epc *epc, u8 func_no, u8 vfunc_no, return; ep->outbound_addr[atu_index] = 0; - dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, atu_index); + dw_pcie_disable_atu(pci, ATU_REGION_DIR_OB, atu_index); clear_bit(atu_index, ep->ob_window_map); } @@ -1077,10 +1077,10 @@ int dw_pcie_ep_raise_msix_irq_doorbell(struct dw_pcie_ep *ep, u8 func_no, if (!ep_func || !ep_func->msix_cap) return -EINVAL; - msg_data = (func_no << PCIE_MSIX_DOORBELL_PF_SHIFT) | + msg_data = FIELD_PREP(MSIX_DOORBELL_PF_MASK, func_no) | (interrupt_num - 1); - dw_pcie_writel_dbi(pci, PCIE_MSIX_DOORBELL, msg_data); + dw_pcie_writel_dbi(pci, MSIX_DOORBELL, msg_data); return 0; } @@ -1194,6 +1194,9 @@ void dw_pcie_ep_deinit(struct dw_pcie_ep *ep) epc->mem->window.page_size); pci_epc_mem_exit(epc); + + if (ep->ops->post_deinit) + ep->ops->post_deinit(ep); } EXPORT_SYMBOL_GPL(dw_pcie_ep_deinit); @@ -1553,7 +1556,7 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) ep->page_size); if (ret < 0) { dev_err(dev, "Failed to initialize address space\n"); - return ret; + goto err_deinit; } ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys, @@ -1568,6 +1571,9 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) err_exit_epc_mem: pci_epc_mem_exit(epc); +err_deinit: + if (ep->ops->post_deinit) + ep->ops->post_deinit(ep); return ret; } diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index f5a38e6fd8d7..869a39107a81 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -82,7 +82,7 @@ void dw_handle_msi_irq(struct dw_pcie_rp *pp) continue; for_each_set_bit(pos, &status, MAX_MSI_IRQS_PER_CTRL) - generic_handle_demux_domain_irq(pp->irq_domain, irq_off + pos); + generic_handle_demux_domain_irq(pp->msi_irq_domain, irq_off + pos); } } @@ -207,7 +207,7 @@ static const struct irq_domain_ops dw_pcie_msi_domain_ops = { .free = dw_pcie_irq_domain_free, }; -int dw_pcie_allocate_domains(struct dw_pcie_rp *pp) +int dw_pcie_allocate_msi_domains(struct dw_pcie_rp *pp) { struct dw_pcie *pci = to_dw_pcie_from_pp(pp); struct irq_domain_info info = { @@ -217,15 +217,15 @@ int dw_pcie_allocate_domains(struct dw_pcie_rp *pp) .host_data = pp, }; - pp->irq_domain = msi_create_parent_irq_domain(&info, &dw_pcie_msi_parent_ops); - if (!pp->irq_domain) { + pp->msi_irq_domain = msi_create_parent_irq_domain(&info, &dw_pcie_msi_parent_ops); + if (!pp->msi_irq_domain) { dev_err(pci->dev, "Failed to create IRQ domain\n"); return -ENOMEM; } return 0; } -EXPORT_SYMBOL_GPL(dw_pcie_allocate_domains); +EXPORT_SYMBOL_GPL(dw_pcie_allocate_msi_domains); void dw_pcie_free_msi(struct dw_pcie_rp *pp) { @@ -236,7 +236,7 @@ void dw_pcie_free_msi(struct dw_pcie_rp *pp) irq_set_chained_handler_and_data(pp->msi_irq[ctrl], NULL, NULL); } - irq_domain_remove(pp->irq_domain); + irq_domain_remove(pp->msi_irq_domain); } EXPORT_SYMBOL_GPL(dw_pcie_free_msi); @@ -342,7 +342,7 @@ int dw_pcie_msi_host_init(struct dw_pcie_rp *pp) pp->msi_irq_chip = &dw_pci_msi_bottom_irq_chip; - ret = dw_pcie_allocate_domains(pp); + ret = dw_pcie_allocate_msi_domains(pp); if (ret) return ret; @@ -440,7 +440,7 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp) atu.parent_bus_addr = pp->cfg0_base + SZ_1M; /* 1MiB is to cover 1 (bus) * 32 (devices) * 8 (functions) */ atu.size = SZ_1M; - atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE; + atu.ctrl2 = ATU_CFG_SHIFT_MODE_ENABLE; ret = dw_pcie_prog_outbound_atu(pci, &atu); if (ret) return ret; @@ -455,7 +455,7 @@ static int dw_pcie_config_ecam_iatu(struct dw_pcie_rp *pp) atu.type = PCIE_TLP_TYPE_CFG1_RDWR; atu.parent_bus_addr = pp->cfg0_base + SZ_2M; atu.size = (SZ_1M * bus_range_max) - SZ_2M; - atu.ctrl2 = PCIE_ATU_CFG_SHIFT_MODE_ENABLE; + atu.ctrl2 = ATU_CFG_SHIFT_MODE_ENABLE; return dw_pcie_prog_outbound_atu(pci, &atu); } @@ -747,8 +747,8 @@ static void __iomem *dw_pcie_other_conf_map_bus(struct pci_bus *bus, if (!dw_pcie_link_up(pci)) return NULL; - busdev = PCIE_ATU_BUS(bus->number) | PCIE_ATU_DEV(PCI_SLOT(devfn)) | - PCIE_ATU_FUNC(PCI_FUNC(devfn)); + busdev = ATU_BUS(bus->number) | ATU_DEV(PCI_SLOT(devfn)) | + ATU_FUNC(PCI_FUNC(devfn)); if (pci_is_root_bus(bus->parent)) type = PCIE_TLP_TYPE_CFG0_RDWR; @@ -884,10 +884,10 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) * the MEM/IO (dma-)ranges setups. */ for (i = 0; i < pci->num_ob_windows; i++) - dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i); + dw_pcie_disable_atu(pci, ATU_REGION_DIR_OB, i); for (i = 0; i < pci->num_ib_windows; i++) - dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i); + dw_pcie_disable_atu(pci, ATU_REGION_DIR_IB, i); /* * NOTE: For outbound address translation, outbound iATU at index 0 is @@ -918,6 +918,20 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) atu.parent_bus_addr = entry->res->start - pci->parent_bus_offset; atu.pci_addr = entry->res->start - entry->offset; + if (pp->bypass_ob_mem_iatu) { + /* + * The no-match pass-through path preserves the correct + * PCI address only for identity-mapped MEM ranges. + */ + if (atu.parent_bus_addr != atu.pci_addr) { + dev_err(pci->dev, + "Cannot bypass outbound iATU for non-identity MEM range %pr\n", + entry->res); + return -EINVAL; + } + continue; + } + /* Adjust iATU size if MSG TLP region was allocated before */ if (pp->msg_res && pp->msg_res->parent == entry->res) res_size = resource_size(entry->res) - @@ -964,8 +978,7 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp) ret = dw_pcie_prog_outbound_atu(pci, &atu); if (ret) { - dev_err(pci->dev, "Failed to set IO range %pr\n", - entry->res); + dev_err(pci->dev, "Failed to set IO range\n"); return ret; } ob_iatu_index++; @@ -1165,9 +1178,9 @@ int dw_pcie_setup_rc(struct dw_pcie_rp *pp) /* Program correct class for RC */ dw_pcie_writew_dbi(pci, PCI_CLASS_DEVICE, PCI_CLASS_BRIDGE_PCI); - val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - val |= PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + val = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + val |= SPEED_CHANGE; + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, val); dw_pcie_dbi_ro_wr_dis(pci); @@ -1224,14 +1237,21 @@ static int dw_pcie_pme_turn_off(struct dw_pcie *pci) int dw_pcie_suspend_noirq(struct dw_pcie *pci) { - bool pme_capable = false; + bool d3cold, pme_capable = false; int ret = 0; u32 val; if (!dw_pcie_link_up(pci)) goto stop_link; - if (!pci_host_common_d3cold_possible(pci->pp.bridge, &pme_capable)) + d3cold = pci_host_common_d3cold_possible(pci->pp.bridge, &pme_capable); + + /* + * Enter L2 if D3cold is possible, or if the platform loses RC power + * during suspend (in which case L2 entry is required to notify + * endpoints before power loss, regardless of D3cold support). + */ + if (!d3cold && !pci->pp.suspend_poweroff) return 0; if (pci->pp.ops->pme_turn_off) { diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index 593388f29bdd..074b007dc296 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -436,9 +436,9 @@ static inline void __iomem *dw_pcie_select_atu(struct dw_pcie *pci, u32 dir, u32 index) { if (dw_pcie_cap_is(pci, IATU_UNROLL)) - return pci->atu_base + PCIE_ATU_UNROLL_BASE(dir, index); + return pci->atu_base + ATU_UNROLL_BASE(dir, index); - dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, dir | index); + dw_pcie_writel_dbi(pci, IATU_VIEWPORT, dir | index); return pci->atu_base; } @@ -480,13 +480,13 @@ static void dw_pcie_writel_atu(struct dw_pcie *pci, u32 dir, u32 index, static inline u32 dw_pcie_readl_atu_ob(struct dw_pcie *pci, u32 index, u32 reg) { - return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg); + return dw_pcie_readl_atu(pci, ATU_REGION_DIR_OB, index, reg); } static inline void dw_pcie_writel_atu_ob(struct dw_pcie *pci, u32 index, u32 reg, u32 val) { - dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_OB, index, reg, val); + dw_pcie_writel_atu(pci, ATU_REGION_DIR_OB, index, reg, val); } static inline u32 dw_pcie_enable_ecrc(u32 val) @@ -527,7 +527,7 @@ static inline u32 dw_pcie_enable_ecrc(u32 val) * handle the scenario is to program TD bit by default. */ - return val | PCIE_ATU_TD; + return val | ATU_TD; } int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, @@ -548,44 +548,46 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, return -EINVAL; } - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_LOWER_BASE, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_LOWER_BASE, lower_32_bits(parent_bus_addr)); - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_UPPER_BASE, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_UPPER_BASE, upper_32_bits(parent_bus_addr)); - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_LIMIT, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_LIMIT, lower_32_bits(limit_addr)); if (dw_pcie_ver_is_ge(pci, 460A)) - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_UPPER_LIMIT, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_UPPER_LIMIT, upper_32_bits(limit_addr)); - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_LOWER_TARGET, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_LOWER_TARGET, lower_32_bits(atu->pci_addr)); - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_UPPER_TARGET, + dw_pcie_writel_atu_ob(pci, atu->index, ATU_UPPER_TARGET, upper_32_bits(atu->pci_addr)); - val = atu->type | atu->routing | PCIE_ATU_FUNC_NUM(atu->func_no); + val = atu->type | atu->routing | ATU_FUNC_NUM(atu->func_no); if (upper_32_bits(limit_addr) > upper_32_bits(parent_bus_addr) && dw_pcie_ver_is_ge(pci, 460A)) - val |= PCIE_ATU_INCREASE_REGION_SIZE; + val |= ATU_INCREASE_REGION_SIZE; if (!dw_pcie_ver_is_ge(pci, 510A)) val = dw_pcie_enable_ecrc(val); - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL1, val); + dw_pcie_writel_atu_ob(pci, atu->index, ATU_REGION_CTRL1, val); - val = PCIE_ATU_ENABLE | atu->ctrl2; + val = ATU_ENABLE | atu->ctrl2; if (atu->type == PCIE_TLP_TYPE_MSG) { /* The data-less messages only for now */ - val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code; + val |= ATU_INHIBIT_PAYLOAD | atu->code; } - dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val); + if (dw_pcie_ver_is_ge(pci, 460A)) + val |= ATU_DMA_BYPASS; + dw_pcie_writel_atu_ob(pci, atu->index, ATU_REGION_CTRL2, val); /* * Make sure ATU enable takes effect before any subsequent config * and I/O accesses. */ for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) { - val = dw_pcie_readl_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2); - if (val & PCIE_ATU_ENABLE) + val = dw_pcie_readl_atu_ob(pci, atu->index, ATU_REGION_CTRL2); + if (val & ATU_ENABLE) return 0; mdelay(LINK_WAIT_IATU); @@ -598,13 +600,13 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, static inline u32 dw_pcie_readl_atu_ib(struct dw_pcie *pci, u32 index, u32 reg) { - return dw_pcie_readl_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg); + return dw_pcie_readl_atu(pci, ATU_REGION_DIR_IB, index, reg); } static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg, u32 val) { - dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val); + dw_pcie_writel_atu(pci, ATU_REGION_DIR_IB, index, reg, val); } int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type, @@ -622,36 +624,36 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type, return -EINVAL; } - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_BASE, + dw_pcie_writel_atu_ib(pci, index, ATU_LOWER_BASE, lower_32_bits(pci_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_BASE, + dw_pcie_writel_atu_ib(pci, index, ATU_UPPER_BASE, upper_32_bits(pci_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LIMIT, + dw_pcie_writel_atu_ib(pci, index, ATU_LIMIT, lower_32_bits(limit_addr)); if (dw_pcie_ver_is_ge(pci, 460A)) - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_LIMIT, + dw_pcie_writel_atu_ib(pci, index, ATU_UPPER_LIMIT, upper_32_bits(limit_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET, + dw_pcie_writel_atu_ib(pci, index, ATU_LOWER_TARGET, lower_32_bits(parent_bus_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET, + dw_pcie_writel_atu_ib(pci, index, ATU_UPPER_TARGET, upper_32_bits(parent_bus_addr)); val = type; if (upper_32_bits(limit_addr) > upper_32_bits(pci_addr) && dw_pcie_ver_is_ge(pci, 460A)) - val |= PCIE_ATU_INCREASE_REGION_SIZE; - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, val); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE); + val |= ATU_INCREASE_REGION_SIZE; + dw_pcie_writel_atu_ib(pci, index, ATU_REGION_CTRL1, val); + dw_pcie_writel_atu_ib(pci, index, ATU_REGION_CTRL2, ATU_ENABLE); /* * Make sure ATU enable takes effect before any subsequent config * and I/O accesses. */ for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) { - val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2); - if (val & PCIE_ATU_ENABLE) + val = dw_pcie_readl_atu_ib(pci, index, ATU_REGION_CTRL2); + if (val & ATU_ENABLE) return 0; mdelay(LINK_WAIT_IATU); @@ -671,24 +673,24 @@ int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index, !IS_ALIGNED(parent_bus_addr, size)) return -EINVAL; - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET, + dw_pcie_writel_atu_ib(pci, index, ATU_LOWER_TARGET, lower_32_bits(parent_bus_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET, + dw_pcie_writel_atu_ib(pci, index, ATU_UPPER_TARGET, upper_32_bits(parent_bus_addr)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, type | - PCIE_ATU_FUNC_NUM(func_no)); - dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, - PCIE_ATU_ENABLE | PCIE_ATU_FUNC_NUM_MATCH_EN | - PCIE_ATU_BAR_MODE_ENABLE | (bar << 8)); + dw_pcie_writel_atu_ib(pci, index, ATU_REGION_CTRL1, type | + ATU_FUNC_NUM(func_no)); + dw_pcie_writel_atu_ib(pci, index, ATU_REGION_CTRL2, + ATU_ENABLE | ATU_FUNC_NUM_MATCH_EN | + ATU_BAR_MODE_ENABLE | (bar << 8)); /* * Make sure ATU enable takes effect before any subsequent config * and I/O accesses. */ for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) { - val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2); - if (val & PCIE_ATU_ENABLE) + val = dw_pcie_readl_atu_ib(pci, index, ATU_REGION_CTRL2); + if (val & ATU_ENABLE) return 0; mdelay(LINK_WAIT_IATU); @@ -701,7 +703,7 @@ int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index, void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index) { - dw_pcie_writel_atu(pci, dir, index, PCIE_ATU_REGION_CTRL2, 0); + dw_pcie_writel_atu(pci, dir, index, ATU_REGION_CTRL2, 0); } const char *dw_pcie_ltssm_status_string(enum dw_pcie_ltssm ltssm) @@ -824,9 +826,9 @@ bool dw_pcie_link_up(struct dw_pcie *pci) if (pci->ops && pci->ops->link_up) return pci->ops->link_up(pci); - val = dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1); - return ((val & PCIE_PORT_DEBUG1_LINK_UP) && - (!(val & PCIE_PORT_DEBUG1_LINK_IN_TRAINING))); + val = dw_pcie_readl_dbi(pci, PORT_LINK_DEBUG1); + return ((val & PORT_LINK_DEBUG1_LINK_UP) && + (!(val & PORT_LINK_DEBUG1_LINK_IN_TRAINING))); } EXPORT_SYMBOL_GPL(dw_pcie_link_up); @@ -834,9 +836,9 @@ void dw_pcie_upconfig_setup(struct dw_pcie *pci) { u32 val; - val = dw_pcie_readl_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL); - val |= PORT_MLTI_UPCFG_SUPPORT; - dw_pcie_writel_dbi(pci, PCIE_PORT_MULTI_LANE_CTRL, val); + val = dw_pcie_readl_dbi(pci, MULTI_LANE_CTRL); + val |= MULTI_LANE_CTRL_UPCFG_SUPPORT; + dw_pcie_writel_dbi(pci, MULTI_LANE_CTRL, val); } EXPORT_SYMBOL_GPL(dw_pcie_upconfig_setup); @@ -894,36 +896,36 @@ static void dw_pcie_link_set_max_link_width(struct dw_pcie *pci, u32 num_lanes) return; /* Set the number of lanes */ - plc = dw_pcie_readl_dbi(pci, PCIE_PORT_LINK_CONTROL); - plc &= ~PORT_LINK_FAST_LINK_MODE; - plc &= ~PORT_LINK_MODE_MASK; + plc = dw_pcie_readl_dbi(pci, PORT_LINK_CTRL); + plc &= ~PORT_LINK_CTRL_FLM; + plc &= ~PORT_LINK_CTRL_LC_MASK; /* Set link width speed control register */ - lwsc = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - lwsc &= ~PORT_LOGIC_LINK_WIDTH_MASK; - lwsc |= PORT_LOGIC_LINK_WIDTH_1_LANES; + lwsc = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + lwsc &= ~LINK_WIDTH_MASK; + lwsc |= LINK_WIDTH_1_LANES; switch (num_lanes) { case 1: - plc |= PORT_LINK_MODE_1_LANES; + plc |= PORT_LINK_CTRL_LC_1_LANES; break; case 2: - plc |= PORT_LINK_MODE_2_LANES; + plc |= PORT_LINK_CTRL_LC_2_LANES; break; case 4: - plc |= PORT_LINK_MODE_4_LANES; + plc |= PORT_LINK_CTRL_LC_4_LANES; break; case 8: - plc |= PORT_LINK_MODE_8_LANES; + plc |= PORT_LINK_CTRL_LC_8_LANES; break; case 16: - plc |= PORT_LINK_MODE_16_LANES; + plc |= PORT_LINK_CTRL_LC_16_LANES; break; default: dev_err(pci->dev, "num-lanes %u: invalid value\n", num_lanes); return; } - dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, plc); - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, lwsc); + dw_pcie_writel_dbi(pci, PORT_LINK_CTRL, plc); + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, lwsc); cap = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); lnkcap = dw_pcie_readl_dbi(pci, cap + PCI_EXP_LNKCAP); @@ -937,48 +939,48 @@ void dw_pcie_iatu_detect(struct dw_pcie *pci) u32 val, min, dir; u64 max; - val = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT); + val = dw_pcie_readl_dbi(pci, IATU_VIEWPORT); if (val == 0xFFFFFFFF) { dw_pcie_cap_set(pci, IATU_UNROLL); max_region = min((int)pci->atu_size / 512, 256); } else { - pci->atu_base = pci->dbi_base + PCIE_ATU_VIEWPORT_BASE; - pci->atu_size = PCIE_ATU_VIEWPORT_SIZE; + pci->atu_base = pci->dbi_base + ATU_VIEWPORT_BASE; + pci->atu_size = ATU_VIEWPORT_SIZE; - dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT, 0xFF); - max_region = dw_pcie_readl_dbi(pci, PCIE_ATU_VIEWPORT) + 1; + dw_pcie_writel_dbi(pci, IATU_VIEWPORT, 0xFF); + max_region = dw_pcie_readl_dbi(pci, IATU_VIEWPORT) + 1; } for (ob = 0; ob < max_region; ob++) { - dw_pcie_writel_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET, 0x11110000); - val = dw_pcie_readl_atu_ob(pci, ob, PCIE_ATU_LOWER_TARGET); + dw_pcie_writel_atu_ob(pci, ob, ATU_LOWER_TARGET, 0x11110000); + val = dw_pcie_readl_atu_ob(pci, ob, ATU_LOWER_TARGET); if (val != 0x11110000) break; } for (ib = 0; ib < max_region; ib++) { - dw_pcie_writel_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET, 0x11110000); - val = dw_pcie_readl_atu_ib(pci, ib, PCIE_ATU_LOWER_TARGET); + dw_pcie_writel_atu_ib(pci, ib, ATU_LOWER_TARGET, 0x11110000); + val = dw_pcie_readl_atu_ib(pci, ib, ATU_LOWER_TARGET); if (val != 0x11110000) break; } if (ob) { - dir = PCIE_ATU_REGION_DIR_OB; + dir = ATU_REGION_DIR_OB; } else if (ib) { - dir = PCIE_ATU_REGION_DIR_IB; + dir = ATU_REGION_DIR_IB; } else { dev_err(pci->dev, "No iATU regions found\n"); return; } - dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_LIMIT, 0x0); - min = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_LIMIT); + dw_pcie_writel_atu(pci, dir, 0, ATU_LIMIT, 0x0); + min = dw_pcie_readl_atu(pci, dir, 0, ATU_LIMIT); if (dw_pcie_ver_is_ge(pci, 460A)) { - dw_pcie_writel_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT, 0xFFFFFFFF); - max = dw_pcie_readl_atu(pci, dir, 0, PCIE_ATU_UPPER_LIMIT); + dw_pcie_writel_atu(pci, dir, 0, ATU_UPPER_LIMIT, 0xFFFFFFFF); + max = dw_pcie_readl_atu(pci, dir, 0, ATU_UPPER_LIMIT); } else { max = 0; } @@ -1273,19 +1275,19 @@ void dw_pcie_setup(struct dw_pcie *pci) /* Configure Gen1 N_FTS */ if (pci->n_fts[0]) { - val = dw_pcie_readl_dbi(pci, PCIE_PORT_AFR); - val &= ~(PORT_AFR_N_FTS_MASK | PORT_AFR_CC_N_FTS_MASK); - val |= PORT_AFR_N_FTS(pci->n_fts[0]); - val |= PORT_AFR_CC_N_FTS(pci->n_fts[0]); - dw_pcie_writel_dbi(pci, PCIE_PORT_AFR, val); + val = dw_pcie_readl_dbi(pci, PORT_AFAC); + val &= ~(PORT_AFAC_N_FTS_MASK | PORT_AFAC_CC_N_FTS_MASK); + val |= PORT_AFAC_N_FTS(pci->n_fts[0]); + val |= PORT_AFAC_CC_N_FTS(pci->n_fts[0]); + dw_pcie_writel_dbi(pci, PORT_AFAC, val); } /* Configure Gen2+ N_FTS */ if (pci->n_fts[1]) { - val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - val &= ~PORT_LOGIC_N_FTS_MASK; + val = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + val &= ~N_FTS_MASK; val |= pci->n_fts[1]; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, val); } if (dw_pcie_cap_is(pci, CDM_CHECK)) { @@ -1295,10 +1297,10 @@ void dw_pcie_setup(struct dw_pcie *pci) dw_pcie_writel_dbi(pci, PCIE_PL_CHK_REG_CONTROL_STATUS, val); } - val = dw_pcie_readl_dbi(pci, PCIE_PORT_LINK_CONTROL); - val &= ~PORT_LINK_FAST_LINK_MODE; - val |= PORT_LINK_DLL_LINK_EN; - dw_pcie_writel_dbi(pci, PCIE_PORT_LINK_CONTROL, val); + val = dw_pcie_readl_dbi(pci, PORT_LINK_CTRL); + val &= ~PORT_LINK_CTRL_FLM; + val |= PORT_LINK_CTRL_DLE; + dw_pcie_writel_dbi(pci, PORT_LINK_CTRL, val); dw_pcie_link_set_max_link_width(pci, pci->num_lanes); } diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 0735ae940924..6d788aa00011 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -71,38 +71,40 @@ #define LINK_WAIT_IATU 9 /* Synopsys-specific PCIe configuration registers */ -#define PCIE_PORT_FORCE 0x708 +#define PORT_FORCE_LINK 0x708 /* Port Force Link */ /* Bit[7:0] LINK_NUM: Link Number. Not used for endpoint */ -#define PORT_LINK_NUM_MASK GENMASK(7, 0) -#define PORT_FORCE_DO_DESKEW_FOR_SRIS BIT(23) - -#define PCIE_PORT_AFR 0x70C -#define PORT_AFR_N_FTS_MASK GENMASK(15, 8) -#define PORT_AFR_N_FTS(n) FIELD_PREP(PORT_AFR_N_FTS_MASK, n) -#define PORT_AFR_CC_N_FTS_MASK GENMASK(23, 16) -#define PORT_AFR_CC_N_FTS(n) FIELD_PREP(PORT_AFR_CC_N_FTS_MASK, n) -#define PORT_AFR_ENTER_ASPM BIT(30) -#define PORT_AFR_L0S_ENTRANCE_LAT_SHIFT 24 -#define PORT_AFR_L0S_ENTRANCE_LAT_MASK GENMASK(26, 24) -#define PORT_AFR_L1_ENTRANCE_LAT_SHIFT 27 -#define PORT_AFR_L1_ENTRANCE_LAT_MASK GENMASK(29, 27) - -#define PCIE_PORT_LINK_CONTROL 0x710 -#define PORT_LINK_DLL_LINK_EN BIT(5) -#define PORT_LINK_FAST_LINK_MODE BIT(7) -#define PORT_LINK_MODE_MASK GENMASK(21, 16) -#define PORT_LINK_MODE(n) FIELD_PREP(PORT_LINK_MODE_MASK, n) -#define PORT_LINK_MODE_1_LANES PORT_LINK_MODE(0x1) -#define PORT_LINK_MODE_2_LANES PORT_LINK_MODE(0x3) -#define PORT_LINK_MODE_4_LANES PORT_LINK_MODE(0x7) -#define PORT_LINK_MODE_8_LANES PORT_LINK_MODE(0xf) -#define PORT_LINK_MODE_16_LANES PORT_LINK_MODE(0x1f) - -#define PCIE_PORT_LANE_SKEW 0x714 -#define PORT_LANE_SKEW_INSERT_MASK GENMASK(23, 0) +#define PORT_FORCE_LINK_NUM_MASK GENMASK(7, 0) /* Link Number. Not used for endpoint */ +#define PORT_FORCE_LINK_DDFS BIT(23) /* do deskew for sris */ + +#define PORT_AFAC 0x70C /* Ack Frequency and L0-L1 ASPM Control */ +/* Ack the number of Fast Training Sequence mask */ +#define PORT_AFAC_N_FTS_MASK GENMASK(15, 8) +/* Ack the number of Fast Training Sequence */ +#define PORT_AFAC_N_FTS(n) FIELD_PREP(PORT_AFAC_N_FTS_MASK, n) +#define PORT_AFAC_CC_N_FTS_MASK GENMASK(23, 16) /* Common Clock N_FTS mask */ +/* Common Clock N_FTS */ +#define PORT_AFAC_CC_N_FTS(n) FIELD_PREP(PORT_AFAC_CC_N_FTS_MASK, n) +#define PORT_AFAC_ENTER_ASPM BIT(30) /* ASPM L1 Entry Control */ +#define PORT_AFAC_L0S_ELM GENMASK(26, 24) /* L0s Entrance Latency mask */ +#define PORT_AFAC_L1_ELM GENMASK(29, 27) /* L1 Entrance Latency mask */ + +#define PORT_LINK_CTRL 0x710 /* Port Link Control Register */ +#define PORT_LINK_CTRL_DLE BIT(5) /* DLL Link Enable */ +#define PORT_LINK_CTRL_FLM BIT(7) /* Fast Link Mode */ +#define PORT_LINK_CTRL_LC_MASK GENMASK(21, 16) /* Link Mode Enable (LINK_CAPABLE) mask */ +/* Link Mode Enable (LINK_CAPABLE) */ +#define PORT_LINK_CTRL_LC(n) FIELD_PREP(PORT_LINK_CTRL_LC_MASK, n) +#define PORT_LINK_CTRL_LC_1_LANES PORT_LINK_CTRL_LC(0x1) /* Link Mode 1 lane */ +#define PORT_LINK_CTRL_LC_2_LANES PORT_LINK_CTRL_LC(0x3) /* Link Mode 2 lane */ +#define PORT_LINK_CTRL_LC_4_LANES PORT_LINK_CTRL_LC(0x7) /* Link Mode 4 lane */ +#define PORT_LINK_CTRL_LC_8_LANES PORT_LINK_CTRL_LC(0xf) /* Link Mode 8 lane */ +#define PORT_LINK_CTRL_LC_16_LANES PORT_LINK_CTRL_LC(0x1f) /* Link Mode 16 lane */ + +#define PORT_LANE_SKEW 0x714 /* Lane Skew */ +#define PORT_LANE_SKEW_INSERT_MASK GENMASK(23, 0) /* - * PCIE_TIMER_CTRL_MAX_FUNC_NUM: Timer Control and Max Function Number + * TIMER_CTRL_MAX_FUNC_NUM: Timer Control and Max Function Number * Register. * * This register holds the ack frequency, latency, replay, fast link @@ -116,68 +118,79 @@ * 0x2 (SF_64): Scaling Factor is 64 (1ms is 16us) * 0x3 (SF_16): Scaling Factor is 16 (1ms is 64us) */ -#define PCIE_TIMER_CTRL_MAX_FUNC_NUM 0x718 -#define PORT_FLT_SF_MASK GENMASK(30, 29) -#define PORT_FLT_SF_VAL_1024 0x0 -#define PORT_FLT_SF_VAL_256 0x1 -#define PORT_FLT_SF_VAL_64 0x2 -#define PORT_FLT_SF_VAL_16 0x3 - -#define PCIE_PORT_DEBUG0 0x728 -#define PORT_LOGIC_LTSSM_STATE_MASK 0x3f -#define PORT_LOGIC_LTSSM_STATE_L0 0x11 -#define PCIE_PORT_DEBUG1 0x72C -#define PCIE_PORT_DEBUG1_LINK_UP BIT(4) -#define PCIE_PORT_DEBUG1_LINK_IN_TRAINING BIT(29) - -#define PCIE_LINK_WIDTH_SPEED_CONTROL 0x80C -#define PORT_LOGIC_N_FTS_MASK GENMASK(7, 0) -#define PORT_LOGIC_SPEED_CHANGE BIT(17) -#define PORT_LOGIC_LINK_WIDTH_MASK GENMASK(12, 8) -#define PORT_LOGIC_LINK_WIDTH(n) FIELD_PREP(PORT_LOGIC_LINK_WIDTH_MASK, n) -#define PORT_LOGIC_LINK_WIDTH_1_LANES PORT_LOGIC_LINK_WIDTH(0x1) -#define PORT_LOGIC_LINK_WIDTH_2_LANES PORT_LOGIC_LINK_WIDTH(0x2) -#define PORT_LOGIC_LINK_WIDTH_4_LANES PORT_LOGIC_LINK_WIDTH(0x4) -#define PORT_LOGIC_LINK_WIDTH_8_LANES PORT_LOGIC_LINK_WIDTH(0x8) - +#define TIMER_CTRL_MAX_FUNC_NUM 0x718 +#define PORT_FLT_SF_MASK GENMASK(30, 29) +#define PORT_FLT_SF_VAL_1024 0x0 +#define PORT_FLT_SF_VAL_256 0x1 +#define PORT_FLT_SF_VAL_64 0x2 +#define PORT_FLT_SF_VAL_16 0x3 + +#define PORT_LINK_DEBUG0 0x728 /* Port Link Debug Register 0 */ +#define PORT_LINK_DEBUG0_LTSSM_MASK 0x3f /* LTSSM current state mask */ +#define PORT_LINK_DEBUG1 0x72C /* Port Link Debug Register 1 */ +/* LTSSM reports PHY link up or LTSSM is in Loopback */ +#define PORT_LINK_DEBUG1_LINK_UP BIT(4) +#define PORT_LINK_DEBUG1_LINK_IN_TRAINING BIT(29) /* LTSSM performing link training */ + +#define LINK_WIDTH_SPEED_CTRL 0x80C /* Link Width and Speed Change Control */ +#define N_FTS_MASK GENMASK(7, 0) +#define SPEED_CHANGE BIT(17) +#define LINK_WIDTH_MASK GENMASK(12, 8) +#define LINK_WIDTH(n) FIELD_PREP(LINK_WIDTH_MASK, n) +#define LINK_WIDTH_1_LANES LINK_WIDTH(0x1) +#define LINK_WIDTH_2_LANES LINK_WIDTH(0x2) +#define LINK_WIDTH_4_LANES LINK_WIDTH(0x4) +#define LINK_WIDTH_8_LANES LINK_WIDTH(0x8) + +/* Integrated MSI Reception Module Lower Address */ #define PCIE_MSI_ADDR_LO 0x820 +/* Integrated MSI Reception Module Upper Address */ #define PCIE_MSI_ADDR_HI 0x824 +/* Integrated MSI Reception Module Interrupt0 Enable */ #define PCIE_MSI_INTR0_ENABLE 0x828 +/* Integrated MSI Reception Module Interrupt0 Mask */ #define PCIE_MSI_INTR0_MASK 0x82C +/* Integrated MSI Reception Module Interrupt0 Status */ #define PCIE_MSI_INTR0_STATUS 0x830 -#define GEN3_RELATED_OFF 0x890 -#define GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL BIT(0) -#define GEN3_RELATED_OFF_EQ_PHASE_2_3 BIT(9) -#define GEN3_RELATED_OFF_RXEQ_RGRDLESS_RXTS BIT(13) -#define GEN3_RELATED_OFF_GEN3_EQ_DISABLE BIT(16) -#define GEN3_RELATED_OFF_RATE_SHADOW_SEL_SHIFT 24 -#define GEN3_RELATED_OFF_RATE_SHADOW_SEL_MASK GENMASK(25, 24) - -#define GEN3_EQ_CONTROL_OFF 0x8A8 -#define GEN3_EQ_CONTROL_OFF_FB_MODE GENMASK(3, 0) -#define GEN3_EQ_CONTROL_OFF_PHASE23_EXIT_MODE BIT(4) -#define GEN3_EQ_CONTROL_OFF_PSET_REQ_VEC GENMASK(23, 8) -#define GEN3_EQ_CONTROL_OFF_FOM_INC_INITIAL_EVAL BIT(24) - -#define GEN3_EQ_FB_MODE_DIR_CHANGE_OFF 0x8AC -#define GEN3_EQ_FMDC_T_MIN_PHASE23 GENMASK(4, 0) -#define GEN3_EQ_FMDC_N_EVALS GENMASK(9, 5) -#define GEN3_EQ_FMDC_MAX_PRE_CURSOR_DELTA GENMASK(13, 10) -#define GEN3_EQ_FMDC_MAX_POST_CURSOR_DELTA GENMASK(17, 14) - -#define COHERENCY_CONTROL_1_OFF 0x8E0 -#define CFG_MEMTYPE_BOUNDARY_LOW_ADDR_MASK GENMASK(31, 2) -#define CFG_MEMTYPE_VALUE BIT(0) - -#define COHERENCY_CONTROL_2_OFF 0x8E4 -#define COHERENCY_CONTROL_3_OFF 0x8E8 - -#define PCIE_PORT_MULTI_LANE_CTRL 0x8C0 -#define PORT_MLTI_UPCFG_SUPPORT BIT(7) - -#define PCIE_VERSION_NUMBER 0x8F8 -#define PCIE_VERSION_TYPE 0x8FC +#define GEN3_CTRL 0x890 /* Gen3 Control Register */ +/* Gen3 Receiver Impedance ZRX-DC Not Compliant */ +#define GEN3_CTRL_GEN3_ZRXDC_NONCOMPL BIT(0) +#define GEN3_CTRL_EQ_PHASE_2_3 BIT(9) /* Equalization Phase 2 and Phase 3 */ +#define GEN3_CTRL_RXEQ_RGRDLESS_RXTS BIT(13) +#define GEN3_CTRL_GEN3_EQ_DISABLE BIT(16) /* Equalization Disable */ +#define GEN3_CTRL_RATE_SHADOW_SEL_MASK GENMASK(25, 24) /* Rate Shadow Select mask */ + +#define GEN3_EQ_CTRL 0x8A8 /* Gen3 EQ Control */ +#define GEN3_EQ_CTRL_FB_MODE GENMASK(3, 0) /* Feedback Mode */ +#define GEN3_EQ_CTRL_PHASE23_EXIT_MODE BIT(4) /* Behavior After 24 ms Timeout */ +#define GEN3_EQ_CTRL_PSET_REQ_VEC GENMASK(23, 8) /* Preset Request Vector */ +/* Include or not the FOM feedback from the initial preset */ +#define GEN3_EQ_CTRL_FOM_INC_INITIAL_EVAL BIT(24) + +/* Gen3 EQ Direction Change Feedback Mode Control */ +#define GEN3_EQ_FB_MODE_DIR_CHANGE 0x8AC +#define GEN3_EQ_FMDC_T_MIN_PHASE23 GENMASK(4, 0) +#define GEN3_EQ_FMDC_N_EVALS GENMASK(9, 5) +#define GEN3_EQ_FMDC_MAX_PRE_CURSOR_DELTA GENMASK(13, 10) +#define GEN3_EQ_FMDC_MAX_POST_CURSOR_DELTA GENMASK(17, 14) + +#define MISC_CONTROL_1 0x8BC /* DBI Read-Only Write Enable */ +#define DBI_RO_WR_EN BIT(0) /* Write to RO Registers Using DBI */ + +#define COHERENCY_CTRL1 0x8E0 /* Cache Coherency Control Register 1 */ +/* Boundary Lower Address For Memory Type */ +#define CFG_MEMTYPE_BOUNDARY_LOW_ADDR_MASK GENMASK(31, 2) +#define CFG_MEMTYPE_VALUE BIT(0) /* Sets the memory type */ + +#define COHERENCY_CTRL2 0x8E4 /* Cache Coherency Control Register 2 */ +#define COHERENCY_CTRL3 0x8E8 /* Cache Coherency Control Register 3 */ + +#define MULTI_LANE_CTRL 0x8C0 /* UpConfigure Multi-lane Control */ +#define MULTI_LANE_CTRL_UPCFG_SUPPORT BIT(7) /* Upconfigure Support */ + +#define PCIE_VERSION_NUMBER 0x8F8 /* PCIe Controller IIP Release Version Number */ +#define PCIE_VERSION_TYPE 0x8FC /* PCIe Controller IIP Release Version Type */ /* * iATU inbound and outbound windows CSRs. Before the IP-core v4.80a each @@ -186,38 +199,36 @@ * v4.80a in a way so the viewport was unrolled into the directly accessible * iATU/eDMA CSRs space. */ -#define PCIE_ATU_VIEWPORT 0x900 -#define PCIE_ATU_REGION_DIR_IB BIT(31) -#define PCIE_ATU_REGION_DIR_OB 0 -#define PCIE_ATU_VIEWPORT_BASE 0x904 -#define PCIE_ATU_UNROLL_BASE(dir, index) \ - (((index) << 9) | ((dir == PCIE_ATU_REGION_DIR_IB) ? BIT(8) : 0)) -#define PCIE_ATU_VIEWPORT_SIZE 0x2C -#define PCIE_ATU_REGION_CTRL1 0x000 -#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13) -#define PCIE_ATU_TD BIT(8) -#define PCIE_ATU_FUNC_NUM(pf) ((pf) << 20) -#define PCIE_ATU_REGION_CTRL2 0x004 -#define PCIE_ATU_ENABLE BIT(31) -#define PCIE_ATU_BAR_MODE_ENABLE BIT(30) -#define PCIE_ATU_CFG_SHIFT_MODE_ENABLE BIT(28) -#define PCIE_ATU_INHIBIT_PAYLOAD BIT(22) -#define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19) -#define PCIE_ATU_LOWER_BASE 0x008 -#define PCIE_ATU_UPPER_BASE 0x00C -#define PCIE_ATU_LIMIT 0x010 -#define PCIE_ATU_LOWER_TARGET 0x014 -#define PCIE_ATU_BUS(x) FIELD_PREP(GENMASK(31, 24), x) -#define PCIE_ATU_DEV(x) FIELD_PREP(GENMASK(23, 19), x) -#define PCIE_ATU_FUNC(x) FIELD_PREP(GENMASK(18, 16), x) -#define PCIE_ATU_UPPER_TARGET 0x018 -#define PCIE_ATU_UPPER_LIMIT 0x020 - -#define PCIE_MISC_CONTROL_1_OFF 0x8BC -#define PCIE_DBI_RO_WR_EN BIT(0) - -#define PCIE_MSIX_DOORBELL 0x948 -#define PCIE_MSIX_DOORBELL_PF_SHIFT 24 +#define IATU_VIEWPORT 0x900 /* iATU Index */ +#define ATU_REGION_DIR_IB BIT(31) +#define ATU_REGION_DIR_OB 0 +#define ATU_VIEWPORT_BASE 0x904 +#define ATU_UNROLL_BASE(dir, index) \ + (((index) << 9) | ((dir == ATU_REGION_DIR_IB) ? BIT(8) : 0)) +#define ATU_VIEWPORT_SIZE 0x2C +#define ATU_REGION_CTRL1 0x000 +#define ATU_INCREASE_REGION_SIZE BIT(13) +#define ATU_TD BIT(8) +#define ATU_FUNC_NUM(pf) ((pf) << 20) +#define ATU_REGION_CTRL2 0x004 +#define ATU_ENABLE BIT(31) +#define ATU_BAR_MODE_ENABLE BIT(30) +#define ATU_CFG_SHIFT_MODE_ENABLE BIT(28) +#define ATU_DMA_BYPASS BIT(27) +#define ATU_INHIBIT_PAYLOAD BIT(22) +#define ATU_FUNC_NUM_MATCH_EN BIT(19) +#define ATU_LOWER_BASE 0x008 +#define ATU_UPPER_BASE 0x00C +#define ATU_LIMIT 0x010 +#define ATU_LOWER_TARGET 0x014 +#define ATU_BUS(x) FIELD_PREP(GENMASK(31, 24), x) +#define ATU_DEV(x) FIELD_PREP(GENMASK(23, 19), x) +#define ATU_FUNC(x) FIELD_PREP(GENMASK(18, 16), x) +#define ATU_UPPER_TARGET 0x018 +#define ATU_UPPER_LIMIT 0x020 + +#define MSIX_DOORBELL 0x948 +#define MSIX_DOORBELL_PF_MASK GENMASK(28, 24) /* * eDMA CSRs. DW PCIe IP-core v4.70a and older had the eDMA registers accessible @@ -258,37 +269,24 @@ #define MARGINING_MAXLANES GENMASK(20, 16) #define MARGINING_SAMPLE_RATE_TIMING GENMASK(13, 8) #define MARGINING_SAMPLE_RATE_VOLTAGE GENMASK(5, 0) -/* - * iATU Unroll-specific register definitions - * From 4.80 core version the address translation will be made by unroll - */ -#define PCIE_ATU_UNR_REGION_CTRL1 0x00 -#define PCIE_ATU_UNR_REGION_CTRL2 0x04 -#define PCIE_ATU_UNR_LOWER_BASE 0x08 -#define PCIE_ATU_UNR_UPPER_BASE 0x0C -#define PCIE_ATU_UNR_LOWER_LIMIT 0x10 -#define PCIE_ATU_UNR_LOWER_TARGET 0x14 -#define PCIE_ATU_UNR_UPPER_TARGET 0x18 -#define PCIE_ATU_UNR_UPPER_LIMIT 0x20 /* * RAS-DES register definitions */ -#define PCIE_RAS_DES_EVENT_COUNTER_CONTROL 0x8 +#define RAS_DES_EVENT_COUNTER_CONTROL 0x8 #define EVENT_COUNTER_ALL_CLEAR 0x3 #define EVENT_COUNTER_ENABLE_ALL 0x7 -#define EVENT_COUNTER_ENABLE_SHIFT 2 -#define EVENT_COUNTER_EVENT_SEL_MASK GENMASK(7, 0) -#define EVENT_COUNTER_EVENT_SEL_SHIFT 16 +#define EVENT_COUNTER_ENABLE_MASK GENMASK(4, 2) +#define EVENT_COUNTER_EVENT_SEL_MASK GENMASK(23, 16) +#define EVENT_COUNTER_GROUP_SEL_MASK GENMASK(27, 24) #define EVENT_COUNTER_EVENT_Tx_L0S 0x2 #define EVENT_COUNTER_EVENT_Rx_L0S 0x3 #define EVENT_COUNTER_EVENT_L1 0x5 #define EVENT_COUNTER_EVENT_L1_1 0x7 #define EVENT_COUNTER_EVENT_L1_2 0x8 -#define EVENT_COUNTER_GROUP_SEL_SHIFT 24 #define EVENT_COUNTER_GROUP_5 0x5 -#define PCIE_RAS_DES_EVENT_COUNTER_DATA 0xc +#define RAS_DES_EVENT_COUNTER_DATA 0xc /* PTM register definitions */ #define PTM_RES_REQ_CTRL 0x8 @@ -376,7 +374,7 @@ enum dw_pcie_core_rst { }; enum dw_pcie_ltssm { - /* Need to align with PCIE_PORT_DEBUG0 bits 0:5 */ + /* Need to align with PORT_LINK_DEBUG0 bits 0:5 */ DW_PCIE_LTSSM_DETECT_QUIET = 0x0, DW_PCIE_LTSSM_DETECT_ACT = 0x1, DW_PCIE_LTSSM_POLL_ACTIVE = 0x2, @@ -454,7 +452,7 @@ struct dw_pcie_rp { int irq; const struct dw_pcie_host_ops *ops; int msi_irq[MAX_MSI_CTRLS]; - struct irq_domain *irq_domain; + struct irq_domain *msi_irq_domain; dma_addr_t msi_data; struct irq_chip *msi_irq_chip; u32 num_vectors; @@ -469,12 +467,16 @@ struct dw_pcie_rp { struct pci_config_window *cfg; bool ecam_enabled; bool native_ecam; + bool bypass_ob_mem_iatu; bool skip_l23_ready; bool skip_pwrctrl_off; + /* Set if the RC loses power during suspend */ + bool suspend_poweroff; }; struct dw_pcie_ep_ops { int (*pre_init)(struct dw_pcie_ep *ep); + void (*post_deinit)(struct dw_pcie_ep *ep); int (*init)(struct dw_pcie_ep *ep); int (*raise_irq)(struct dw_pcie_ep *ep, u8 func_no, unsigned int type, u16 interrupt_num); @@ -813,9 +815,9 @@ static inline void dw_pcie_dbi_ro_wr_en(struct dw_pcie *pci) u32 reg; u32 val; - reg = PCIE_MISC_CONTROL_1_OFF; + reg = MISC_CONTROL_1; val = dw_pcie_readl_dbi(pci, reg); - val |= PCIE_DBI_RO_WR_EN; + val |= DBI_RO_WR_EN; dw_pcie_writel_dbi(pci, reg, val); } @@ -824,9 +826,9 @@ static inline void dw_pcie_dbi_ro_wr_dis(struct dw_pcie *pci) u32 reg; u32 val; - reg = PCIE_MISC_CONTROL_1_OFF; + reg = MISC_CONTROL_1; val = dw_pcie_readl_dbi(pci, reg); - val &= ~PCIE_DBI_RO_WR_EN; + val &= ~DBI_RO_WR_EN; dw_pcie_writel_dbi(pci, reg, val); } @@ -851,9 +853,9 @@ static inline enum dw_pcie_ltssm dw_pcie_get_ltssm(struct dw_pcie *pci) if (pci->ops && pci->ops->get_ltssm) return pci->ops->get_ltssm(pci); - val = dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0); + val = dw_pcie_readl_dbi(pci, PORT_LINK_DEBUG0); - return (enum dw_pcie_ltssm)FIELD_GET(PORT_LOGIC_LTSSM_STATE_MASK, val); + return (enum dw_pcie_ltssm)FIELD_GET(PORT_LINK_DEBUG0_LTSSM_MASK, val); } const char *dw_pcie_ltssm_status_string(enum dw_pcie_ltssm ltssm); @@ -868,7 +870,7 @@ void dw_pcie_free_msi(struct dw_pcie_rp *pp); int dw_pcie_setup_rc(struct dw_pcie_rp *pp); int dw_pcie_host_init(struct dw_pcie_rp *pp); void dw_pcie_host_deinit(struct dw_pcie_rp *pp); -int dw_pcie_allocate_domains(struct dw_pcie_rp *pp); +int dw_pcie_allocate_msi_domains(struct dw_pcie_rp *pp); void __iomem *dw_pcie_own_conf_map_bus(struct pci_bus *bus, unsigned int devfn, int where); #else @@ -909,7 +911,7 @@ static inline void dw_pcie_host_deinit(struct dw_pcie_rp *pp) { } -static inline int dw_pcie_allocate_domains(struct dw_pcie_rp *pp) +static inline int dw_pcie_allocate_msi_domains(struct dw_pcie_rp *pp) { return 0; } diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/drivers/pci/controller/dwc/pcie-dw-rockchip.c index 1c8fdbd4cd17..bd86e32cc2b1 100644 --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c @@ -21,6 +21,7 @@ #include <linux/phy/phy.h> #include <linux/platform_device.h> #include <linux/regmap.h> +#include <linux/regulator/consumer.h> #include <linux/reset.h> #include <linux/workqueue.h> #include <trace/events/pci_controller.h> diff --git a/drivers/pci/controller/dwc/pcie-eswin.c b/drivers/pci/controller/dwc/pcie-eswin.c index ce8d64f8a395..1bb44696057e 100644 --- a/drivers/pci/controller/dwc/pcie-eswin.c +++ b/drivers/pci/controller/dwc/pcie-eswin.c @@ -388,6 +388,7 @@ static const struct of_device_id eswin_pcie_of_match[] = { { .compatible = "eswin,eic7700-pcie", .data = &eswin_eic7700_data }, {} }; +MODULE_DEVICE_TABLE(of, eswin_pcie_of_match); static struct platform_driver eswin_pcie_driver = { .probe = eswin_pcie_probe, diff --git a/drivers/pci/controller/dwc/pcie-fu740.c b/drivers/pci/controller/dwc/pcie-fu740.c index d0a34f680397..6c7ce1603170 100644 --- a/drivers/pci/controller/dwc/pcie-fu740.c +++ b/drivers/pci/controller/dwc/pcie-fu740.c @@ -215,9 +215,9 @@ static int fu740_pcie_start_link(struct dw_pcie *pci) tmp |= orig; dw_pcie_writel_dbi(pci, cap_exp + PCI_EXP_LNKCAP, tmp); - tmp = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - tmp |= PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, tmp); + tmp = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + tmp |= SPEED_CHANGE; + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, tmp); ret = dw_pcie_wait_for_link(pci); if (ret) { diff --git a/drivers/pci/controller/dwc/pcie-histb.c b/drivers/pci/controller/dwc/pcie-histb.c index a52071589377..432a54c5bfce 100644 --- a/drivers/pci/controller/dwc/pcie-histb.c +++ b/drivers/pci/controller/dwc/pcie-histb.c @@ -18,6 +18,7 @@ #include <linux/pci.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> +#include <linux/regulator/consumer.h> #include <linux/resource.h> #include <linux/reset.h> diff --git a/drivers/pci/controller/dwc/pcie-nxp-s32g.c b/drivers/pci/controller/dwc/pcie-nxp-s32g.c index 31e1169b8ab6..386e44dd0477 100644 --- a/drivers/pci/controller/dwc/pcie-nxp-s32g.c +++ b/drivers/pci/controller/dwc/pcie-nxp-s32g.c @@ -107,7 +107,7 @@ static void s32g_pcie_reset_mstr_ace(struct dw_pcie *pci) u32 ddr_base_high = upper_32_bits(S32G_MEMORY_BOUNDARY_ADDR); dw_pcie_dbi_ro_wr_en(pci); - dw_pcie_writel_dbi(pci, COHERENCY_CONTROL_3_OFF, 0x0); + dw_pcie_writel_dbi(pci, COHERENCY_CTRL3, 0x0); /* * Ncore is a cache-coherent interconnect module that enables the @@ -125,9 +125,9 @@ static void s32g_pcie_reset_mstr_ace(struct dw_pcie *pci) * Define the start of DDR as seen by Linux as this boundary between * "memory" and "peripherals", with peripherals being below. */ - dw_pcie_writel_dbi(pci, COHERENCY_CONTROL_1_OFF, + dw_pcie_writel_dbi(pci, COHERENCY_CTRL1, (ddr_base_low & CFG_MEMTYPE_BOUNDARY_LOW_ADDR_MASK)); - dw_pcie_writel_dbi(pci, COHERENCY_CONTROL_2_OFF, ddr_base_high); + dw_pcie_writel_dbi(pci, COHERENCY_CTRL2, ddr_base_high); dw_pcie_dbi_ro_wr_dis(pci); } @@ -154,13 +154,13 @@ static int s32g_init_pcie_controller(struct dw_pcie_rp *pp) dw_pcie_dbi_ro_wr_en(pci); - val = dw_pcie_readl_dbi(pci, PCIE_PORT_FORCE); - val |= PORT_FORCE_DO_DESKEW_FOR_SRIS; - dw_pcie_writel_dbi(pci, PCIE_PORT_FORCE, val); + val = dw_pcie_readl_dbi(pci, PORT_FORCE_LINK); + val |= PORT_FORCE_LINK_DDFS; + dw_pcie_writel_dbi(pci, PORT_FORCE_LINK, val); - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val |= GEN3_RELATED_OFF_EQ_PHASE_2_3; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val |= GEN3_CTRL_EQ_PHASE_2_3; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); dw_pcie_dbi_ro_wr_dis(pci); diff --git a/drivers/pci/controller/dwc/pcie-qcom-common.c b/drivers/pci/controller/dwc/pcie-qcom-common.c index 0da73caf2011..0b39648d2214 100644 --- a/drivers/pci/controller/dwc/pcie-qcom-common.c +++ b/drivers/pci/controller/dwc/pcie-qcom-common.c @@ -15,9 +15,9 @@ void qcom_pcie_common_set_equalization(struct dw_pcie *pci) u16 speed; /* - * GEN3_RELATED_OFF register is repurposed to apply equalization + * GEN3_CTRL register is repurposed to apply equalization * settings at various data transmission rates through registers namely - * GEN3_EQ_*. The RATE_SHADOW_SEL bit field of GEN3_RELATED_OFF + * GEN3_EQ_*. The RATE_SHADOW_SEL bit field of GEN3_CTRL * determines the data rate for which these equalization settings are * applied. */ @@ -28,25 +28,25 @@ void qcom_pcie_common_set_equalization(struct dw_pcie *pci) break; } - reg = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - reg &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL; - FIELD_MODIFY(GEN3_RELATED_OFF_RATE_SHADOW_SEL_MASK, ®, + reg = dw_pcie_readl_dbi(pci, GEN3_CTRL); + reg &= ~GEN3_CTRL_GEN3_ZRXDC_NONCOMPL; + FIELD_MODIFY(GEN3_CTRL_RATE_SHADOW_SEL_MASK, ®, speed - PCIE_SPEED_8_0GT); - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, reg); + dw_pcie_writel_dbi(pci, GEN3_CTRL, reg); - reg = dw_pcie_readl_dbi(pci, GEN3_EQ_FB_MODE_DIR_CHANGE_OFF); + reg = dw_pcie_readl_dbi(pci, GEN3_EQ_FB_MODE_DIR_CHANGE); FIELD_MODIFY(GEN3_EQ_FMDC_T_MIN_PHASE23, ®, 0x1); FIELD_MODIFY(GEN3_EQ_FMDC_N_EVALS, ®, 0xd); FIELD_MODIFY(GEN3_EQ_FMDC_MAX_PRE_CURSOR_DELTA, ®, 0x5); FIELD_MODIFY(GEN3_EQ_FMDC_MAX_POST_CURSOR_DELTA, ®, 0x5); - dw_pcie_writel_dbi(pci, GEN3_EQ_FB_MODE_DIR_CHANGE_OFF, reg); + dw_pcie_writel_dbi(pci, GEN3_EQ_FB_MODE_DIR_CHANGE, reg); - reg = dw_pcie_readl_dbi(pci, GEN3_EQ_CONTROL_OFF); - reg &= ~(GEN3_EQ_CONTROL_OFF_FB_MODE | - GEN3_EQ_CONTROL_OFF_PHASE23_EXIT_MODE | - GEN3_EQ_CONTROL_OFF_FOM_INC_INITIAL_EVAL | - GEN3_EQ_CONTROL_OFF_PSET_REQ_VEC); - dw_pcie_writel_dbi(pci, GEN3_EQ_CONTROL_OFF, reg); + reg = dw_pcie_readl_dbi(pci, GEN3_EQ_CTRL); + reg &= ~(GEN3_EQ_CTRL_FB_MODE | + GEN3_EQ_CTRL_PHASE23_EXIT_MODE | + GEN3_EQ_CTRL_FOM_INC_INITIAL_EVAL | + GEN3_EQ_CTRL_PSET_REQ_VEC); + dw_pcie_writel_dbi(pci, GEN3_EQ_CTRL, reg); } } EXPORT_SYMBOL_GPL(qcom_pcie_common_set_equalization); diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c index 56184e6ca6e6..e6afdc6198ac 100644 --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c @@ -19,6 +19,7 @@ #include <linux/phy/phy.h> #include <linux/platform_device.h> #include <linux/pm_domain.h> +#include <linux/pm_runtime.h> #include <linux/regmap.h> #include <linux/reset.h> #include <linux/module.h> @@ -944,6 +945,8 @@ static void qcom_pcie_ep_remove(struct platform_device *pdev) if (pcie_ep->link_status == QCOM_PCIE_EP_LINK_DISABLED) return; + pci_epc_deinit_notify(pcie_ep->pci.ep.epc); + dw_pcie_ep_deinit(&pcie_ep->pci.ep); qcom_pcie_disable_resources(pcie_ep); } diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c index b58a607b713f..d903f10211c2 100644 --- a/drivers/pci/controller/dwc/pcie-qcom.c +++ b/drivers/pci/controller/dwc/pcie-qcom.c @@ -20,6 +20,7 @@ #include <linux/kernel.h> #include <linux/limits.h> #include <linux/init.h> +#include <linux/mutex.h> #include <linux/of.h> #include <linux/of_pci.h> #include <linux/pci.h> @@ -73,6 +74,23 @@ #define PARF_BDF_TO_SID_TABLE_N 0x2000 #define PARF_BDF_TO_SID_CFG 0x2c00 +/* + * Outbound blocker range registers. The blocked range has a write pair + * (WR_BASE/WR_LIMIT) and a read pair (RD_BASE/RD_LIMIT); each address is + * split into a low (32-bit) and a HI (upper 32-bit) register. + */ +#define PARF_BLOCK_SLV_AXI_WR_BASE 0x360 +#define PARF_BLOCK_SLV_AXI_WR_BASE_HI 0x364 +#define PARF_BLOCK_SLV_AXI_WR_LIMIT 0x368 +#define PARF_BLOCK_SLV_AXI_WR_LIMIT_HI 0x36c +#define PARF_BLOCK_SLV_AXI_RD_BASE 0x370 +#define PARF_BLOCK_SLV_AXI_RD_BASE_HI 0x374 +#define PARF_BLOCK_SLV_AXI_RD_LIMIT 0x378 +#define PARF_BLOCK_SLV_AXI_RD_LIMIT_HI 0x37c + +#define PARF_ECAM_BASE 0x380 +#define PARF_ECAM_BASE_HI 0x384 + /* ELBI registers */ #define ELBI_SYS_CTRL 0x04 #define ELBI_SYS_STTS 0x08 @@ -90,6 +108,7 @@ /* PARF_SYS_CTRL register fields */ #define MAC_PHY_POWERDOWN_IN_P2_D_MUX_EN BIT(29) +#define OUTBOUND_BLOCKER_EN BIT(26) #define MST_WAKEUP_EN BIT(13) #define SLV_WAKEUP_EN BIT(12) #define MSTR_ACLK_CGC_DIS BIT(10) @@ -308,6 +327,7 @@ struct qcom_pcie { struct gpio_desc *reset; int global_irq; bool use_pm_opp; + struct mutex reset_lock; }; #define to_qcom_pcie(x) dev_get_drvdata((x)->dev) @@ -445,6 +465,25 @@ static void qcom_pcie_configure_dbi_atu_base(struct qcom_pcie *pcie) } } +static void qcom_pcie_init_outbound_blocker(struct qcom_pcie *pcie) +{ + struct dw_pcie *pci = pcie->pci; + + /* ECAM base must match the DBI base address */ + writel(lower_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE); + writel(upper_32_bits(pci->dbi_phys_addr), pcie->parf + PARF_ECAM_BASE_HI); + + writel(0, pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE); + writel(0, pcie->parf + PARF_BLOCK_SLV_AXI_WR_BASE_HI); + writel(U32_MAX, pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT); + writel(U32_MAX, pcie->parf + PARF_BLOCK_SLV_AXI_WR_LIMIT_HI); + + writel(0, pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE); + writel(0, pcie->parf + PARF_BLOCK_SLV_AXI_RD_BASE_HI); + writel(U32_MAX, pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT); + writel(U32_MAX, pcie->parf + PARF_BLOCK_SLV_AXI_RD_LIMIT_HI); +} + static void qcom_pcie_2_1_0_ltssm_enable(struct qcom_pcie *pcie) { struct dw_pcie *pci = pcie->pci; @@ -990,6 +1029,8 @@ static int qcom_pcie_post_init_2_3_3(struct qcom_pcie *pcie) dw_pcie_dbi_ro_wr_dis(pci); + qcom_pcie_init_outbound_blocker(pcie); + return 0; } @@ -1104,6 +1145,8 @@ static int qcom_pcie_post_init_2_7_0(struct qcom_pcie *pcie) qcom_pcie_set_slot_cap(pcie->pci); + qcom_pcie_init_outbound_blocker(pcie); + return 0; } @@ -1143,50 +1186,90 @@ static void qcom_pcie_deinit_2_7_0(struct qcom_pcie *pcie) static int qcom_pcie_config_sid_1_9_0(struct qcom_pcie *pcie) { - /* iommu map structure */ - struct { - u32 bdf; - u32 phandle; - u32 smmu_sid; - u32 smmu_sid_len; - } *map; void __iomem *bdf_to_sid_base = pcie->parf + PARF_BDF_TO_SID_TABLE_N; struct device *dev = pcie->pci->dev; + struct device_node *iommu_np; u8 qcom_pcie_crc8_table[CRC8_TABLE_SIZE]; - int i, nr_map, size = 0; - u32 smmu_sid_base; + const __be32 *map; + u32 iommu_cells, entry_cells, phandle, smmu_sid_base; + int i, nr_cells, nr_map, size = 0; u32 val; - of_get_property(dev->of_node, "iommu-map", &size); - if (!size) + map = of_get_property(dev->of_node, "iommu-map", &size); + if (!map || !size) return 0; + if (size % sizeof(*map)) { + dev_err(dev, "Malformed iommu-map property\n"); + return -EINVAL; + } + nr_cells = size / sizeof(*map); + + /* + * Each iommu-map entry is: rid-base (1 cell), phandle (1 cell), + * IOMMU specifier (#iommu-cells cells), length (1 cell). Read + * #iommu-cells from the IOMMU provider referenced by the first + * entry to compute the per-entry stride. + */ + phandle = be32_to_cpu(map[1]); + iommu_np = of_find_node_by_phandle(phandle); + if (!iommu_np) { + dev_err(dev, "Failed to find IOMMU node in iommu-map\n"); + return -ENODEV; + } + + if (of_property_read_u32(iommu_np, "#iommu-cells", &iommu_cells)) + iommu_cells = 1; + of_node_put(iommu_np); + + entry_cells = 3 + iommu_cells; + + /* + * Retain backward compatibility with DTs that describe iommu-map + * with 4-cell entries against an IOMMU declaring #iommu-cells = 2, + * matching the fallback in drivers/of/base.c::of_check_bad_map(). + */ + if (iommu_cells == 2 && !(nr_cells % 4)) { + bool legacy = true; + + for (i = 0; i < nr_cells; i += 4) { + if (be32_to_cpu(map[i + 1]) != phandle || + be32_to_cpu(map[i + 3]) != 1) { + legacy = false; + break; + } + } + + if (legacy) { + dev_warn_once(dev, "iommu-map has 1-cell entries with #iommu-cells=2, using 1-cell\n"); + entry_cells = 4; + } + } + + if (nr_cells % entry_cells) { + dev_err(dev, "Malformed iommu-map property\n"); + return -EINVAL; + } + nr_map = nr_cells / entry_cells; + /* Enable BDF to SID translation by disabling bypass mode (default) */ val = readl(pcie->parf + PARF_BDF_TO_SID_CFG); val &= ~BDF_TO_SID_BYPASS; writel(val, pcie->parf + PARF_BDF_TO_SID_CFG); - map = kzalloc(size, GFP_KERNEL); - if (!map) - return -ENOMEM; - - of_property_read_u32_array(dev->of_node, "iommu-map", (u32 *)map, - size / sizeof(u32)); - - nr_map = size / (sizeof(*map)); - crc8_populate_msb(qcom_pcie_crc8_table, QCOM_PCIE_CRC8_POLYNOMIAL); /* Registers need to be zero out first */ memset_io(bdf_to_sid_base, 0, CRC8_TABLE_SIZE * sizeof(u32)); /* Extract the SMMU SID base from the first entry of iommu-map */ - smmu_sid_base = map[0].smmu_sid; + smmu_sid_base = be32_to_cpu(map[2]); /* Look for an available entry to hold the mapping */ for (i = 0; i < nr_map; i++) { - __be16 bdf_be = cpu_to_be16(map[i].bdf); - u32 val; + u32 bdf = be32_to_cpu(map[i * entry_cells]); + u32 sid = be32_to_cpu(map[i * entry_cells + 2]); + __be16 bdf_be = cpu_to_be16(bdf); u8 hash; hash = crc8(qcom_pcie_crc8_table, (u8 *)&bdf_be, sizeof(bdf_be), 0); @@ -1208,12 +1291,10 @@ static int qcom_pcie_config_sid_1_9_0(struct qcom_pcie *pcie) } /* BDF [31:16] | SID [15:8] | NEXT [7:0] */ - val = map[i].bdf << 16 | (map[i].smmu_sid - smmu_sid_base) << 8 | 0; + val = bdf << 16 | (sid - smmu_sid_base) << 8 | 0; writel(val, bdf_to_sid_base + hash * sizeof(u32)); } - kfree(map); - return 0; } @@ -1295,9 +1376,9 @@ static int qcom_pcie_post_init_2_9_0(struct qcom_pcie *pcie) writel(DEVICE_TYPE_RC, pcie->parf + PARF_DEVICE_TYPE); writel(BYPASS | MSTR_AXI_CLK_EN | AHB_CLK_EN, pcie->parf + PARF_MHI_CLOCK_RESET_CTRL); - writel(GEN3_RELATED_OFF_RXEQ_RGRDLESS_RXTS | - GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL, - pci->dbi_base + GEN3_RELATED_OFF); + writel(GEN3_CTRL_RXEQ_RGRDLESS_RXTS | + GEN3_CTRL_GEN3_ZRXDC_NONCOMPL, + pci->dbi_base + GEN3_CTRL); writel(MST_WAKEUP_EN | SLV_WAKEUP_EN | MSTR_ACLK_CGC_DIS | SLV_ACLK_CGC_DIS | CORE_CLK_CGC_DIS | @@ -1322,6 +1403,8 @@ static int qcom_pcie_post_init_2_9_0(struct qcom_pcie *pcie) for (i = 0; i < 256; i++) writel(0, pcie->parf + PARF_BDF_TO_SID_TABLE_N + (4 * i)); + qcom_pcie_init_outbound_blocker(pcie); + return 0; } @@ -1382,6 +1465,18 @@ static void qcom_pcie_configure_ports(struct qcom_pcie *pcie) dw_pcie_program_t_power_on(pcie->pci, port->l1ss_t_power_on); } +static void qcom_pcie_enable_outbound_blocker(struct qcom_pcie *pcie) +{ + u32 sys_ctrl; + + sys_ctrl = readl(pcie->parf + PARF_SYS_CTRL); + sys_ctrl |= OUTBOUND_BLOCKER_EN; + writel(sys_ctrl, pcie->parf + PARF_SYS_CTRL); + + /* Flush the write so the blocker is enabled before this function returns */ + readl(pcie->parf + PARF_SYS_CTRL); +} + static int qcom_pcie_host_init(struct dw_pcie_rp *pp) { struct dw_pcie *pci = to_dw_pcie_from_pp(pp); @@ -1775,6 +1870,8 @@ static int qcom_pcie_reset_root_port(struct pci_host_bridge *bridge, u32 val; int ret; + guard(mutex)(&pcie->reset_lock); + /* Wait for the pending transactions to be completed */ ret = readl_relaxed_poll_timeout(pcie->parf + PARF_STATUS, val, val & FLUSH_COMPLETED, 10, @@ -1821,6 +1918,9 @@ static int qcom_pcie_reset_root_port(struct pci_host_bridge *bridge, qcom_pcie_start_link(pci); ret = dw_pcie_wait_for_link(pci); + /* This linkdown might have been a hot-unplug */ + if (ret == -ENODEV) + ret = 0; if (ret) return ret; @@ -1878,6 +1978,11 @@ static irqreturn_t qcom_pcie_global_irq_thread(int irq, void *data) if (test_and_clear_bit(INT_ALL_LINK_DOWN, &status)) { dev_dbg(dev, "Received Link down event\n"); + + mutex_lock(&pcie->reset_lock); + qcom_pcie_enable_outbound_blocker(pcie); + mutex_unlock(&pcie->reset_lock); + for_each_pci_bridge(port, pp->bridge->bus) { if (pci_pcie_type(port) == PCI_EXP_TYPE_ROOT_PORT) pci_host_handle_link_down(port); @@ -1917,11 +2022,13 @@ static int qcom_pcie_ecam_host_init(struct pci_config_window *cfg) */ raw_spin_lock_init(&pp->lock); - ret = dw_pcie_msi_host_init(pp); - if (ret) - return ret; + pp->use_imsi_rx = !pci_host_of_has_msi_map(dev); + if (pp->use_imsi_rx) { + ret = dw_pcie_msi_host_init(pp); + if (ret) + return ret; + } - pp->use_imsi_rx = true; dw_pcie_msi_init(pp); return devm_add_action_or_reset(dev, qcom_pci_free_msi, pp); @@ -1953,7 +2060,7 @@ static bool qcom_pcie_is_child_node(struct device *dev, return false; } -/* Parse PERST# from all nodes in depth first manner starting from @np */ +/* Collect PERST# GPIOs from PCI bridge nodes depth-first, starting at @np */ static int qcom_pcie_parse_perst(struct qcom_pcie *pcie, struct qcom_pcie_port *port, struct device_node *np) @@ -2019,6 +2126,9 @@ skip_perst_parsing: parse_child_node: for_each_available_child_of_node_scoped(np, child) { + if (!of_node_is_type(child, "pci")) + continue; + ret = qcom_pcie_parse_perst(pcie, port, child); if (ret) return ret; @@ -2204,6 +2314,10 @@ static int qcom_pcie_probe(struct platform_device *pdev) INIT_LIST_HEAD(&pcie->ports); + ret = devm_mutex_init(dev, &pcie->reset_lock); + if (ret) + goto err_pm_runtime_put; + pci->dev = dev; pci->ops = &dw_pcie_ops; pp = &pci->pp; @@ -2341,6 +2455,10 @@ static int qcom_pcie_suspend_noirq(struct device *dev) { struct qcom_pcie *pcie; int ret = 0; + const struct qcom_pcie_cfg *pcie_cfg = of_device_get_match_data(dev); + + if (pcie_cfg && pcie_cfg->firmware_managed) + return 0; pcie = dev_get_drvdata(dev); if (!pcie) @@ -2399,6 +2517,10 @@ static int qcom_pcie_resume_noirq(struct device *dev) { struct qcom_pcie *pcie; int ret; + const struct qcom_pcie_cfg *pcie_cfg = of_device_get_match_data(dev); + + if (pcie_cfg && pcie_cfg->firmware_managed) + return 0; pcie = dev_get_drvdata(dev); if (!pcie) diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c index fbe465a29068..fceecbd16355 100644 --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c @@ -19,6 +19,7 @@ #include <linux/of_address.h> #include <linux/of_irq.h> #include <linux/pci.h> +#include <linux/phy/phy.h> #include <linux/platform_device.h> #include <linux/pm_runtime.h> #include <linux/reset.h> @@ -36,6 +37,7 @@ /* MSI Capability */ #define MSICAP0 0x0050 +#define MSICAP0_MMESCAP_MASK GENMASK(19, 17) #define MSICAP0_MSIE BIT(16) /* PCIe Interrupt Status 0 */ @@ -74,6 +76,11 @@ #define PCIEPWRMNGCTRL 0x0070 #define APP_CLK_REQ_N BIT(11) #define APP_CLK_PM_EN BIT(10) +#define APP_READY_ENTR_L23 BIT(6) +#define APP_REQ_ENTR_L1 BIT(5) + +/* PCI Express capability */ +#define EXPCAP(x) (0x0070 + (x)) #define RCAR_NUM_SPEED_CHANGE_RETRIES 10 #define RCAR_MAX_LINK_SPEED 4 @@ -87,8 +94,10 @@ MODULE_FIRMWARE(RCAR_GEN4_PCIE_FIRMWARE_NAME); struct rcar_gen4_pcie; struct rcar_gen4_pcie_drvdata { - void (*additional_common_init)(struct rcar_gen4_pcie *rcar); + int (*init)(struct rcar_gen4_pcie *rcar); + void (*deinit)(struct rcar_gen4_pcie *rcar); int (*ltssm_control)(struct rcar_gen4_pcie *rcar, bool enable); + int (*speed_control)(struct rcar_gen4_pcie *rcar); enum dw_pcie_device_mode mode; }; @@ -96,7 +105,9 @@ struct rcar_gen4_pcie { struct dw_pcie dw; void __iomem *base; void __iomem *phy_base; + struct phy *phy; struct platform_device *pdev; + struct reset_control *perst; const struct rcar_gen4_pcie_drvdata *drvdata; }; #define to_rcar_gen4_pcie(_dw) container_of(_dw, struct rcar_gen4_pcie, dw) @@ -122,17 +133,17 @@ static int rcar_gen4_pcie_speed_change(struct dw_pcie *dw) u32 val; int i; - val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL); - val &= ~PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + val = dw_pcie_readl_dbi(dw, LINK_WIDTH_SPEED_CTRL); + val &= ~SPEED_CHANGE; + dw_pcie_writel_dbi(dw, LINK_WIDTH_SPEED_CTRL, val); - val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL); - val |= PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + val = dw_pcie_readl_dbi(dw, LINK_WIDTH_SPEED_CTRL); + val |= SPEED_CHANGE; + dw_pcie_writel_dbi(dw, LINK_WIDTH_SPEED_CTRL, val); for (i = 0; i < RCAR_NUM_SPEED_CHANGE_RETRIES; i++) { - val = dw_pcie_readl_dbi(dw, PCIE_LINK_WIDTH_SPEED_CONTROL); - if (!(val & PORT_LOGIC_SPEED_CHANGE)) + val = dw_pcie_readl_dbi(dw, LINK_WIDTH_SPEED_CTRL); + if (!(val & SPEED_CHANGE)) return 0; usleep_range(10000, 11000); } @@ -140,20 +151,10 @@ static int rcar_gen4_pcie_speed_change(struct dw_pcie *dw) return -ETIMEDOUT; } -/* - * Enable LTSSM of this controller and manually initiate the speed change. - * Always return 0. - */ -static int rcar_gen4_pcie_start_link(struct dw_pcie *dw) +static int rcar_gen4_pcie_speed_control(struct rcar_gen4_pcie *rcar) { - struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); - int i, changes, ret; - - if (rcar->drvdata->ltssm_control) { - ret = rcar->drvdata->ltssm_control(rcar, true); - if (ret) - return ret; - } + struct dw_pcie *dw = &rcar->dw; + int i, changes; /* * Require direct speed change with retrying here if the max_link_speed @@ -177,6 +178,45 @@ static int rcar_gen4_pcie_start_link(struct dw_pcie *dw) return 0; } +static int rcar_gen5_pcie_speed_control(struct rcar_gen4_pcie *rcar) +{ + struct dw_pcie *dw = &rcar->dw; + u32 lnkcap = dw_pcie_readl_dbi(dw, EXPCAP(PCI_EXP_LNKCAP)); + u32 lnksta = dw_pcie_readw_dbi(dw, EXPCAP(PCI_EXP_LNKSTA)); + u32 val; + + if ((lnksta & PCI_EXP_LNKSTA_CLS) == (lnkcap & PCI_EXP_LNKCAP_SLS)) + return 0; + + /* Retrain link */ + val = dw_pcie_readw_dbi(dw, EXPCAP(PCI_EXP_LNKCTL)); + val |= PCI_EXP_LNKCTL_RL; + dw_pcie_writew_dbi(dw, EXPCAP(PCI_EXP_LNKCTL), val); + + /* Wait for link retrain, 500ms must be enough for all link rates. */ + return read_poll_timeout(dw_pcie_readw_dbi, lnksta, !(lnksta & PCI_EXP_LNKSTA_LT), + 1000, 5 * PCIE_RESET_CONFIG_WAIT_MS * USEC_PER_MSEC, + false, dw, EXPCAP(PCI_EXP_LNKSTA)); +} + +/* + * Enable LTSSM of this controller and manually initiate the speed change. + * Always return 0. + */ +static int rcar_gen4_pcie_start_link(struct dw_pcie *dw) +{ + struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); + int ret; + + if (rcar->drvdata->ltssm_control) { + ret = rcar->drvdata->ltssm_control(rcar, true); + if (ret) + return ret; + } + + return rcar->drvdata->speed_control(rcar); +} + static void rcar_gen4_pcie_stop_link(struct dw_pcie *dw) { struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); @@ -197,6 +237,8 @@ static int rcar_gen4_pcie_common_init(struct rcar_gen4_pcie *rcar) return ret; } + reset_control_deassert(dw->core_rsts[DW_PCIE_CORE_RST].rstc); + if (!reset_control_status(dw->core_rsts[DW_PCIE_PWR_RST].rstc)) { reset_control_assert(dw->core_rsts[DW_PCIE_PWR_RST].rstc); /* @@ -241,8 +283,74 @@ static int rcar_gen4_pcie_common_init(struct rcar_gen4_pcie *rcar) reset_control_status(dw->core_rsts[DW_PCIE_PWR_RST].rstc); fsleep(1000); - if (rcar->drvdata->additional_common_init) - rcar->drvdata->additional_common_init(rcar); + return 0; + +err_unprepare: + reset_control_assert(dw->core_rsts[DW_PCIE_CORE_RST].rstc); + clk_bulk_disable_unprepare(DW_PCIE_NUM_CORE_CLKS, dw->core_clks); + + return ret; +} + +static int rcar_gen4_v4h_v4m_pcie_init(struct rcar_gen4_pcie *rcar) +{ + struct dw_pcie *dw = &rcar->dw; + u32 val; + int ret; + + /* R-Car Gen4 common initialization. */ + ret = rcar_gen4_pcie_common_init(rcar); + if (ret) + return ret; + + /* R-Car V4H and V4M specific additional initialization. */ + val = dw_pcie_readl_dbi(dw, PORT_LANE_SKEW); + val &= ~PORT_LANE_SKEW_INSERT_MASK; + if (dw->num_lanes < 4) + val |= BIT(6); + dw_pcie_writel_dbi(dw, PORT_LANE_SKEW, val); + + val = readl(rcar->base + PCIEPWRMNGCTRL); + val |= APP_CLK_REQ_N | APP_CLK_PM_EN; + writel(val, rcar->base + PCIEPWRMNGCTRL); + + return 0; +} + +static int rcar_gen5_pcie_init(struct rcar_gen4_pcie *rcar) +{ + struct dw_pcie *dw = &rcar->dw; + int ret; + u32 val; + + /* R-Car Gen4 and Gen5 common initialization. */ + ret = rcar_gen4_pcie_common_init(rcar); + if (ret) + return ret; + + /* R-Car Gen5 specific additional initialization. */ + ret = phy_init(rcar->phy); + if (ret) + goto err_unprepare; + + dw_pcie_dbi_ro_wr_en(dw); + + val = dw_pcie_readl_dbi(dw, PORT_LANE_SKEW); + val &= ~PORT_LANE_SKEW_INSERT_MASK; + if (dw->num_lanes < 8) + val |= BIT(6); + dw_pcie_writel_dbi(dw, PORT_LANE_SKEW, val); + + val = dw_pcie_readl_dbi(dw, MSICAP0); + FIELD_MODIFY(MSICAP0_MMESCAP_MASK, &val, 4); + dw_pcie_writel_dbi(dw, MSICAP0, val); + + dw_pcie_dbi_ro_wr_dis(dw); + + val = readl(rcar->base + PCIEPWRMNGCTRL); + val |= APP_CLK_REQ_N | APP_CLK_PM_EN | + APP_READY_ENTR_L23 | APP_REQ_ENTR_L1; + writel(val, rcar->base + PCIEPWRMNGCTRL); return 0; @@ -257,9 +365,16 @@ static void rcar_gen4_pcie_common_deinit(struct rcar_gen4_pcie *rcar) struct dw_pcie *dw = &rcar->dw; reset_control_assert(dw->core_rsts[DW_PCIE_PWR_RST].rstc); + reset_control_assert(dw->core_rsts[DW_PCIE_CORE_RST].rstc); clk_bulk_disable_unprepare(DW_PCIE_NUM_CORE_CLKS, dw->core_clks); } +static void rcar_gen5_pcie_deinit(struct rcar_gen4_pcie *rcar) +{ + phy_exit(rcar->phy); + rcar_gen4_pcie_common_deinit(rcar); +} + static int rcar_gen4_pcie_prepare(struct rcar_gen4_pcie *rcar) { struct device *dev = rcar->dw.dev; @@ -285,12 +400,29 @@ static void rcar_gen4_pcie_unprepare(struct rcar_gen4_pcie *rcar) static int rcar_gen4_pcie_get_resources(struct rcar_gen4_pcie *rcar) { + struct device *dev = rcar->dw.dev; + struct device_node *root_port; + rcar->phy_base = devm_platform_ioremap_resource_byname(rcar->pdev, "phy"); - if (IS_ERR(rcar->phy_base)) - return PTR_ERR(rcar->phy_base); + if (IS_ERR(rcar->phy_base)) { + rcar->phy_base = NULL; + rcar->phy = devm_phy_get(dev, NULL); + if (IS_ERR(rcar->phy)) + return PTR_ERR(rcar->phy); + } + + root_port = of_get_next_available_child(dev->of_node, NULL); + if (root_port) { + rcar->perst = of_reset_control_get_optional_exclusive(root_port, "perst"); + of_node_put(root_port); + if (IS_ERR(rcar->perst)) + return dev_err_probe(dev, PTR_ERR(rcar->perst), "Failed to get PERST#\n"); + } /* Renesas-specific registers */ rcar->base = devm_platform_ioremap_resource_byname(rcar->pdev, "app"); + if (IS_ERR(rcar->base)) + reset_control_put(rcar->perst); return PTR_ERR_OR_ZERO(rcar->base); } @@ -411,6 +543,72 @@ err: return ret; } +static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge, + struct pci_dev *dev) +{ + /* + * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes + * Max_Payload_Size (MPS). PCIe specification indicates that the MPS + * must not exceed minimum MPS of any element along the packet path. + * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes + * for header type 0 and 128 Bytes for header type 1. The PCIe core + * will not allow MPS to be set higher than MPSS; warn here in case + * something went very wrong in the core. + * + * For details, refer to chapter "104.1.1 Features" in either of: + * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or + * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or + * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026. + */ + WARN_ON(pcie_get_mps(dev) > 256); + + /* + * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for + * MRRS (Max Read Request Size) states: + * + * Please set "Max Read Request Size" to 128 bytes or 256 bytes. + * If "Max Read Request Size" is set to anything other than the + * above, the transferred data will not match the expected value. + * + * This limitation also seems to apply to devices issuing MRd TLPs. + * This limitation can be triggered by using non-HMB NVMe SSD with + * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus. + * Any write into the SSD (MRd TLP issued by the SSD) longer than + * 256 Bytes wraps around at 256 Byte boundary, and the same data + * are written into the SSD starting at offset 0 and at 256 Bytes. + * + * Limit Max_Read_Request_Size to at most 256 Bytes for each + * device connected to this PCIe controller to avoid this behavior. + * + * For details, refer to aforementioned chapter in either of: + * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or + * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or + * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026. + */ + bridge->no_inc_mrrs = 1; + if (pcie_get_readrq(dev) > 256) { + pci_info(dev, "Limiting MRRS to 256 bytes\n"); + pcie_set_readrq(dev, 256); + } + + return 0; +} + +static void rcar_gen4_pcie_host_perst_assert(struct dw_pcie_rp *pp, bool assert) +{ + struct dw_pcie *dw = to_dw_pcie_from_pp(pp); + struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); + + if (dw->pe_rst) { + gpiod_set_value_cansleep(dw->pe_rst, assert); + } else { + if (assert) + reset_control_assert(rcar->perst); + else + reset_control_deassert(rcar->perst); + } +} + /* Host mode */ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp) { @@ -418,9 +616,12 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp) struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); int ret; - gpiod_set_value_cansleep(dw->pe_rst, 1); + if (pp->bridge) + pp->bridge->enable_device = rcar_gen4_pcie_enable_device; - ret = rcar_gen4_pcie_common_init(rcar); + rcar_gen4_pcie_host_perst_assert(pp, true); + + ret = rcar->drvdata->init(rcar); if (ret) return ret; @@ -439,12 +640,12 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp) msleep(PCIE_T_PVPERL_MS); /* pe_rst requires 100msec delay */ - gpiod_set_value_cansleep(dw->pe_rst, 0); + rcar_gen4_pcie_host_perst_assert(pp, false); return 0; err: - rcar_gen4_pcie_common_deinit(rcar); + rcar->drvdata->deinit(rcar); return ret; } @@ -453,8 +654,8 @@ static void rcar_gen4_pcie_host_deinit(struct dw_pcie_rp *pp) struct dw_pcie *dw = to_dw_pcie_from_pp(pp); struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); - gpiod_set_value_cansleep(dw->pe_rst, 1); - rcar_gen4_pcie_common_deinit(rcar); + rcar_gen4_pcie_host_perst_assert(pp, true); + rcar->drvdata->deinit(rcar); } static const struct dw_pcie_host_ops rcar_gen4_pcie_host_ops = { @@ -470,6 +671,8 @@ static int rcar_gen4_add_dw_pcie_rp(struct rcar_gen4_pcie *rcar) return -ENODEV; pp->num_vectors = MAX_MSI_IRQS; + /* Reserve an iATU window for the generic PME_Turn_Off implementation. */ + pp->use_atu_msg = true; pp->ops = &rcar_gen4_pcie_host_ops; return dw_pcie_host_init(pp); @@ -487,7 +690,9 @@ static int rcar_gen4_pcie_ep_pre_init(struct dw_pcie_ep *ep) struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); int ret; - ret = rcar_gen4_pcie_common_init(rcar); + writel(0, rcar->base + PCIEDMAINTSTSEN); + + ret = rcar->drvdata->init(rcar); if (ret) return ret; @@ -496,10 +701,13 @@ static int rcar_gen4_pcie_ep_pre_init(struct dw_pcie_ep *ep) return 0; } -static void rcar_gen4_pcie_ep_deinit(struct rcar_gen4_pcie *rcar) +static void rcar_gen4_pcie_ep_post_deinit(struct dw_pcie_ep *ep) { + struct dw_pcie *dw = to_dw_pcie_from_ep(ep); + struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw); + writel(0, rcar->base + PCIEDMAINTSTSEN); - rcar_gen4_pcie_common_deinit(rcar); + rcar->drvdata->deinit(rcar); } static int rcar_gen4_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no, @@ -552,6 +760,7 @@ static unsigned int rcar_gen4_pcie_ep_get_dbi2_offset(struct dw_pcie_ep *ep, static const struct dw_pcie_ep_ops pcie_ep_ops = { .pre_init = rcar_gen4_pcie_ep_pre_init, + .post_deinit = rcar_gen4_pcie_ep_post_deinit, .raise_irq = rcar_gen4_pcie_ep_raise_irq, .get_features = rcar_gen4_pcie_ep_get_features, .get_dbi_offset = rcar_gen4_pcie_ep_get_dbi_offset, @@ -570,16 +779,13 @@ static int rcar_gen4_add_dw_pcie_ep(struct rcar_gen4_pcie *rcar) ep->ops = &pcie_ep_ops; ret = dw_pcie_ep_init(ep); - if (ret) { - rcar_gen4_pcie_ep_deinit(rcar); + if (ret) return ret; - } ret = dw_pcie_ep_init_registers(ep); if (ret) { dev_err(dev, "Failed to initialize DWC endpoint registers\n"); dw_pcie_ep_deinit(ep); - rcar_gen4_pcie_ep_deinit(rcar); } pci_epc_init_notify(ep->epc); @@ -590,7 +796,6 @@ static int rcar_gen4_add_dw_pcie_ep(struct rcar_gen4_pcie *rcar) static void rcar_gen4_remove_dw_pcie_ep(struct rcar_gen4_pcie *rcar) { dw_pcie_ep_deinit(&rcar->dw.ep); - rcar_gen4_pcie_ep_deinit(rcar); } /* Common */ @@ -625,7 +830,7 @@ static int rcar_gen4_pcie_probe(struct platform_device *pdev) err = rcar_gen4_pcie_prepare(rcar); if (err) - return err; + goto err_prepare; err = rcar_gen4_add_dw_pcie(rcar); if (err) @@ -636,6 +841,9 @@ static int rcar_gen4_pcie_probe(struct platform_device *pdev) err_unprepare: rcar_gen4_pcie_unprepare(rcar); +err_prepare: + reset_control_put(rcar->perst); + return err; } @@ -659,6 +867,7 @@ static void rcar_gen4_pcie_remove(struct platform_device *pdev) rcar_gen4_remove_dw_pcie(rcar); rcar_gen4_pcie_unprepare(rcar); + reset_control_put(rcar->perst); } static int r8a779f0_pcie_ltssm_control(struct rcar_gen4_pcie *rcar, bool enable) @@ -683,20 +892,26 @@ static int r8a779f0_pcie_ltssm_control(struct rcar_gen4_pcie *rcar, bool enable) return 0; } -static void rcar_gen4_pcie_additional_common_init(struct rcar_gen4_pcie *rcar) +static int rcar_gen5_pcie_ltssm_control(struct rcar_gen4_pcie *rcar, bool enable) { - struct dw_pcie *dw = &rcar->dw; u32 val; - val = dw_pcie_readl_dbi(dw, PCIE_PORT_LANE_SKEW); - val &= ~PORT_LANE_SKEW_INSERT_MASK; - if (dw->num_lanes < 4) - val |= BIT(6); - dw_pcie_writel_dbi(dw, PCIE_PORT_LANE_SKEW, val); + val = readl(rcar->base + PCIERSTCTRL1); + if (enable) { + val |= APP_LTSSM_ENABLE; + val &= ~APP_HOLD_PHY_RST; + } else { + val &= ~APP_LTSSM_ENABLE; + val |= APP_HOLD_PHY_RST; + } + writel(val, rcar->base + PCIERSTCTRL1); - val = readl(rcar->base + PCIEPWRMNGCTRL); - val |= APP_CLK_REQ_N | APP_CLK_PM_EN; - writel(val, rcar->base + PCIEPWRMNGCTRL); + if (enable) + phy_power_on(rcar->phy); + else + phy_power_off(rcar->phy); + + return 0; } static void rcar_gen4_pcie_phy_reg_update_bits(struct rcar_gen4_pcie *rcar, @@ -803,9 +1018,9 @@ static int rcar_gen4_pcie_ltssm_control(struct rcar_gen4_pcie *rcar, bool enable return 0; } - val = dw_pcie_readl_dbi(dw, PCIE_PORT_FORCE); - val |= PORT_FORCE_DO_DESKEW_FOR_SRIS; - dw_pcie_writel_dbi(dw, PCIE_PORT_FORCE, val); + val = dw_pcie_readl_dbi(dw, PORT_FORCE_LINK); + val |= PORT_FORCE_LINK_DDFS; + dw_pcie_writel_dbi(dw, PORT_FORCE_LINK, val); val = readl(rcar->base + PCIEMSR0); val |= APP_SRIS_MODE; @@ -849,28 +1064,66 @@ static int rcar_gen4_pcie_ltssm_control(struct rcar_gen4_pcie *rcar, bool enable return 0; } +static int rcar_gen4_pcie_suspend_noirq(struct device *dev) +{ + struct rcar_gen4_pcie *rcar = dev_get_drvdata(dev); + + if (rcar->drvdata->mode != DW_PCIE_RC_TYPE) + return 0; + + return dw_pcie_suspend_noirq(&rcar->dw); +} + +static int rcar_gen4_pcie_resume_noirq(struct device *dev) +{ + struct rcar_gen4_pcie *rcar = dev_get_drvdata(dev); + + if (rcar->drvdata->mode != DW_PCIE_RC_TYPE) + return 0; + + return dw_pcie_resume_noirq(&rcar->dw); +} + static struct rcar_gen4_pcie_drvdata drvdata_r8a779f0_pcie = { + .init = rcar_gen4_pcie_common_init, + .deinit = rcar_gen4_pcie_common_deinit, .ltssm_control = r8a779f0_pcie_ltssm_control, + .speed_control = rcar_gen4_pcie_speed_control, .mode = DW_PCIE_RC_TYPE, }; static struct rcar_gen4_pcie_drvdata drvdata_r8a779f0_pcie_ep = { + .init = rcar_gen4_pcie_common_init, + .deinit = rcar_gen4_pcie_common_deinit, .ltssm_control = r8a779f0_pcie_ltssm_control, + .speed_control = rcar_gen4_pcie_speed_control, .mode = DW_PCIE_EP_TYPE, }; static struct rcar_gen4_pcie_drvdata drvdata_rcar_gen4_pcie = { - .additional_common_init = rcar_gen4_pcie_additional_common_init, + .init = rcar_gen4_v4h_v4m_pcie_init, + .deinit = rcar_gen4_pcie_common_deinit, .ltssm_control = rcar_gen4_pcie_ltssm_control, + .speed_control = rcar_gen4_pcie_speed_control, .mode = DW_PCIE_RC_TYPE, }; static struct rcar_gen4_pcie_drvdata drvdata_rcar_gen4_pcie_ep = { - .additional_common_init = rcar_gen4_pcie_additional_common_init, + .init = rcar_gen4_v4h_v4m_pcie_init, + .deinit = rcar_gen4_pcie_common_deinit, .ltssm_control = rcar_gen4_pcie_ltssm_control, + .speed_control = rcar_gen4_pcie_speed_control, .mode = DW_PCIE_EP_TYPE, }; +static struct rcar_gen4_pcie_drvdata drvdata_rcar_gen5_pcie = { + .init = rcar_gen5_pcie_init, + .deinit = rcar_gen5_pcie_deinit, + .ltssm_control = rcar_gen5_pcie_ltssm_control, + .speed_control = rcar_gen5_pcie_speed_control, + .mode = DW_PCIE_RC_TYPE, +}; + static const struct of_device_id rcar_gen4_pcie_of_match[] = { { .compatible = "renesas,r8a779f0-pcie", @@ -888,14 +1141,23 @@ static const struct of_device_id rcar_gen4_pcie_of_match[] = { .compatible = "renesas,rcar-gen4-pcie-ep", .data = &drvdata_rcar_gen4_pcie_ep, }, + { + .compatible = "renesas,rcar-gen5-pcie4", + .data = &drvdata_rcar_gen5_pcie, + }, {}, }; MODULE_DEVICE_TABLE(of, rcar_gen4_pcie_of_match); +static DEFINE_NOIRQ_DEV_PM_OPS(rcar_gen4_pcie_pm_ops, + rcar_gen4_pcie_suspend_noirq, + rcar_gen4_pcie_resume_noirq); + static struct platform_driver rcar_gen4_pcie_driver = { .driver = { .name = "pcie-rcar-gen4", .of_match_table = rcar_gen4_pcie_of_match, + .pm = pm_sleep_ptr(&rcar_gen4_pcie_pm_ops), .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, .probe = rcar_gen4_pcie_probe, diff --git a/drivers/pci/controller/dwc/pcie-spacemit-k1.c b/drivers/pci/controller/dwc/pcie-spacemit-k1.c index 0564c46e2f48..92a71ea90ac3 100644 --- a/drivers/pci/controller/dwc/pcie-spacemit-k1.c +++ b/drivers/pci/controller/dwc/pcie-spacemit-k1.c @@ -12,9 +12,12 @@ #include <linux/err.h> #include <linux/gfp.h> #include <linux/mfd/syscon.h> +#include <linux/of.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> +#include <linux/pm_runtime.h> #include <linux/regmap.h> +#include <linux/regulator/consumer.h> #include <linux/reset.h> #include <linux/types.h> diff --git a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c index 2d737b49ea8f..352591dcef83 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194-acpi.c +++ b/drivers/pci/controller/dwc/pcie-tegra194-acpi.c @@ -39,8 +39,8 @@ static int tegra194_acpi_init(struct pci_config_window *cfg) static void atu_reg_write(struct tegra194_pcie_ecam *pcie_ecam, int index, u32 val, u32 reg) { - u32 offset = PCIE_ATU_UNROLL_BASE(PCIE_ATU_REGION_DIR_OB, index) + - PCIE_ATU_VIEWPORT_BASE; + u32 offset = ATU_UNROLL_BASE(ATU_REGION_DIR_OB, index) + + ATU_VIEWPORT_BASE; writel(val, pcie_ecam->iatu_base + offset + reg); } @@ -50,17 +50,17 @@ static void program_outbound_atu(struct tegra194_pcie_ecam *pcie_ecam, u64 pci_addr, u64 size) { atu_reg_write(pcie_ecam, index, lower_32_bits(cpu_addr), - PCIE_ATU_LOWER_BASE); + ATU_LOWER_BASE); atu_reg_write(pcie_ecam, index, upper_32_bits(cpu_addr), - PCIE_ATU_UPPER_BASE); + ATU_UPPER_BASE); atu_reg_write(pcie_ecam, index, lower_32_bits(pci_addr), - PCIE_ATU_LOWER_TARGET); + ATU_LOWER_TARGET); atu_reg_write(pcie_ecam, index, lower_32_bits(cpu_addr + size - 1), - PCIE_ATU_LIMIT); + ATU_LIMIT); atu_reg_write(pcie_ecam, index, upper_32_bits(pci_addr), - PCIE_ATU_UPPER_TARGET); - atu_reg_write(pcie_ecam, index, type, PCIE_ATU_REGION_CTRL1); - atu_reg_write(pcie_ecam, index, PCIE_ATU_ENABLE, PCIE_ATU_REGION_CTRL2); + ATU_UPPER_TARGET); + atu_reg_write(pcie_ecam, index, type, ATU_REGION_CTRL1); + atu_reg_write(pcie_ecam, index, ATU_ENABLE, ATU_REGION_CTRL2); } static void __iomem *tegra194_map_bus(struct pci_bus *bus, @@ -81,8 +81,8 @@ static void __iomem *tegra194_map_bus(struct pci_bus *bus, return NULL; } - busdev = PCIE_ATU_BUS(bus->number) | PCIE_ATU_DEV(PCI_SLOT(devfn)) | - PCIE_ATU_FUNC(PCI_FUNC(devfn)); + busdev = ATU_BUS(bus->number) | ATU_DEV(PCI_SLOT(devfn)) | + ATU_FUNC(PCI_FUNC(devfn)); if (bus->parent->number == cfg->busr.start) { if (PCI_SLOT(devfn) == 0) diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 63a0f56cad8e..26e2ac5c5869 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -27,6 +27,7 @@ #include <linux/platform_device.h> #include <linux/pm_runtime.h> #include <linux/random.h> +#include <linux/regulator/consumer.h> #include <linux/reset.h> #include <linux/resource.h> #include <linux/types.h> @@ -385,9 +386,9 @@ static irqreturn_t tegra_pcie_rp_irq_handler(int irq, void *arg) val |= APPL_CAR_RESET_OVRD_CYA_OVERRIDE_CORE_RST_N; appl_writel(pcie, val, APPL_CAR_RESET_OVRD); - val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - val |= PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + val = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + val |= SPEED_CHANGE; + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, val); } } @@ -621,15 +622,15 @@ static inline u32 event_counter_prog(struct tegra_pcie_dw *pcie, u32 event) u32 val; val = dw_pcie_readl_dbi(&pcie->pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_CONTROL); - val &= ~(EVENT_COUNTER_EVENT_SEL_MASK << EVENT_COUNTER_EVENT_SEL_SHIFT); - val |= EVENT_COUNTER_GROUP_5 << EVENT_COUNTER_GROUP_SEL_SHIFT; - val |= event << EVENT_COUNTER_EVENT_SEL_SHIFT; - val |= EVENT_COUNTER_ENABLE_ALL << EVENT_COUNTER_ENABLE_SHIFT; + RAS_DES_EVENT_COUNTER_CONTROL); + val &= ~EVENT_COUNTER_EVENT_SEL_MASK; + val |= FIELD_PREP(EVENT_COUNTER_GROUP_SEL_MASK, EVENT_COUNTER_GROUP_5); + val |= FIELD_PREP(EVENT_COUNTER_EVENT_SEL_MASK, event); + val |= FIELD_PREP(EVENT_COUNTER_ENABLE_MASK, EVENT_COUNTER_ENABLE_ALL); dw_pcie_writel_dbi(&pcie->pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_CONTROL, val); + RAS_DES_EVENT_COUNTER_CONTROL, val); val = dw_pcie_readl_dbi(&pcie->pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_DATA); + RAS_DES_EVENT_COUNTER_DATA); return val; } @@ -657,14 +658,14 @@ static int aspm_state_cnt(struct seq_file *s, void *data) /* Clear all counters */ dw_pcie_writel_dbi(&pcie->pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_CONTROL, + RAS_DES_EVENT_COUNTER_CONTROL, EVENT_COUNTER_ALL_CLEAR); /* Re-enable counting */ - val = EVENT_COUNTER_ENABLE_ALL << EVENT_COUNTER_ENABLE_SHIFT; - val |= EVENT_COUNTER_GROUP_5 << EVENT_COUNTER_GROUP_SEL_SHIFT; + val = FIELD_PREP(EVENT_COUNTER_ENABLE_MASK, EVENT_COUNTER_ENABLE_ALL); + val |= FIELD_PREP(EVENT_COUNTER_GROUP_SEL_MASK, EVENT_COUNTER_GROUP_5); dw_pcie_writel_dbi(&pcie->pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_CONTROL, val); + RAS_DES_EVENT_COUNTER_CONTROL, val); return 0; } @@ -680,10 +681,10 @@ static void init_host_aspm(struct tegra_pcie_dw *pcie) PCI_EXT_CAP_ID_VNDR); /* Enable ASPM counters */ - val = EVENT_COUNTER_ENABLE_ALL << EVENT_COUNTER_ENABLE_SHIFT; - val |= EVENT_COUNTER_GROUP_5 << EVENT_COUNTER_GROUP_SEL_SHIFT; + val = FIELD_PREP(EVENT_COUNTER_ENABLE_MASK, EVENT_COUNTER_ENABLE_ALL); + val |= FIELD_PREP(EVENT_COUNTER_GROUP_SEL_MASK, EVENT_COUNTER_GROUP_5); dw_pcie_writel_dbi(pci, pcie->ras_des_cap + - PCIE_RAS_DES_EVENT_COUNTER_CONTROL, val); + RAS_DES_EVENT_COUNTER_CONTROL, val); /* Program T_cmrt and T_pwr_on values */ val = dw_pcie_readl_dbi(pci, l1ss + PCI_L1SS_CAP); @@ -713,13 +714,13 @@ static void init_host_aspm(struct tegra_pcie_dw *pcie) } /* Program L0s and L1 entrance latencies */ - val = dw_pcie_readl_dbi(pci, PCIE_PORT_AFR); - val &= ~PORT_AFR_L0S_ENTRANCE_LAT_MASK; - val |= (pcie->aspm_l0s_enter_lat << PORT_AFR_L0S_ENTRANCE_LAT_SHIFT); - val &= ~PORT_AFR_L1_ENTRANCE_LAT_MASK; - val |= (pcie->aspm_l1_enter_lat << PORT_AFR_L1_ENTRANCE_LAT_SHIFT); - val |= PORT_AFR_ENTER_ASPM; - dw_pcie_writel_dbi(pci, PCIE_PORT_AFR, val); + val = dw_pcie_readl_dbi(pci, PORT_AFAC); + val &= ~PORT_AFAC_L0S_ELM; + val |= FIELD_PREP(PORT_AFAC_L0S_ELM, pcie->aspm_l0s_enter_lat); + val &= ~PORT_AFAC_L1_ELM; + val |= FIELD_PREP(PORT_AFAC_L1_ELM, pcie->aspm_l1_enter_lat); + val |= PORT_AFAC_ENTER_ASPM; + dw_pcie_writel_dbi(pci, PORT_AFAC, val); } static void init_debugfs(struct tegra_pcie_dw *pcie) @@ -870,29 +871,29 @@ static void config_gen3_gen4_eq_presets(struct tegra_pcie_dw *pcie) dw_pcie_writeb_dbi(pci, offset + i, val); } - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_RATE_SHADOW_SEL_MASK; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_RATE_SHADOW_SEL_MASK; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); - val = dw_pcie_readl_dbi(pci, GEN3_EQ_CONTROL_OFF); - FIELD_MODIFY(GEN3_EQ_CONTROL_OFF_PSET_REQ_VEC, &val, 0x3ff); - val &= ~GEN3_EQ_CONTROL_OFF_FB_MODE; - dw_pcie_writel_dbi(pci, GEN3_EQ_CONTROL_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_EQ_CTRL); + FIELD_MODIFY(GEN3_EQ_CTRL_PSET_REQ_VEC, &val, 0x3ff); + val &= ~GEN3_EQ_CTRL_FB_MODE; + dw_pcie_writel_dbi(pci, GEN3_EQ_CTRL, val); - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_RATE_SHADOW_SEL_MASK; - val |= (0x1 << GEN3_RELATED_OFF_RATE_SHADOW_SEL_SHIFT); - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_RATE_SHADOW_SEL_MASK; + val |= FIELD_PREP(GEN3_CTRL_RATE_SHADOW_SEL_MASK, 0x1); + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); - val = dw_pcie_readl_dbi(pci, GEN3_EQ_CONTROL_OFF); - FIELD_MODIFY(GEN3_EQ_CONTROL_OFF_PSET_REQ_VEC, &val, + val = dw_pcie_readl_dbi(pci, GEN3_EQ_CTRL); + FIELD_MODIFY(GEN3_EQ_CTRL_PSET_REQ_VEC, &val, pcie->of_data->gen4_preset_vec); - val &= ~GEN3_EQ_CONTROL_OFF_FB_MODE; - dw_pcie_writel_dbi(pci, GEN3_EQ_CONTROL_OFF, val); + val &= ~GEN3_EQ_CTRL_FB_MODE; + dw_pcie_writel_dbi(pci, GEN3_EQ_CTRL, val); - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_RATE_SHADOW_SEL_MASK; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_RATE_SHADOW_SEL_MASK; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); } static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp) @@ -940,9 +941,9 @@ static int tegra_pcie_dw_host_init(struct dw_pcie_rp *pp) init_host_aspm(pcie); if (!pcie->of_data->has_l1ss_exit_fix) { - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_GEN3_ZRXDC_NONCOMPL; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); } if (pcie->update_fc_fixup) { @@ -1869,9 +1870,9 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) pci_epc_deinit_notify(pcie->pci.ep.epc); dw_pcie_ep_cleanup(&pcie->pci.ep); - val = dw_pcie_readl_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL); - val &= ~PORT_LOGIC_SPEED_CHANGE; - dw_pcie_writel_dbi(pci, PCIE_LINK_WIDTH_SPEED_CONTROL, val); + val = dw_pcie_readl_dbi(pci, LINK_WIDTH_SPEED_CTRL); + val &= ~SPEED_CHANGE; + dw_pcie_writel_dbi(pci, LINK_WIDTH_SPEED_CTRL, val); if (pcie->update_fc_fixup) { val = dw_pcie_readl_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF); @@ -1884,9 +1885,9 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) init_host_aspm(pcie); if (!pcie->of_data->has_l1ss_exit_fix) { - val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF); - val &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL; - dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); + val = dw_pcie_readl_dbi(pci, GEN3_CTRL); + val &= ~GEN3_CTRL_GEN3_ZRXDC_NONCOMPL; + dw_pcie_writel_dbi(pci, GEN3_CTRL, val); } pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci, diff --git a/drivers/pci/controller/dwc/pcie-ultrarisc.c b/drivers/pci/controller/dwc/pcie-ultrarisc.c index 51e89180490d..523f454c6c52 100644 --- a/drivers/pci/controller/dwc/pcie-ultrarisc.c +++ b/drivers/pci/controller/dwc/pcie-ultrarisc.c @@ -52,18 +52,18 @@ static int ultrarisc_pcie_host_init(struct dw_pcie_rp *pp) val &= ~FAST_LINK_MODE; dw_pcie_writel_dbi(pci, PCIE_CUS_CORE, val); - val = dw_pcie_readl_dbi(pci, PCIE_TIMER_CTRL_MAX_FUNC_NUM); + val = dw_pcie_readl_dbi(pci, TIMER_CTRL_MAX_FUNC_NUM); FIELD_MODIFY(PORT_FLT_SF_MASK, &val, PORT_FLT_SF_VAL_64); - dw_pcie_writel_dbi(pci, PCIE_TIMER_CTRL_MAX_FUNC_NUM, val); + dw_pcie_writel_dbi(pci, TIMER_CTRL_MAX_FUNC_NUM, val); cap_exp = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP); val = dw_pcie_readl_dbi(pci, cap_exp + PCI_EXP_LNKCTL2); FIELD_MODIFY(PCI_EXP_LNKCTL2_TLS, &val, PCI_EXP_LNKCTL2_TLS_16_0GT); dw_pcie_writel_dbi(pci, cap_exp + PCI_EXP_LNKCTL2, val); - val = dw_pcie_readl_dbi(pci, PCIE_PORT_FORCE); - FIELD_MODIFY(PORT_LINK_NUM_MASK, &val, 0); - dw_pcie_writel_dbi(pci, PCIE_PORT_FORCE, val); + val = dw_pcie_readl_dbi(pci, PORT_FORCE_LINK); + FIELD_MODIFY(PORT_FORCE_LINK_NUM_MASK, &val, 0); + dw_pcie_writel_dbi(pci, PORT_FORCE_LINK, val); val = dw_pcie_readl_dbi(pci, cap_exp + PCI_EXP_DEVCTL2); FIELD_MODIFY(PCI_EXP_DEVCTL2_COMP_TIMEOUT, &val, @@ -154,6 +154,12 @@ static int ultrarisc_pcie_probe(struct platform_device *pdev) pp = &pci->pp; + /* + * DP1000 routes MEM transactions without outbound iATU windows, which + * avoids the 16-window/4GB-per-window aperture limit. + */ + pp->bypass_ob_mem_iatu = true; + platform_set_drvdata(pdev, ultra); ret = ultrarisc_pcie_init_clks(ultra); diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index ecb81ac73019..2bbdb73cfb45 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -1864,28 +1864,25 @@ static int advk_pcie_probe(struct platform_device *pdev) ret = advk_sw_pci_bridge_init(pcie); if (ret) { dev_err(dev, "Failed to register emulated root PCI bridge\n"); - return ret; + goto err_disable_phy; } ret = advk_pcie_init_irq_domain(pcie); if (ret) { dev_err(dev, "Failed to initialize irq\n"); - return ret; + goto err_cleanup_bridge; } ret = advk_pcie_init_msi_irq_domain(pcie); if (ret) { dev_err(dev, "Failed to initialize irq\n"); - advk_pcie_remove_irq_domain(pcie); - return ret; + goto err_remove_irq_domain; } ret = advk_pcie_init_rp_irq_domain(pcie); if (ret) { dev_err(dev, "Failed to initialize irq\n"); - advk_pcie_remove_msi_irq_domain(pcie); - advk_pcie_remove_irq_domain(pcie); - return ret; + goto err_remove_msi_irq_domain; } bridge->sysdata = pcie; @@ -1893,14 +1890,22 @@ static int advk_pcie_probe(struct platform_device *pdev) bridge->map_irq = advk_pcie_map_irq; ret = pci_host_probe(bridge); - if (ret < 0) { - advk_pcie_remove_rp_irq_domain(pcie); - advk_pcie_remove_msi_irq_domain(pcie); - advk_pcie_remove_irq_domain(pcie); - return ret; - } + if (ret < 0) + goto err_remove_rp_irq_domain; return 0; + +err_remove_rp_irq_domain: + advk_pcie_remove_rp_irq_domain(pcie); +err_remove_msi_irq_domain: + advk_pcie_remove_msi_irq_domain(pcie); +err_remove_irq_domain: + advk_pcie_remove_irq_domain(pcie); +err_cleanup_bridge: + pci_bridge_emul_cleanup(&pcie->bridge); +err_disable_phy: + advk_pcie_disable_phy(pcie); + return ret; } static void advk_pcie_remove(struct platform_device *pdev) diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c index 69a9d77039e8..cce1c1c0da58 100644 --- a/drivers/pci/controller/pci-mvebu.c +++ b/drivers/pci/controller/pci-mvebu.c @@ -1235,6 +1235,11 @@ static void mvebu_pcie_port_clk_put(void *data) clk_put(port->clk); } +static void mvebu_pcie_port_of_node_put(void *data) +{ + of_node_put(data); +} + static int mvebu_pcie_parse_port(struct mvebu_pcie *pcie, struct mvebu_pcie_port *port, struct device_node *child) { @@ -1451,7 +1456,6 @@ static int mvebu_pcie_probe(struct platform_device *pdev) struct mvebu_pcie *pcie; struct pci_host_bridge *bridge; struct device_node *np = dev->of_node; - struct device_node *child; int num, i, ret; bridge = devm_pci_alloc_host_bridge(dev, sizeof(struct mvebu_pcie)); @@ -1473,18 +1477,20 @@ static int mvebu_pcie_probe(struct platform_device *pdev) return -ENOMEM; i = 0; - for_each_available_child_of_node(np, child) { + for_each_available_child_of_node_scoped(np, child) { struct mvebu_pcie_port *port = &pcie->ports[i]; ret = mvebu_pcie_parse_port(pcie, port, child); - if (ret < 0) { - of_node_put(child); + if (ret < 0) return ret; - } else if (ret == 0) { + else if (ret == 0) continue; - } - port->dn = child; + port->dn = of_node_get(child); + ret = devm_add_action_or_reset(dev, mvebu_pcie_port_of_node_put, + child); + if (ret) + return ret; i++; } pcie->nports = i; @@ -1492,6 +1498,7 @@ static int mvebu_pcie_probe(struct platform_device *pdev) for (i = 0; i < pcie->nports; i++) { struct mvebu_pcie_port *port = &pcie->ports[i]; int irq = port->intx_irq; + struct device_node *child; child = port->dn; if (!child) diff --git a/drivers/pci/controller/pci-tegra.c b/drivers/pci/controller/pci-tegra.c index 59b56446c507..96bd522fa67b 100644 --- a/drivers/pci/controller/pci-tegra.c +++ b/drivers/pci/controller/pci-tegra.c @@ -36,6 +36,7 @@ #include <linux/phy/phy.h> #include <linux/pinctrl/consumer.h> #include <linux/platform_device.h> +#include <linux/pm_runtime.h> #include <linux/reset.h> #include <linux/sizes.h> #include <linux/slab.h> diff --git a/drivers/pci/controller/pci-xgene.c b/drivers/pci/controller/pci-xgene.c index 83c9a2930eec..a5b8bbc04a22 100644 --- a/drivers/pci/controller/pci-xgene.c +++ b/drivers/pci/controller/pci-xgene.c @@ -320,18 +320,11 @@ static void xgene_pcie_linkup(struct xgene_pcie *port, static int xgene_pcie_init_port(struct xgene_pcie *port) { struct device *dev = port->dev; - int rc; - port->clk = clk_get(dev, NULL); + port->clk = devm_clk_get_enabled(dev, NULL); if (IS_ERR(port->clk)) { dev_err(dev, "clock not available\n"); - return -ENODEV; - } - - rc = clk_prepare_enable(port->clk); - if (rc) { - dev_err(dev, "clock enable failed\n"); - return rc; + return PTR_ERR(port->clk); } return 0; diff --git a/drivers/pci/controller/pcie-aspeed.c b/drivers/pci/controller/pcie-aspeed.c index 74b03a55643f..ecb7562f84c0 100644 --- a/drivers/pci/controller/pcie-aspeed.c +++ b/drivers/pci/controller/pcie-aspeed.c @@ -761,16 +761,21 @@ static int aspeed_pcie_port_init(struct aspeed_pcie_port *port) port->slot); ret = phy_init(port->phy); - if (ret) + if (ret) { + clk_disable_unprepare(port->clk); return dev_err_probe(dev, ret, "failed to init phy pcie for slot (%d)\n", port->slot); + } ret = phy_set_mode_ext(port->phy, PHY_MODE_PCIE, PHY_MODE_PCIE_RC); - if (ret) + if (ret) { + phy_exit(port->phy); + clk_disable_unprepare(port->clk); return dev_err_probe(dev, ret, "failed to set phy mode for slot (%d)\n", port->slot); + } reset_control_deassert(port->perst); msleep(PCIE_RESET_CONFIG_WAIT_MS); diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index 8a0c353d2abf..26d1305f5e70 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -277,6 +277,9 @@ struct inbound_win { u64 cpu_addr; }; +/* QUIRKS */ +#define BQUIRK(pcie, quirk) ((pcie)->cfg->quirks & CFG_QUIRK_ ## quirk) + /* * The RESCAL block is tied to PCIe controller #1, regardless of the number of * controllers, and turning off PCIe controller #1 prevents access to the RESCAL @@ -285,17 +288,38 @@ struct inbound_win { * or a hang (AXI). */ #define CFG_QUIRK_AVOID_BRIDGE_SHUTDOWN BIT(0) +/* The outbound windows must be within the 0-4GB region */ +#define CFG_QUIRK_OB_WIN_32BIT_ADDR BIT(1) +/* Each of the outbound windows size must be <= SZ_128M */ +#define CFG_QUIRK_OB_WIN_MAXSZ_128MB BIT(2) +/* PCIe HW can only access config space with 32bit R/W */ +#define CFG_QUIRK_32BIT_PCI_OPS BIT(3) +/* PCIe HW does not have an internal bus timer */ +#define CFG_QUIRK_NO_RGR1_TIMER BIT(4) +/* PCIe PERST# must be asserted before internal bridge turned on */ +#define CFG_QUIRK_EARLY_PERST_ASSERT BIT(5) +/* PCIe SW PERST behavior only works up to a specific HW version */ +#define CFG_QUIRK_PERST_PCIE_REV_CUTOFF BIT(6) + +/* FLAGS */ +#define BFLAG(pcie, flag) ((pcie)->cfg->flags & CFG_FLG_ ## flag) + +/* The PCIe HW registers have phy control of a "RESCAL" reset block */ +#define CFG_FLG_HAS_PHY BIT(0) +/* The SoC PCIe HW can dump to the console PCIe error info */ +#define CFG_FLG_HAS_ERR_REPORT BIT(1) +/* SoC uses a Broadcom variant of the MIPS-1 ISA */ +#define CFG_FLG_IS_BMIPS BIT(2) struct pcie_cfg_data { const int *offsets; - const enum pcie_soc_base soc_base; - const bool has_phy; - const u32 quirks; + enum pcie_soc_base soc_base; + u32 flags; + u32 quirks; u8 num_inbound_wins; int (*perst_set)(struct brcm_pcie *pcie, u32 val); int (*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val); int (*post_setup)(struct brcm_pcie *pcie); - bool has_err_report; }; struct subdev_regulators { @@ -346,24 +370,19 @@ struct brcm_pcie { spinlock_t bridge_lock; }; -static inline bool is_bmips(const struct brcm_pcie *pcie) -{ - return pcie->cfg->soc_base == BCM7435 || pcie->cfg->soc_base == BCM7425; -} - static int brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val) { unsigned long flags; int ret; - if (pcie->cfg->has_err_report) + if (BFLAG(pcie, HAS_ERR_REPORT)) spin_lock_irqsave(&pcie->bridge_lock, flags); ret = pcie->cfg->bridge_sw_init_set(pcie, val); /* If we fail, assume the bridge is in reset (off) */ pcie->bridge_in_reset = ret ? true : val; - if (pcie->cfg->has_err_report) + if (BFLAG(pcie, HAS_ERR_REPORT)) spin_unlock_irqrestore(&pcie->bridge_lock, flags); return ret; @@ -508,7 +527,7 @@ static void brcm_pcie_set_outbound_win(struct brcm_pcie *pcie, PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_LIMIT_MASK); writel(tmp, pcie->base + PCIE_MEM_WIN0_BASE_LIMIT(win)); - if (is_bmips(pcie)) + if (BQUIRK(pcie, OB_WIN_32BIT_ADDR)) return; /* Write the cpu & limit addr upper bits */ @@ -1144,8 +1163,8 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) if (ret) return ret; - /* Ensure that PERST# is asserted; some bootloaders may deassert it. */ - if (pcie->cfg->soc_base == BCM2711) { + if (BQUIRK(pcie, EARLY_PERST_ASSERT)) { + /* Ensure that PERST# is asserted; some bootloaders may deassert it. */ ret = pcie->cfg->perst_set(pcie, 1); if (ret) { pcie->cfg->bridge_sw_init_set(pcie, 0); @@ -1161,7 +1180,7 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) return ret; tmp = readl(base + HARD_DEBUG(pcie)); - if (is_bmips(pcie)) + if (BFLAG(pcie, IS_BMIPS)) tmp &= ~PCIE_BMIPS_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK; else tmp &= ~PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK; @@ -1174,7 +1193,7 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) * is encoded as 0=128, 1=256, 2=512, 3=Rsvd, for BCM7278 it * is encoded as 0=Rsvd, 1=128, 2=256, 3=512. */ - if (is_bmips(pcie)) + if (BFLAG(pcie, IS_BMIPS)) burst = 0x1; /* 256 bytes */ else if (pcie->cfg->soc_base == BCM2711) burst = 0x0; /* 128 bytes */ @@ -1281,11 +1300,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) return -EINVAL; } - if (is_bmips(pcie)) { + if (BQUIRK(pcie, OB_WIN_MAXSZ_128MB)) { u64 start = res->start; unsigned int j, nwins = resource_size(res) / SZ_128M; - /* bmips PCIe outbound windows have a 128MB max size */ + /* PCIe outbound windows have a 128MB max size */ if (nwins > BRCM_NUM_PCIE_OUT_WINS) nwins = BRCM_NUM_PCIE_OUT_WINS; for (j = 0; j < nwins; j++, start += SZ_128M) @@ -1326,8 +1345,8 @@ static void brcm_extend_rbus_timeout(struct brcm_pcie *pcie) const unsigned int REG_OFFSET = PCIE_RGR1_SW_INIT_1(pcie) - 8; u32 timeout_us = 4000000; /* 4 seconds, our setting for L1SS */ - /* 7712 does not have this (RGR1) timer */ - if (pcie->cfg->soc_base == BCM7712) + /* Don't do access if there is no timer */ + if (BQUIRK(pcie, NO_RGR1_TIMER)) return; /* Each unit in timeout register is 1/216,000,000 seconds */ @@ -1589,12 +1608,12 @@ static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start) static inline int brcm_phy_start(struct brcm_pcie *pcie) { - return pcie->cfg->has_phy ? brcm_phy_cntl(pcie, 1) : 0; + return BFLAG(pcie, HAS_PHY) ? brcm_phy_cntl(pcie, 1) : 0; } static inline int brcm_phy_stop(struct brcm_pcie *pcie) { - return pcie->cfg->has_phy ? brcm_phy_cntl(pcie, 0) : 0; + return BFLAG(pcie, HAS_PHY) ? brcm_phy_cntl(pcie, 0) : 0; } static int brcm_pcie_turn_off(struct brcm_pcie *pcie) @@ -1619,7 +1638,7 @@ static int brcm_pcie_turn_off(struct brcm_pcie *pcie) u32p_replace_bits(&tmp, 1, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK); writel(tmp, base + HARD_DEBUG(pcie)); - if (!(pcie->cfg->quirks & CFG_QUIRK_AVOID_BRIDGE_SHUTDOWN)) + if (!(BQUIRK(pcie, AVOID_BRIDGE_SHUTDOWN))) /* Shutdown PCIe bridge */ ret = brcm_pcie_bridge_sw_init_set(pcie, 1); @@ -1898,7 +1917,7 @@ static void brcm_pcie_remove(struct platform_device *pdev) pci_stop_root_bus(bridge->bus); pci_remove_root_bus(bridge->bus); pci_unlock_rescan_remove(); - if (pcie->cfg->has_err_report) + if (BFLAG(pcie, HAS_ERR_REPORT)) brcm_unregister_die_notifiers(pcie); __brcm_pcie_remove(pcie); @@ -1950,6 +1969,7 @@ static const struct pcie_cfg_data bcm2711_cfg = { .perst_set = brcm_pcie_perst_set_generic, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .num_inbound_wins = 3, + .quirks = CFG_QUIRK_EARLY_PERST_ASSERT, }; static const struct pcie_cfg_data bcm2712_cfg = { @@ -1958,7 +1978,8 @@ static const struct pcie_cfg_data bcm2712_cfg = { .perst_set = brcm_pcie_perst_set_7278, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .post_setup = brcm_pcie_post_setup_bcm2712, - .quirks = CFG_QUIRK_AVOID_BRIDGE_SHUTDOWN, + .quirks = CFG_QUIRK_AVOID_BRIDGE_SHUTDOWN | + CFG_QUIRK_NO_RGR1_TIMER, .num_inbound_wins = 10, }; @@ -1968,6 +1989,7 @@ static const struct pcie_cfg_data bcm4908_cfg = { .perst_set = brcm_pcie_perst_set_4908, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .num_inbound_wins = 3, + .quirks = CFG_QUIRK_PERST_PCIE_REV_CUTOFF, }; static const struct pcie_cfg_data bcm7278_cfg = { @@ -1984,6 +2006,9 @@ static const struct pcie_cfg_data bcm7425_cfg = { .perst_set = brcm_pcie_perst_set_generic, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .num_inbound_wins = 3, + .quirks = CFG_QUIRK_OB_WIN_32BIT_ADDR + | CFG_QUIRK_OB_WIN_MAXSZ_128MB | CFG_QUIRK_32BIT_PCI_OPS, + .flags = CFG_FLG_IS_BMIPS, }; static const struct pcie_cfg_data bcm7435_cfg = { @@ -1992,6 +2017,9 @@ static const struct pcie_cfg_data bcm7435_cfg = { .perst_set = brcm_pcie_perst_set_generic, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .num_inbound_wins = 3, + .quirks = CFG_QUIRK_OB_WIN_32BIT_ADDR + | CFG_QUIRK_OB_WIN_MAXSZ_128MB, + .flags = CFG_FLG_IS_BMIPS, }; static const struct pcie_cfg_data bcm7216_cfg = { @@ -1999,9 +2027,8 @@ static const struct pcie_cfg_data bcm7216_cfg = { .soc_base = BCM7278, .perst_set = brcm_pcie_perst_set_7278, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_7278, - .has_phy = true, + .flags = CFG_FLG_HAS_PHY | CFG_FLG_HAS_ERR_REPORT, .num_inbound_wins = 3, - .has_err_report = true, }; static const struct pcie_cfg_data bcm7712_cfg = { @@ -2010,6 +2037,7 @@ static const struct pcie_cfg_data bcm7712_cfg = { .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic, .soc_base = BCM7712, .num_inbound_wins = 10, + .quirks = CFG_QUIRK_NO_RGR1_TIMER, }; static const struct of_device_id brcm_pcie_match[] = { @@ -2034,7 +2062,7 @@ static struct pci_ops brcm_pcie_ops = { .remove_bus = brcm_pcie_remove_bus, }; -static struct pci_ops brcm7425_pcie_ops = { +static struct pci_ops brcm_pcie_ops32 = { .map_bus = brcm7425_pcie_map_bus, .read = pci_generic_config_read32, .write = pci_generic_config_write32, @@ -2143,7 +2171,7 @@ static int brcm_pcie_probe(struct platform_device *pdev) goto fail; pcie->hw_rev = readl(pcie->base + PCIE_MISC_REVISION); - if (pcie->cfg->soc_base == BCM4908 && + if (BQUIRK(pcie, PERST_PCIE_REV_CUTOFF) && pcie->hw_rev >= BRCM_PCIE_HW_REV_3_20) { dev_err(pcie->dev, "hardware revision with unsupported PERST# setup\n"); ret = -ENODEV; @@ -2164,8 +2192,8 @@ static int brcm_pcie_probe(struct platform_device *pdev) } } - bridge->ops = pcie->cfg->soc_base == BCM7425 ? - &brcm7425_pcie_ops : &brcm_pcie_ops; + bridge->ops = BQUIRK(pcie, 32BIT_PCI_OPS) + ? &brcm_pcie_ops32 : &brcm_pcie_ops; bridge->sysdata = pcie; platform_set_drvdata(pdev, pcie); @@ -2179,7 +2207,7 @@ static int brcm_pcie_probe(struct platform_device *pdev) return ret; } - if (pcie->cfg->has_err_report) { + if (BFLAG(pcie, HAS_ERR_REPORT)) { spin_lock_init(&pcie->bridge_lock); brcm_register_die_notifiers(pcie); } diff --git a/drivers/pci/controller/pcie-iproc-bcma.c b/drivers/pci/controller/pcie-iproc-bcma.c index 593418c2bc3a..1e258cd8fed4 100644 --- a/drivers/pci/controller/pcie-iproc-bcma.c +++ b/drivers/pci/controller/pcie-iproc-bcma.c @@ -11,7 +11,9 @@ #include <linux/phy/phy.h> #include <linux/bcma/bcma.h> #include <linux/ioport.h> +#include <linux/of_address.h> +#include "../pci.h" #include "pcie-iproc.h" @@ -31,6 +33,29 @@ static int iproc_bcma_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin) return bcma_core_irq(bdev, 5); } +static void iproc_bcma_pcie_check_dt_window(struct iproc_pcie *pcie) +{ + struct device_node *np = pcie->dev->of_node; + struct of_pci_range_parser parser; + struct of_pci_range range; + struct resource res; + + if (!np || of_pci_range_parser_init(&parser, np)) + return; + + for_each_of_pci_range(&parser, &range) { + if ((range.flags & IORESOURCE_TYPE_BITS) != IORESOURCE_MEM) + continue; + + if (of_pci_range_to_resource(&range, np, &res)) + continue; + + if (res.start != pcie->mem.start || res.end != pcie->mem.end) + dev_warn(pcie->dev, "DT window %pR does not match EROM window %pR, using EROM\n", + &res, &pcie->mem); + } +} + static int iproc_bcma_pcie_probe(struct bcma_device *bdev) { struct device *dev = &bdev->dev; @@ -38,10 +63,20 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) struct pci_host_bridge *bridge; int ret; - bridge = devm_pci_alloc_host_bridge(dev, sizeof(*pcie)); + bridge = pci_alloc_host_bridge(sizeof(*pcie)); if (!bridge) return -ENOMEM; + bridge->dev.parent = dev; + + ret = devm_add_action_or_reset(dev, devm_pci_alloc_host_bridge_release, + bridge); + if (ret) + return -ENOMEM; + + bridge->swizzle_irq = pci_common_swizzle; + bridge->map_irq = iproc_bcma_pcie_map_irq; + pcie = pci_host_bridge_priv(bridge); pcie->dev = dev; @@ -59,13 +94,12 @@ static int iproc_bcma_pcie_probe(struct bcma_device *bdev) pcie->mem.end = bdev->addr_s[0] + SZ_128M - 1; pcie->mem.name = "PCIe MEM space"; pcie->mem.flags = IORESOURCE_MEM; + iproc_bcma_pcie_check_dt_window(pcie); pci_add_resource(&bridge->windows, &pcie->mem); ret = devm_request_pci_bus_resources(dev, &bridge->windows); if (ret) return ret; - bridge->map_irq = iproc_bcma_pcie_map_irq; - bcma_set_drvdata(bdev, pcie); return iproc_pcie_setup(pcie, &bridge->windows); diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index bcbd08c82a5c..e2fb6e372ede 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -7,6 +7,8 @@ */ #include <linux/bitfield.h> +#include <linux/bitops.h> +#include <linux/bits.h> #include <linux/clk.h> #include <linux/clk-provider.h> #include <linux/delay.h> @@ -334,10 +336,10 @@ static int mtk_pcie_set_trans_table(struct mtk_gen3_pcie *pcie, while (remaining && (*num < PCIE_MAX_TRANS_TABLES)) { /* Table size needs to be a power of 2 */ - table_size = BIT(fls(remaining) - 1); + table_size = BIT_ULL(fls64(remaining) - 1); if (cpu_addr > 0) { - addr_align = BIT(ffs(cpu_addr) - 1); + addr_align = BIT_ULL(__ffs64(cpu_addr)); table_size = min(table_size, addr_align); } @@ -349,7 +351,8 @@ static int mtk_pcie_set_trans_table(struct mtk_gen3_pcie *pcie, } table = pcie->base + PCIE_TRANS_TABLE_BASE_REG + *num * PCIE_ATR_TLB_SET_OFFSET; - writel_relaxed(lower_32_bits(cpu_addr) | PCIE_ATR_SIZE(fls(table_size) - 1), table); + writel_relaxed(lower_32_bits(cpu_addr) | + PCIE_ATR_SIZE(fls64(table_size) - 1), table); writel_relaxed(upper_32_bits(cpu_addr), table + PCIE_ATR_SRC_ADDR_MSB_OFFSET); writel_relaxed(lower_32_bits(pci_addr), table + PCIE_ATR_TRSL_ADDR_LSB_OFFSET); writel_relaxed(upper_32_bits(pci_addr), table + PCIE_ATR_TRSL_ADDR_MSB_OFFSET); @@ -375,9 +378,11 @@ static int mtk_pcie_set_trans_table(struct mtk_gen3_pcie *pcie, (*num)++; } - if (remaining) - dev_warn(pcie->dev, "not enough translate table for addr: %#llx, limited to [%d]\n", + if (remaining) { + dev_err(pcie->dev, "not enough translate table for addr: %#llx, limited to [%d]\n", (unsigned long long)cpu_addr, PCIE_MAX_TRANS_TABLES); + return -ENOMEM; + } return 0; } diff --git a/drivers/pci/controller/pcie-mediatek.c b/drivers/pci/controller/pcie-mediatek.c index a60d1ae076f8..3a3e91cd0c21 100644 --- a/drivers/pci/controller/pcie-mediatek.c +++ b/drivers/pci/controller/pcie-mediatek.c @@ -583,9 +583,9 @@ static int mtk_pcie_init_irq_domain(struct mtk_pcie_port *port, int ret; /* Setup INTx */ - pcie_intc_node = of_get_next_child(node, NULL); + pcie_intc_node = of_get_child_by_name(node, "interrupt-controller"); if (!pcie_intc_node) { - dev_err(dev, "no PCIe Intc node found\n"); + dev_err(dev, "no PCIe INTx interrupt-controller node found\n"); return -ENODEV; } diff --git a/drivers/pci/controller/pcie-rockchip-ep.c b/drivers/pci/controller/pcie-rockchip-ep.c index 799461335762..9ebc227a1ef8 100644 --- a/drivers/pci/controller/pcie-rockchip-ep.c +++ b/drivers/pci/controller/pcie-rockchip-ep.c @@ -553,19 +553,6 @@ static void rockchip_pcie_ep_link_training(struct work_struct *work) if (ret) goto again; - /* - * Check the current speed: if gen2 speed was requested and we are not - * at gen2 speed yet, retrain again for gen2. - */ - val = rockchip_pcie_read(rockchip, PCIE_CORE_CTRL); - if (!PCIE_LINK_IS_GEN2(val) && rockchip->link_gen == 2) { - /* Enable retrain for gen2 */ - rockchip_pcie_ep_retrain_link(rockchip); - readl_poll_timeout(rockchip->apb_base + PCIE_CORE_CTRL, - val, PCIE_LINK_IS_GEN2(val), 50, - LINK_TRAIN_TIMEOUT); - } - /* Check again that the link is up */ if (!rockchip_pcie_ep_link_up(rockchip)) goto again; diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c index d203c4876d30..5b071bdc4f90 100644 --- a/drivers/pci/controller/pcie-rockchip-host.c +++ b/drivers/pci/controller/pcie-rockchip-host.c @@ -24,6 +24,7 @@ #include <linux/of_pci.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> +#include <linux/regulator/consumer.h> #include "../pci.h" #include "pcie-rockchip.h" @@ -328,26 +329,6 @@ static int rockchip_pcie_host_init_port(struct rockchip_pcie *rockchip) goto err_power_off_phy; } - if (rockchip->link_gen == 2) { - /* - * Enable retrain for gen2. This should be configured only after - * gen1 finished. - */ - status = rockchip_pcie_read(rockchip, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL2); - status &= ~PCI_EXP_LNKCTL2_TLS; - status |= PCI_EXP_LNKCTL2_TLS_5_0GT; - rockchip_pcie_write(rockchip, status, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL2); - status = rockchip_pcie_read(rockchip, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL); - status |= PCI_EXP_LNKCTL_RL; - rockchip_pcie_write(rockchip, status, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL); - - err = readl_poll_timeout(rockchip->apb_base + PCIE_CORE_CTRL, - status, PCIE_LINK_IS_GEN2(status), 20, - 500 * USEC_PER_MSEC); - if (err) - dev_dbg(dev, "PCIe link training gen2 timeout, fall back to gen1!\n"); - } - /* Check the final link width from negotiated lane counter from MGMT */ status = rockchip_pcie_read(rockchip, PCIE_CORE_CTRL); status = 0x1 << ((status & PCIE_CORE_PL_CONF_LANE_MASK) >> diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c index 8d78ab1b31f0..45cc888ac553 100644 --- a/drivers/pci/controller/pcie-rockchip.c +++ b/drivers/pci/controller/pcie-rockchip.c @@ -66,8 +66,10 @@ int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip) } rockchip->link_gen = of_pci_get_max_link_speed(node); - if (rockchip->link_gen < 0 || rockchip->link_gen > 2) - rockchip->link_gen = 2; + if (rockchip->link_gen < 0 || rockchip->link_gen >= 2) { + rockchip->link_gen = 1; + dev_warn(dev, "invalid max-link-speed, limited to 2.5 GT/s\n"); + } for (i = 0; i < ROCKCHIP_NUM_PM_RSTS; i++) rockchip->pm_rsts[i].id = rockchip_pci_pm_rsts[i]; @@ -147,12 +149,8 @@ int rockchip_pcie_init_port(struct rockchip_pcie *rockchip) goto err_exit_phy; } - if (rockchip->link_gen == 2) - rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_2, - PCIE_CLIENT_CONFIG); - else - rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1, - PCIE_CLIENT_CONFIG); + rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1, + PCIE_CLIENT_CONFIG); regs = PCIE_CLIENT_ARI_ENABLE | PCIE_CLIENT_CONF_LANE_NUM(rockchip->lanes); diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h index 3e82a69b9c00..b5da15601b58 100644 --- a/drivers/pci/controller/pcie-rockchip.h +++ b/drivers/pci/controller/pcie-rockchip.h @@ -42,7 +42,6 @@ #define PCIE_CLIENT_MODE_RC HWORD_SET_BIT(0x0040) #define PCIE_CLIENT_MODE_EP HWORD_CLR_BIT(0x0040) #define PCIE_CLIENT_GEN_SEL_1 HWORD_CLR_BIT(0x0080) -#define PCIE_CLIENT_GEN_SEL_2 HWORD_SET_BIT(0x0080) #define PCIE_CLIENT_LEGACY_INT_CTRL (PCIE_CLIENT_BASE + 0x0c) #define PCIE_CLIENT_INT_IN_ASSERT HWORD_SET_BIT(0x0002) #define PCIE_CLIENT_INT_IN_DEASSERT HWORD_CLR_BIT(0x0002) @@ -197,8 +196,6 @@ (((x) & PCIE_CORE_PL_CONF_LS_MASK) == PCIE_CORE_PL_CONF_LS_READY) #define PCIE_LINK_UP(x) \ (((x) & PCIE_CLIENT_LINK_STATUS_MASK) == PCIE_CLIENT_LINK_STATUS_UP) -#define PCIE_LINK_IS_GEN2(x) \ - (((x) & PCIE_CORE_PL_CONF_SPEED_MASK) == PCIE_CORE_PL_CONF_SPEED_5G) #define RC_REGION_0_ADDR_TRANS_H 0x00000000 #define RC_REGION_0_ADDR_TRANS_L 0x00000000 diff --git a/drivers/pci/controller/pcie-rzg3s-host.c b/drivers/pci/controller/pcie-rzg3s-host.c index a1a78bdb7af1..077cfb0834b3 100644 --- a/drivers/pci/controller/pcie-rzg3s-host.c +++ b/drivers/pci/controller/pcie-rzg3s-host.c @@ -1006,22 +1006,38 @@ static const struct irq_domain_ops rzg3s_pcie_intx_domain_ops = { .xlate = irq_domain_xlate_onetwocell, }; +static void rzg3s_pcie_teardown_intx(struct rzg3s_pcie_host *host, + int count) +{ + while (--count >= 0) { + irq_set_chained_handler_and_data(host->intx_irqs[count], NULL, + NULL); + } + + if (host->intx_domain) + irq_domain_remove(host->intx_domain); +} + static int rzg3s_pcie_init_irqdomain(struct rzg3s_pcie_host *host) { struct device *dev = host->dev; struct platform_device *pdev = to_platform_device(dev); + int i, ret; - for (int i = 0; i < PCI_NUM_INTX; i++) { + for (i = 0; i < PCI_NUM_INTX; i++) { char irq_name[5] = {0}; int irq; scnprintf(irq_name, ARRAY_SIZE(irq_name), "int%c", 'a' + i); irq = platform_get_irq_byname(pdev, irq_name); - if (irq < 0) - return dev_err_probe(dev, -EINVAL, - "Failed to parse and map INT%c IRQ\n", - 'A' + i); + if (irq < 0) { + ret = irq; + dev_err_probe(dev, ret, + "Failed to parse and map INT%c IRQ\n", + 'A' + i); + goto teardown_intx; + } host->intx_irqs[i] = irq; irq_set_chained_handler_and_data(irq, @@ -1033,21 +1049,27 @@ static int rzg3s_pcie_init_irqdomain(struct rzg3s_pcie_host *host) PCI_NUM_INTX, &rzg3s_pcie_intx_domain_ops, host); - if (!host->intx_domain) - return dev_err_probe(dev, -EINVAL, - "Failed to add irq domain for INTx IRQs\n"); + if (!host->intx_domain) { + ret = -EINVAL; + dev_err_probe(dev, ret, + "Failed to add irq domain for INTx IRQs\n"); + goto teardown_intx; + } irq_domain_update_bus_token(host->intx_domain, DOMAIN_BUS_WIRED); if (IS_ENABLED(CONFIG_PCI_MSI)) { - int ret = rzg3s_pcie_init_msi(host); + ret = rzg3s_pcie_init_msi(host); - if (ret) { - irq_domain_remove(host->intx_domain); - return ret; - } + if (ret) + goto teardown_intx; } return 0; + +teardown_intx: + rzg3s_pcie_teardown_intx(host, i); + + return ret; } static void rzg3s_pcie_teardown_irqdomain(struct rzg3s_pcie_host *host) @@ -1055,7 +1077,7 @@ static void rzg3s_pcie_teardown_irqdomain(struct rzg3s_pcie_host *host) if (IS_ENABLED(CONFIG_PCI_MSI)) rzg3s_pcie_teardown_msi(host); - irq_domain_remove(host->intx_domain); + rzg3s_pcie_teardown_intx(host, PCI_NUM_INTX); } static int rzg3s_pcie_set_max_link_speed(struct rzg3s_pcie_host *host) @@ -1077,7 +1099,7 @@ static int rzg3s_pcie_set_max_link_speed(struct rzg3s_pcie_host *host) FIELD_GET(RZG3S_PCI_PCSTAT1_LTSSM_STATE, tmp) == ltssm_state_l0, PCIE_LINK_WAIT_SLEEP_MS * MILLI, PCIE_LINK_WAIT_SLEEP_MS * MILLI * - PCIE_LINK_WAIT_MAX_RETRIES); + PCIE_LINK_WAIT_MAX_RETRIES * 10); if (ret) return ret; @@ -1897,6 +1919,7 @@ static int rzg3s_pcie_probe(struct platform_device *pdev) return 0; host_probe_teardown: + clk_disable_unprepare(host->port.refclk); rzg3s_pcie_teardown_irqdomain(host); host->data->config_deinit(host); rpm_put: @@ -2074,6 +2097,25 @@ static const struct rzg3s_pcie_soc_data rzg3e_soc_data = { }, }; +static const struct rzg3s_pcie_soc_data rzg3l_soc_data = { + .power_resets = rzg3e_soc_power_resets, + .num_power_resets = ARRAY_SIZE(rzg3e_soc_power_resets), + .num_pcie_controllers = 1, + .config_pre_init = rzg3e_pcie_config_pre_init, + .config_post_init = rzg3e_pcie_config_post_init, + .config_deinit = rzg3e_pcie_config_deinit, + .sysc_info = { + [RZG3S_PCIE_CONTROLLER_ID_0] = { + .functions = { + [RZG3S_SYSC_FUNC_ID_L1_ALLOW] = { + .offset = 0x3a0, + .mask = BIT(8), + }, + }, + }, + }, +}; + static const struct rzg3s_pcie_soc_data rzv2h_soc_data = { .power_resets = rzg3e_soc_power_resets, .num_power_resets = ARRAY_SIZE(rzg3e_soc_power_resets), @@ -2124,6 +2166,10 @@ static const struct of_device_id rzg3s_pcie_of_match[] = { .data = &rzg3s_soc_data, }, { + .compatible = "renesas,r9a08g046-pcie", + .data = &rzg3l_soc_data, + }, + { .compatible = "renesas,r9a09g047-pcie", .data = &rzg3e_soc_data, }, diff --git a/drivers/pci/controller/pcie-tegra264.c b/drivers/pci/controller/pcie-tegra264.c index 653136db401e..c97576ab4a31 100644 --- a/drivers/pci/controller/pcie-tegra264.c +++ b/drivers/pci/controller/pcie-tegra264.c @@ -49,8 +49,7 @@ #define XAL_RC_BAR_CNTL_STANDARD_64B_BAR_EN BIT(2) /* XTL registers */ -#define XTL_RC_PCIE_CFG_LINK_CAPS 0x56 -#define XTL_RC_PCIE_CFG_LINK_STATUS 0x5a +#define XTL_RC_PCIE_CAP 0x48 /* PCIe Capability */ #define XTL_RC_MGMT_PERST_CONTROL 0x218 #define XTL_RC_MGMT_PERST_CONTROL_PERST_O_N BIT(0) @@ -118,11 +117,11 @@ static void tegra264_pcie_icc_set(struct tegra264_pcie *pcie) * possible, so this is as good as it gets for now. */ if (pcie->link_up) { - value = readw(pcie->ecam + XTL_RC_PCIE_CFG_LINK_STATUS); + value = readw(pcie->ecam + XTL_RC_PCIE_CAP + PCI_EXP_LNKSTA); speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, value); width = FIELD_GET(PCI_EXP_LNKSTA_NLW, value); } else { - value = readw(pcie->ecam + XTL_RC_PCIE_CFG_LINK_CAPS); + value = readl(pcie->ecam + XTL_RC_PCIE_CAP + PCI_EXP_LNKCAP); speed = FIELD_GET(PCI_EXP_LNKCAP_SLS, value); width = FIELD_GET(PCI_EXP_LNKCAP_MLW, value); } @@ -263,7 +262,7 @@ static bool tegra264_pcie_supports_hotplug(struct tegra264_pcie *pcie) static bool tegra264_pcie_link_up(struct tegra264_pcie *pcie, enum pci_bus_speed *speed) { - u16 value = readw(pcie->ecam + XTL_RC_PCIE_CFG_LINK_STATUS); + u16 value = readw(pcie->ecam + XTL_RC_PCIE_CAP + PCI_EXP_LNKSTA); if (value & PCI_EXP_LNKSTA_DLLLA) { if (speed) diff --git a/drivers/pci/controller/pcie-xilinx-cpm.c b/drivers/pci/controller/pcie-xilinx-cpm.c index d38f27e20761..1fdd7f920590 100644 --- a/drivers/pci/controller/pcie-xilinx-cpm.c +++ b/drivers/pci/controller/pcie-xilinx-cpm.c @@ -6,6 +6,8 @@ */ #include <linux/bitfield.h> +#include <linux/delay.h> +#include <linux/gpio/consumer.h> #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/irqchip.h> @@ -16,6 +18,7 @@ #include <linux/of_address.h> #include <linux/of_pci.h> #include <linux/of_platform.h> +#include <linux/reset.h> #include "../pci.h" #include "pcie-xilinx-common.h" @@ -113,6 +116,8 @@ struct xilinx_cpm_variant { * @irq: Error interrupt number * @lock: lock protecting shared register access * @variant: CPM version check pointer + * @perst_gpio: GPIO descriptor for PERST# signal handling + * @rstc: Pointer to the PCIe controller reset */ struct xilinx_cpm_pcie { struct device *dev; @@ -125,6 +130,8 @@ struct xilinx_cpm_pcie { int irq; raw_spinlock_t lock; const struct xilinx_cpm_variant *variant; + struct gpio_desc *perst_gpio; + struct reset_control *rstc; }; static u32 pcie_read(struct xilinx_cpm_pcie *port, u32 reg) @@ -389,7 +396,7 @@ static int xilinx_cpm_pcie_init_irq_domain(struct xilinx_cpm_pcie *port) struct device_node *pcie_intc_node; /* Setup INTx */ - pcie_intc_node = of_get_next_child(node, NULL); + pcie_intc_node = of_get_child_by_name(node, "interrupt-controller"); if (!pcie_intc_node) { dev_err(dev, "No PCIe Intc node found\n"); return -EINVAL; @@ -470,6 +477,22 @@ static int xilinx_cpm_setup_irq(struct xilinx_cpm_pcie *port) } /** + * xilinx_cpm_pcie_reset - Reset the PCIe controller and deassert PERST# + * @port: PCIe port information + * + * Reset the PCIe controller and then release the PERST# signal so that the + * link can train once the bridge is enabled. + */ +static void xilinx_cpm_pcie_reset(struct xilinx_cpm_pcie *port) +{ + reset_control_assert(port->rstc); + udelay(PCIE_T_PERST_US); + reset_control_deassert(port->rstc); + gpiod_set_value_cansleep(port->perst_gpio, 0); + msleep(PCIE_RESET_CONFIG_WAIT_MS); +} + +/** * xilinx_cpm_pcie_init_port - Initialize hardware * @port: PCIe port information */ @@ -480,6 +503,9 @@ static void xilinx_cpm_pcie_init_port(struct xilinx_cpm_pcie *port) if (variant->version == CPM5NC_HOST) return; + if (port->perst_gpio || port->rstc) + xilinx_cpm_pcie_reset(port); + if (cpm_pcie_link_up(port)) dev_info(port->dev, "PCIe Link is UP\n"); else @@ -513,6 +539,42 @@ static void xilinx_cpm_pcie_init_port(struct xilinx_cpm_pcie *port) } /** + * xilinx_cpm_pcie_parse_port - Parse the PCIe Root Port child node + * @port: PCIe port information + * + * Read the PERST# GPIO from the Root Port child node. + * + * Return: '0' on success and error value on failure + */ +static int xilinx_cpm_pcie_parse_port(struct xilinx_cpm_pcie *port) +{ + struct device *dev = port->dev; + + /* + * This platform currently supports only one Root Port, so the loop + * will execute only once. + * TODO: Enhance the driver to handle multiple Root Ports in the future. + */ + for_each_child_of_node_with_prefix(dev->of_node, pcie_port_node, "pcie") { + port->perst_gpio = devm_fwnode_gpiod_get(dev, + of_fwnode_handle(pcie_port_node), + "reset", GPIOD_OUT_HIGH, + NULL); + if (IS_ERR(port->perst_gpio)) { + if (PTR_ERR(port->perst_gpio) == -ENOENT) { + port->perst_gpio = NULL; + return 0; + } + return dev_err_probe(dev, PTR_ERR(port->perst_gpio), + "Failed to request reset GPIO\n"); + } + return 0; + } + + return 0; +} + +/** * xilinx_cpm_pcie_parse_dt - Parse Device tree * @port: PCIe port information * @bus_range: Bus resource @@ -525,6 +587,19 @@ static int xilinx_cpm_pcie_parse_dt(struct xilinx_cpm_pcie *port, struct device *dev = port->dev; struct platform_device *pdev = to_platform_device(dev); struct resource *res; + int ret; + + /* CPM5NC does not support PERST# handling yet */ + if (port->variant->version != CPM5NC_HOST) { + port->rstc = devm_reset_control_get_optional_exclusive(dev, NULL); + if (IS_ERR(port->rstc)) + return dev_err_probe(dev, PTR_ERR(port->rstc), + "Failed to request reset\n"); + + ret = xilinx_cpm_pcie_parse_port(port); + if (ret) + return ret; + } port->cpm_base = devm_platform_ioremap_resource_byname(pdev, "cpm_slcr"); diff --git a/drivers/pci/controller/pcie-xilinx-dma-pl.c b/drivers/pci/controller/pcie-xilinx-dma-pl.c index b037c8f315e4..e18eeecdc8a9 100644 --- a/drivers/pci/controller/pcie-xilinx-dma-pl.c +++ b/drivers/pci/controller/pcie-xilinx-dma-pl.c @@ -581,7 +581,7 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port) port->pldma_domain = irq_domain_create_linear(of_fwnode_handle(pcie_intc_node), 32, &event_domain_ops, port); if (!port->pldma_domain) - return -ENOMEM; + goto err_put_node; irq_domain_update_bus_token(port->pldma_domain, DOMAIN_BUS_NEXUS); @@ -589,7 +589,7 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port) &intx_domain_ops, port); if (!port->intx_domain) { dev_err(dev, "Failed to get a INTx IRQ domain\n"); - return -ENOMEM; + goto err_put_node; } irq_domain_update_bus_token(port->intx_domain, DOMAIN_BUS_WIRED); @@ -597,13 +597,17 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port) ret = xilinx_pl_dma_pcie_init_msi_irq_domain(port); if (ret != 0) { irq_domain_remove(port->intx_domain); - return -ENOMEM; + goto err_put_node; } of_node_put(pcie_intc_node); raw_spin_lock_init(&port->lock); return 0; + +err_put_node: + of_node_put(pcie_intc_node); + return -ENOMEM; } static int xilinx_pl_dma_pcie_setup_irq(struct pl_dma_pcie *port) diff --git a/drivers/pci/controller/plda/pcie-starfive.c b/drivers/pci/controller/plda/pcie-starfive.c index 02d5aa6dc14b..b693eabdf85b 100644 --- a/drivers/pci/controller/plda/pcie-starfive.c +++ b/drivers/pci/controller/plda/pcie-starfive.c @@ -21,6 +21,7 @@ #include <linux/platform_device.h> #include <linux/pm_runtime.h> #include <linux/regmap.h> +#include <linux/regulator/consumer.h> #include <linux/reset.h> #include "../../pci.h" diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index 241023ecf677..9d97a832785e 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -92,6 +92,23 @@ enum vmd_features { * referred to as MEMBAR2 or MSI-X BAR. */ VMD_FEAT_USE_BIOS_INFO = (1 << 6), + + /* + * Some Intel VMD devices are affected by interrupt ordering errata, + * including MTL016 (0x7d0b) and ARL004 (0xad0b): + * the VMD may signal its MSI before the posted writes that carry the + * child device's DMA data have landed in memory. The demuxed handler + * then runs against a not-yet-coherent completion queue and misses the + * completion entirely, so the I/O is only recovered when the block + * layer timeout fires and polls the queue ("timeout, completion + * polled"). Intel's documented workaround is to issue a dummy read + * to the MSI initiator (the child device) before handling the + * interrupt: the read completion cannot pass the device's earlier + * posted writes, so it pulls them into memory per PCIe ordering + * rules. A read that terminates at the VMD itself does not order + * against the child's writes and is not sufficient (measured). + */ + VMD_FEAT_INTERRUPT_QUIRK = (1 << 7), }; #define VMD_BIOS_PM_QUIRK_LTR 0x1003 /* 3145728 ns */ @@ -114,6 +131,9 @@ static DEFINE_RAW_SPINLOCK(list_lock); * @irq: back pointer to parent. * @enabled: true if driver enabled IRQ * @virq: the virtual IRQ value provided to the requesting driver. + * @flush_addr: config space address of the initiating device, read before + * demuxing to flush its posted writes; NULL if the + * VMD is not affected. * * Every MSI/MSI-X IRQ requested for a device in a VMD domain will be mapped to * a VMD IRQ using this structure. @@ -123,8 +143,14 @@ struct vmd_irq { struct vmd_irq_list *irq; bool enabled; unsigned int virq; + void __iomem *flush_addr; }; +struct vmd_dev; + +static void __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus, + unsigned int devfn, int reg, int len); + /** * struct vmd_irq_list - list of driver requested IRQs mapping to a VMD vector * @irq_list: the list of irq's the VMD one demuxes to. @@ -132,12 +158,15 @@ struct vmd_irq { * @count: number of child IRQs assigned to this vector; used to track * sharing. * @virq: The underlying VMD Linux interrupt number + * @vmd: back pointer to the owning VMD device; serializes the MTL016 + * flush read against other config space access. */ struct vmd_irq_list { struct list_head irq_list; struct srcu_struct srcu; unsigned int count; unsigned int virq; + struct vmd_dev *vmd; }; struct vmd_dev { @@ -295,6 +324,13 @@ static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq, INIT_LIST_HEAD(&vmdirq->node); vmdirq->irq = vmd_next_irq(vmd, desc); vmdirq->virq = virq + i; + if (vmd->features & VMD_FEAT_INTERRUPT_QUIRK) { + struct pci_dev *pdev = msi_desc_to_pci_dev(desc); + + vmdirq->flush_addr = vmd_cfg_addr(vmd, pdev->bus, + pdev->devfn, + PCI_VENDOR_ID, 2); + } irq_domain_set_info(domain, virq + i, vmdirq->irq->virq, &vmd_msi_controller, vmdirq, @@ -756,8 +792,20 @@ static irqreturn_t vmd_irq(int irq, void *data) int idx; idx = srcu_read_lock(&irqs->srcu); - list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node) + list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node) { + /* + * MTL016: the MSI may have outrun the initiating device's + * posted writes (e.g. its NVMe completion entry). A read + * that completes at the initiator flushes them, so the + * demuxed handler observes a coherent completion queue. + * The value is discarded; only the ordering matters. + */ + if (vmdirq->flush_addr) { + guard(raw_spinlock)(&irqs->vmd->cfg_lock); + readw(vmdirq->flush_addr); + } generic_handle_irq(vmdirq->virq); + } srcu_read_unlock(&irqs->srcu, idx); return IRQ_HANDLED; @@ -788,6 +836,7 @@ static int vmd_alloc_irqs(struct vmd_dev *vmd) return err; INIT_LIST_HEAD(&vmd->irqs[i].irq_list); + vmd->irqs[i].vmd = vmd; vmd->irqs[i].virq = pci_irq_vector(dev, i); err = devm_request_irq(&dev->dev, vmd->irqs[i].virq, vmd_irq, IRQF_NO_THREAD, @@ -1250,9 +1299,9 @@ static const struct pci_device_id vmd_ids[] = { {PCI_VDEVICE(INTEL, 0xa77f), .driver_data = VMD_FEATS_CLIENT,}, {PCI_VDEVICE(INTEL, 0x7d0b), - .driver_data = VMD_FEATS_CLIENT,}, + .driver_data = VMD_FEATS_CLIENT | VMD_FEAT_INTERRUPT_QUIRK,}, {PCI_VDEVICE(INTEL, 0xad0b), - .driver_data = VMD_FEATS_CLIENT,}, + .driver_data = VMD_FEATS_CLIENT | VMD_FEAT_INTERRUPT_QUIRK,}, {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B), .driver_data = VMD_FEATS_CLIENT,}, {PCI_VDEVICE(INTEL, 0xb60b), |