| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd fixes from Jason Gunthorpe:
"Several bug fixes found by tools and fuzzing:
- Incorrect domain passed during replace to ack faults
- Block the access API from using dmabuf
- Missing unlock on error unwind
- Race seeing a partially setup vdevice in the xarray
- Do not allow vdevices to have multiple stream ids in SMMUv3
- Possible UAF if racing faults with domain changes"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommu/iommufd: Fix IOPF group ownership UAF
iommu/arm-smmu-v3-iommufd: Require exactly one Stream ID for a vDEVICE
iommufd/viommu: Publish a vDEVICE only after vdevice_init() succeeds
iommufd/viommu: Release the igroup lock on the vdevice_size error path
iommufd: Reject DMABUF pages from the access pin path
iommufd: Fix wrong hwpt passed to iommufd_auto_response_faults on replace
|
|
iopf_group_alloc() links each last-page IOPF group into the generic IOPF
pending list before invoking the domain fault handler.
iommufd_fault_iopf_handler() also queued an accepted group in the
IOMMUFD deliver list without removing it from the generic pending list.
When detach or HWPT replacement drops the device's IOPF reference count
to zero, an IOMMU driver may call iopf_queue_remove_device(). That
function responds to and frees groups through the generic pending list
without removing the same groups from IOMMUFD's deliver list or response
xarray. A later read, response, or cleanup can then access the freed
group and cause a UAF.
Fix this by dequeuing an accepted group from the generic pending list
before IOMMUFD queues it for userspace response.
Make iopf_group_response() send a response regardless of pending-list
membership, so the dequeued group can still be completed by IOMMUFD.
Link: https://patch.msgid.link/r/3CFD314D0FE4D7EC+20260720085017.3998878-2-peiyang_he@smail.nju.edu.cn
Closes: https://lore.kernel.org/all/B4F28798E2E784CA+d29f723c-b2b5-4b67-8d1c-4f7b9b0b27cb@smail.nju.edu.cn/
Fixes: 34765cbc679c ("iommufd: Associate fault object with iommufd_hw_pgtable")
Cc: stable@vger.kernel.org
Tested-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
dmar_latency_disable() intends to zero out only the single
latency_statistic entry for the given type, but the memset size was
computed as sizeof(*lstat) * DMAR_LATENCY_NUM, which clears the entire
array starting from &lstat[type].
When type > 0, this writes beyond the end of the allocated array,
corrupting adjacent memory.
Fix by using sizeof(*lstat) to clear only the target entry.
Fixes: 55ee5e67a59a ("iommu/vt-d: Add common code for dmar latency performance monitors")
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The ivrs_acpihid command-line parser appends entries to a fixed
four-element early_acpihid_map array. Unlike the sibling IOAPIC and HPET
parsers, it does not reject a fifth entry before incrementing the map size.
Check the capacity at the common found label before parsing the HID and
UID or writing the entry.
Fixes: ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter")
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Hardware implementations report Scalable-Mode Page-walk Coherency Support
via the SMPWCS field in the extended capability register. If the hardware
does not support page-walk coherency, a clflush is required every time
the page table entries (which are walked by the IOMMU hardware) are
updated.
In the SVA case, page tables are managed by the CPU mm core, not by the
IOMMU driver. Because the IOMMU driver has no way of knowing whether the
CPU page table management code has ensured coherency via clflush, the
driver must deny SVA if the hardware does not support coherent paging.
Fixes: ff3dc6521f78 ("iommu/vt-d: Fix CPU and IOMMU SVM feature matching checks")
Cc: stable@vger.kernel.org
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
iommu_completion_wait()
need_sync is a per-IOMMU flag shared by all domains and devices behind
that IOMMU. It is set whenever a command is queued with sync == true and
cleared when a completion-wait (CWAIT) command is queued. However, a
cleared need_sync only means that a covering CWAIT has been queued, not
that all previously queued commands have actually completed in hardware.
iommu_completion_wait() read need_sync locklessly and returned early
when it was false. This breaks the "block until all previously queued
commands have completed" contract in a multi-CPU scenario:
CPU2: queue inv-B => need_sync = true
CPU1: queue CWAIT(N); need_sync = false; then wait_on_sem(N)
CPU2: read need_sync == false => return 0 (no wait!)
CPU2 returns without waiting for any sequence number even though its
inv-B may not have completed yet (CWAIT(N), queued after inv-B, has not
been signaled). CPU2 then proceeds to, for example, free page-table
pages while the IOMMU can still walk stale translations, opening a
use-after-free window. This is a logical race in the meaning of the
flag, not a memory-visibility issue, so barriers alone do not help.
Fix it without losing the optimization of avoiding redundant CWAIT
commands: take iommu->lock before testing need_sync, and when it is
false do not return early but wait for the last allocated sequence
number (cmd_sem_val). Since need_sync == false implies no sync command
was queued after the last CWAIT, that CWAIT is FIFO-ordered after every
not-yet-completed command, so waiting for its sequence number guarantees
all prior commands (possibly queued by another CPU) have completed. The
common path with pending work is unchanged and no extra hardware command
is issued.
Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
Fixes: 815b33fdc279 ("x86/amd-iommu: Cleanup completion-wait handling")
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
A couple of runs of different AI tools have generated something like the
following bug report:
In nested_domain_free(), when refcount_dec_and_test() returns false
(other nested domains still reference the same gdom_info), the function
returns without calling kfree(ndom), leaking the nested_domain
structure. This problem wasn't introduced by this patch, but exists in
the code from commit 757d2b1fdf5b that the patch modifies. Each
nested_domain (ndom) is allocated individually in
amd_iommu_alloc_domain_nested() via kzalloc_obj(*ndom). The .free
callback is the sole point responsible for freeing this domain. When
the refcount is > 0, only the xa_unlock_irqrestore is performed and the
function returns, leaving ndom permanently allocated. This leak occurs
every time a nested domain sharing a gDomID is destroyed while other
domains still use that gDomID.
There is a similar leak later in this function in the WARN_ON() test when
the mapping is already NULL. Switch to a RAII-based cleanup for ndom, since
it should always be freed in this function.
Fixes: 757d2b1fdf5b ("iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain invalidation")
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Lockdep complains:
[ 259.410489] =====================================================
[ 259.417287] WARNING: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected
[ 259.424667] 7.0.0-g51db1d8d2113 #54 Not tainted
[ 259.429718] -----------------------------------------------------
[ 259.436516] qemu-system-x86/10143 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
[ 259.444670] ff3b2b1c60305170 (&xa->xa_lock#25){+.+.}-{3:3}, at: __domain_flush_pages+0x17c/0x4b0
[ 259.454485]
and this task is already holding:
[ 259.460991] ff3b2b1c98504cc0 (&domain->lock){-.-.}-{3:3}, at: amd_iommu_iotlb_sync+0x25/0x60
[ 259.470408] which would create a new lock dependency:
[ 259.476041] (&domain->lock){-.-.}-{3:3} -> (&xa->xa_lock#25){+.+.}-{3:3}
[ 259.483615]
but this new dependency connects a HARDIRQ-irq-safe lock:
[ 259.492447] (&domain->lock){-.-.}-{3:3}
[ 259.492449]
... which became HARDIRQ-irq-safe at:
[ 259.503705] lock_acquire+0xb6/0x2e0
[ 259.507790] _raw_spin_lock_irqsave+0x3e/0x60
[ 259.512748] amd_iommu_flush_iotlb_all+0x20/0x50
[ 259.517996] iommu_dma_free_iova.isra.0+0x1b8/0x1e0
[ 259.523534] __iommu_dma_unmap+0xc2/0x140
[ 259.528100] iommu_dma_unmap_phys+0x55/0xc0
[ 259.532863] dma_unmap_phys+0x274/0x2e0
[ 259.537238] dma_unmap_page_attrs+0x17/0x30
[ 259.542000] nvme_unmap_data+0x13e/0x280
[ 259.546473] nvme_pci_complete_batch+0x45/0x70
[ 259.551524] nvme_irq+0x83/0x90
[ 259.555123] __handle_irq_event_percpu+0x92/0x360
[ 259.560466] handle_irq_event+0x39/0x80
[ 259.564841] handle_edge_irq+0xb2/0x1a0
[ 259.569214] __common_interrupt+0x4e/0x130
[ 259.573882] common_interrupt+0x88/0xa0
[ 259.578256] asm_common_interrupt+0x27/0x40
[ 259.583019] cpuidle_enter_state+0x119/0x5d0
[ 259.587877] cpuidle_enter+0x2e/0x50
[ 259.591962] do_idle+0x153/0x2c0
[ 259.595657] cpu_startup_entry+0x29/0x30
[ 259.600128] start_secondary+0x118/0x150
[ 259.604601] common_startup_64+0x13e/0x141
[ 259.609266]
to a HARDIRQ-irq-unsafe lock:
[ 259.615384] (&xa->xa_lock#25){+.+.}-{3:3}
[ 259.615386]
... which became HARDIRQ-irq-unsafe at:
[ 259.627039] ...
[ 259.627039] lock_acquire+0xb6/0x2e0
[ 259.633071] _raw_spin_lock+0x2f/0x50
[ 259.637250] amd_iommu_alloc_domain_nested+0x140/0x3c0
[ 259.643078] iommufd_hwpt_alloc+0x272/0x800 [iommufd]
[ 259.648813] iommufd_fops_ioctl+0x14e/0x200 [iommufd]
[ 259.654547] __x64_sys_ioctl+0x9d/0xf0
...
Since amd_iommu_domain_flush_pages() necessarily holds domain->lock to do the
flush, switch the allocation side in gdom_info_load_or_alloc_locked() to
HARDIRQ-safe allocation. The IOMMU_DESTROY->free path has the same issue,
so switch that path to HARDIRQ-safe locking as well.
Fixes: 757d2b1fdf5b ("iommu/amd: Introduce gDomID-to-hDomID Mapping and handle parent domain invalidation")
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
arm_vsmmu_vsid_to_sid() maps a guest's vSID to a single physical Stream ID
taken from master->streams[0], assuming a device has exactly one stream. A
device with several streams gets only its first one mapped, so a guest vSID
invalidation cannot reach the others' ATC and IOTLB entries; a device with
none makes master->streams a ZERO_SIZE_PTR, read out of bounds.
Add an arm_vsmmu_vdevice_init() op to reject the vDEVICE with -EOPNOTSUPP
when master->num_streams is not one, rather than mapping it silently.
Fixes: d68beb276ba26 ("iommu/arm-smmu-v3: Support IOMMU_HWPT_INVALIDATE using a VIOMMU object")
Link: https://patch.msgid.link/r/b15f2b73520f389f3f57881da2f040e7bdc18876.1783311134.git.nicolinc@nvidia.com
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Assisted-by: Claude:claude-opus-4-8
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_vdevice_alloc_ioctl() adds the vDEVICE to the viommu->vdevs xarray
with xa_cmpxchg() before the driver's vdevice_init() op runs. That op is
where a driver validates the device and may reject it, but the xarray entry
is already live by then: a concurrent IOMMU_HWPT_INVALIDATE can look it up
with iommufd_viommu_find_dev() and run the driver invalidation path against
a device that vdevice_init() would have refused.
Reserve the index with xa_insert(): it stores a zero entry that reads back
as NULL, and returns -EBUSY on a duplicate virt_id. Run vdevice_init() and
store the vDEVICE pointer only once it succeeds. A failed vdevice_init()
releases the reservation, so lookups observe the vDEVICE only after it is
fully initialized and accepted.
Fixes: ed42eee797ff3 ("iommufd/viommu: Add driver-defined vDEVICE support")
Link: https://patch.msgid.link/r/1e05999347f4bf583edbc6a1312c857d5548708c.1783311134.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Assisted-by: Claude:claude-opus-4-8
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_vdevice_alloc_ioctl() takes idev->igroup->lock, then validates the
driver's vdevice_size against the core structure size with a WARN_ON_ONCE.
On failure that guard jumps to out_put_idev, below out_unlock_igroup, so it
skips the mutex_unlock(), leaving the igroup lock held and deadlocking the
next vDEVICE operation on that group.
Jump to out_unlock_igroup instead.
Fixes: ed42eee797ff3 ("iommufd/viommu: Add driver-defined vDEVICE support")
Link: https://patch.msgid.link/r/e903f775d491296a525097e2a90b3eb6a47cf2ef.1783311134.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Assisted-by: Claude:claude-opus-4-8
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
DMABUF pages are not supported for iommufd access pinning.
iommufd_access_pin_pages() returns struct page pointers for
in-kernel CPU access, but DMABUF-backed iopt_pages do not carry
a userspace address that can be passed to the GUP path.
iopt_pages_rw_access() already rejects IOPT_ADDRESS_DMABUF before doing
CPU access. Apply the same rejection to iopt_area_add_access() before it
takes pages->mutex and calls iopt_pages_fill_xarray().
Otherwise a DMABUF-backed iopt_pages can reach the hole-fill path, where
pfn_reader_user_pin() interprets the union as uptr and
calls pin_user_pages_fast()/pin_user_pages_remote().
This fix also avoids the lockdep warning reported from that path, where
pages_dmabuf_mutex_key is held while gup_fast_fallback() may acquire
mmap_lock.
Link: https://patch.msgid.link/r/CD68F549BF3761B7+20260709050800.520607-1-peiyang_he@smail.nju.edu.cn
Reported-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Closes: https://lore.kernel.org/all/E8540D7D05768C91+8b2ef227-3368-494e-909d-7b28e1489dfb@smail.nju.edu.cn/
Fixes: 71db84a092c3 ("iommufd: Add DMABUF to iopt_pages")
Cc: stable@vger.kernel.org
Tested-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_hwpt_replace_device() calls:
iommufd_auto_response_faults(hwpt, old_handle);
passing the *new* hwpt together with the handle of
the device's *old* domain. This should be a parameter mismatch:
1. Semantically, iommufd_auto_response_faults(x, handle) scans
x->fault's deliver list and response xarray for groups matching
"handle". A group is queued under the hwpt that was attached at
fault-delivery time. old_handle is fetched *before* the domain switch,
so its group lives on old->fault, not on the new hwpt->fault.
2. Historically, the first argument was "old". The routine was
introduced by commit b7d8833677ba ("iommufd: Fault-capable hwpt
attach/detach/replace") as __fault_domain_replace_dev() in
fault.c, correctly calling iommufd_auto_response_faults(old, curr).
Commit fb21b1568ada ("iommufd: Make attach_handle generic than
fault specific") moved this into iommufd_hwpt_replace_device() in
device.c and swapped it to "hwpt". This should be a refactor regression,
not an intentional change.
Fix this by passing "old" instead.
Link: https://patch.msgid.link/r/9D652384339C69D5+20260710122952.885325-1-peiyang_he@smail.nju.edu.cn
Fixes: fb21b1568ada ("iommufd: Make attach_handle generic than fault specific")
Cc: stable@vger.kernel.org
Signed-off-by: Peiyang He <peiyang_he@smail.nju.edu.cn>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
files)
Replace the #include of <linux/mod_devicetable.h> by the more specific
<linux/device-id/*.h> where applicable. For most cases the include
can be dropped completely, only a few drivers need one or two headers
added.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://patch.msgid.link/1a3f2007c5c5dcf555c09a4035ce3ae8ef1b6c49.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux
Pull hyperv updates from Wei Liu:
- Use wakeup mailbox to boot APs in Hyper-V VTL2 TDX guests (Yunhong
Jiang, Ricardo Neri)
- Move the Hyper-V IOMMU to its own subdirectory (Mukesh Rathor)
- Cosmetic changes to mshv and balloon driver (Junrui Luo, Markus
Elfring)
* tag 'hyperv-next-signed-20260621' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux:
mshv: add bounds check on vp_index in mshv_intercept_isr()
hv_balloon: Simplify data output in hv_balloon_debug_show()
x86/hyperv: Cosmetic changes in irqdomain.c for readability
iommu/hyperv: Create hyperv subdirectory under drivers/iommu
x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs
x86/hyperv/vtl: Mark the wakeup mailbox page as private
x86/acpi: Add a helper to get the address of the wakeup mailbox
x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests
x86/realmode: Make the location of the trampoline configurable
x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform()
x86/dt: Parse the Wakeup Mailbox for Intel processors
dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors
x86/acpi: Add functions to setup and access the wakeup mailbox
x86/topology: Add missing struct declaration and attribute dependency
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd updates from Jason Gunthorpe:
"All various fixes:
- Typo breaking the veventq uAPI for 32 bit userspace
- Several Sashiko found errors in the veventq and fault fd paths
- Fix incorrect use of dmabuf locks, and possible races with iommufd
destroy and dmabuf revoke
- Sashiko errors found in the uAPI validation for IOMMU_HWPT_INVALIDATE"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommu: Avoid copying the user array twice in the full-array copy helper
iommufd/selftest: Add invalidation entry_num and entry_len boundary tests
iommufd: Set upper bounds on cache invalidation entry_num and entry_len
iommufd: Clarify IOAS_MAP_FILE dma-buf support
iommufd: Destroy the pages content after detaching from dmabuf
iommufd: Take dma_resv lock before dma_buf_unpin() in release path
iommufd/selftest: Cover invalid read counts on vEVENTQ FD
iommufd: Avoid partial fault group delivery in iommufd_fault_fops_read()
iommufd: Break the loop on failure in iommufd_fault_fops_read()
iommufd: Reject invalid read count in iommufd_fault_fops_read()
iommufd: Propagate allocation failure in iommufd_veventq_deliver_fetch()
iommufd: Reject invalid read count in iommufd_veventq_fops_read()
iommufd: Rewind header length in done if iommufd_veventq_fops_read() fails
iommufd/selftest: Add boundary tests for veventq_depth
iommufd: Set veventq_depth upper bound
iommufd: Move vevent memory allocation outside spinlock
iommufd: Fix data_len byte-count vs element-count mismatch
iommufd: Use sizeof(*hdr) instead of sizeof(hdr) in veventq read
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux
Pull iommu updates from Joerg Roedel:
"Core Code:
- Fix dma-iommu scatterlist length handling in the P2PDMA path
- Extend the generic IOMMU page-table code with detailed gather
support for more precise invalidations
- Add pending-gather tracking to generic page-table invalidation
handling
- Add support for smaller virtual address sizes in the generic AMDv1
page-table format, including KUnit coverage
- Fix page-size bitmap calculation for smaller VA configurations
- Rework Arm io-pgtable allocation/freeing to consistently use the
iommu-pages API and address-conversion helpers
- Add PCI ATS infrastructure for devices that require ATS, including
always-on ATS handling for pre-CXL devices
AMD IOMMU:
- Fix several IOTLB invalidation details, including PDE handling,
flush-all behavior, and command address encoding
- Honor IVINFO[VASIZE] when deriving address limits
- Fix premature loop termination in init_iommu_one()
- Add Hygon family 18h model 4h IOAPIC support
- Clean up legacy-mode handling, stale comments, dead IVMD
exclusion-range code, and unused address-size macros
Arm SMMU / Arm SMMU v3:
- SMMUv2:
- Device-tree binding updates for Qualcomm Hawi, Nord and Shikra
SoCs
- Constrain the clocks which can be specified for recent Qualcomm
SoCs
- Fix broken compatible string for Qualcomm prefetcher
configuration an add new entry for the Glymur MDSS
- Ensure SMMU is powered-up when writing context bank for Adreno
client
- SMMUv3:
- Fix off-by-one in queue allocation retry loop
- Enable hardware update of access/dirty bits from the SMMU
- Re-jig command construction to use separate inline helpers for
each command type
Intel VT-d:
- Add the PCI segment number to DMA fault messages
- Improve support for non-PRI mode SVA
- Ensure atomicity during context entry teardown
- Fix RB-tree corruption in the probe error path
RISC-V IOMMU:
- Add NAPOT range invalidation support
- Use detailed gather information for invalidation decisions
- Compute the best stride for single invalidations
- Advertise Svpbmt support to the generic page-table code
- Add capability definitions and clean up command macro encoding
VeriSilicon IOMMU:
- Add a new VeriSilicon IOMMU driver
- Add devicetree binding documentation and MAINTAINERS coverage
- Add the RK3588 VeriSilicon IOMMU node
- Apply small cleanups and warning fixes in the new driver
Rockchip IOMMU:
- Disable the fetch DTE time limit
Apple DART:
- Correct a stale CONFIG_PCIE_APPLE macro name in a comment"
* tag 'iommu-updates-v7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux: (66 commits)
iommu/dma-iommu: Fix wrong scatterlist length assignment in P2PDMA path
iommu/amd: Control INVALIDATE_IOMMU_PAGES PDE from the gather
iommu/amd: Make CMD_INV_IOMMU_ALL_PAGES_ADDRESS match the spec
iommu/amd: Have amd_iommu_domain_flush_pages() use last
iommu/amd: Pass last in through to build_inv_address()
iommu/amd: Simplify build_inv_address()
iommu/apple-dart: correct CONFIG_PCIE_APPLE macro name in comment
iommu/vt-d: Fix RB-tree corruption in probe error path
iommu/vt-d: Improve IOMMU fault information
iommu/vt-d: Remove typo from pasid_pte_config_nested()
iommu/vt-d: Clear Present bit before tearing down scalable-mode context entry
iommu/vt-d: Avoid WARNING in sva unbind path
dt-bindings: arm-smmu: Correct and add constraints for Hawi, Shikra and Kaanapali
dt-bindings: arm-smmu: Add compatible for Qualcomm Nord SoC
iommu/amd: Don't split flush for amd_iommu_domain_flush_all()
iommu/rockchip: disable fetch dte time limit
iommu/arm-smmu-v3: Allow ATS to be always on
PCI: Allow ATS to be always on for pre-CXL devices
PCI: Add pci_ats_required() for CXL.cache capable devices
iommu/vsi: Use list_for_each_entry()
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
Pull devicetree updates from Rob Herring:
"DT core:
- Add support for handling multiple cells in "iommu-map" entries
- Support only 1 entry in /reserved-memory "reg" entries. Support for
more than 1 entry has been broken
- Fix a UAF on alloc_reserved_mem_array() failure
- Make "ibm,phandle" handling logic specific to PPC
- Use memcpy() instead of strcpy() for known length strings
- Ensure __of_find_n_match_cpu_property() handles malformed "reg"
entries
- Add various checks that expected strings are strings before
accessing them
- Drop redundant memset() when unflattening DT
DT bindings:
- Add a DTS style checker. Currently hooked up to dt_binding_check to
check examples
- Convert st,nomadik platform, ti,omap-dmm, and ti,irq-crossbar
bindings to DT schema
- Add Apple System Management Controller hwmon, Qualcomm Hamoa
Embedded Controller, Qualcomm IPQ6018 PWM controller, fsl,mc1323,
Samsung SOFEF01-M DDIC panel, Freescale i.MX53 Television Encoder,
Samsung S2M series PMIC extcon, and MT6365 PMIC AuxADC schemas
- Extend bindings for QCom Maili and Nord PDC, QCom Hali fastrpc,
qcom,eliza-imem, qcom,oryon-1-5 CPU, and MT6365 Keys
- Consolidate "sram" property definitions
- Fix constraints on "nvmem" properties which only contain phandles
and no arg cells
- Another pass of fixing "phandle-array" constraints
- Add Gira vendor prefix"
* tag 'devicetree-for-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (50 commits)
dt-bindings: interrupt-controller: qcom,pdc: Add Maili compatible string
dt-bindings: interrupt-controller: ti,irq-crossbar: Convert to DT schema
dt-bindings: vendor-prefixes: add Gira
dt-bindings: embedded-controller: Add Qualcomm reference device EC description
dt-bindings: pwm: add IPQ6018 binding
dt-bindings: hwmon: Add Apple System Management Controller hwmon schema
docs: dt: writing-schema: Clarify what is required in a schema
of: Respect #{iommu,msi}-cells in maps
of: Factor arguments passed to of_map_id() into a struct
of: Add convenience wrappers for of_map_id()
of: reserved_mem: zero total_reserved_mem_cnt if no valid /reserved-memory entry
of: reserved_mem: handle NULL name in of_reserved_mem_lookup()
dt-bindings: cache: l2c2x0: Add missing power-domains
dt-bindings: interrupt-controller: renesas,r9a09g077-icu: Fix reg size in example
dt-bindings: nvmem: consumer: Make 'nvmem' an array of one-item entries
drivers/of/overlay: Use memcpy() to copy known length strings
dt-bindings: add self-test fixtures for style checker
dt-bindings: wire style checker into dt_binding_check
scripts/jobserver-exec: propagate child exit status
dt-bindings: add DTS style checker
...
|
|
gitolite.kernel.org:pub/scm/linux/kernel/git/driver-core/driver-core
Pull driver core updates from Danilo Krummrich:
"Deferred probe:
- Fix race where deferred probe timeout work could be permanently
canceled by using mod_delayed_work()
- Fix missing jiffies conversion in deferred_probe_extend_timeout()
- Guard timeout extension with delayed_work_pending() to prevent
premature firing
- Use system_percpu_wq instead of the deprecated system_wq
- Update deferred_probe_timeout documentation
device:
- Replace direct struct device bitfield access (can_match, dma_iommu,
dma_skip_sync, dma_ops_bypass, state_synced, dma_coherent,
of_node_reused, offline, offline_disabled) with flag-based
accessors using bit operations
- Reject devices with unregistered buses
- Delete unused DEVICE_ATTR_PREALLOC()
- Add low-level device attribute macros with const show/store
callbacks, allowing device attributes to reside in read-only memory
- Move core device attributes to read-only memory
- Constify group array pointers in driver_add_groups() /
driver_remove_groups(), struct bus_type, and struct device_driver
device property:
- Fix fwnode reference leak in fwnode_graph_get_endpoint_by_id()
- Initialize all fields of fwnode_handle in fwnode_init()
- Provide swnode_get()/swnode_put() wrappers around kobject_get/put()
- Allow passing struct software_node_ref_args pointers directly to
PROPERTY_ENTRY_REF()
driver_override:
- Migrate amba, cdx, vmbus, and rpmsg to the generic driver_override
infrastructure, fixing a UAF from unsynchronized access to
driver_override in bus match() callbacks
- Remove the now-unused driver_set_override()
firmware loader:
- Fix recursive lock deadlock in device_cache_fw_images() when async
work falls back to synchronous execution
- Fix device reference leak in firmware_upload_register()
platform:
- Pass KBUILD_MODNAME through the platform driver registration macro
to create module symlinks in sysfs for built-in drivers; move
module_kset initialization to a pure_initcall and tegra cbb
registration to core_initcall to ensure correct ordering
- Pass THIS_MODULE implicitly through a coresight_init_driver() macro
sysfs:
- Upgrade OOB write detection in sysfs_kf_seq_show() from printk to
WARN
- Add return value clamping to sysfs_kf_read()
Rust:
- ACPI:
Fix missing match data for PRP0001 by exporting
acpi_of_match_device()
- Auxiliary:
Replace drvdata() with dedicated registration data on
auxiliary_device. drvdata() exposed the driver's bus device private
data beyond the driver's own scope, creating ordering constraints
and forcing the data to outlive all registrations that access it.
Registration data is instead scoped structurally to the
Registration object, making lifecycle ordering enforced by
construction rather than convention.
- Rust-native device driver lifetimes (HRT):
Allow Rust device drivers to carry a lifetime parameter on their
bus device private data, tied to the device binding scope -- the
interval during which a bus device is bound to a driver. Device
resources like pci::Bar<'a> and IoMem<'a> can be stored directly in
the driver's bus device private data with a lifetime bounded by the
binding scope, so the compiler enforces at build time that they do
not outlive the binding. This removes Devres indirection from every
access site and eliminates try_access() failure paths in
destructors.
Bus driver traits use a Generic Associated Type (GAT) Data<'bound>
to introduce the lifetime on the private data, rather than
parameterizing the Driver trait itself. Auxiliary registration
data, where the lifetime is not introduced by a trait callback but
must be threaded through Registration, uses the ForLt trait (a
type-level abstraction for types generic over a lifetime).
Misc:
- Fix DT overlayed devices not probing by reverting the broken
treewide overlay fix and re-running fw_devlink consumer pickup when
an overlay is applied to a bound device
- Use root_device_register() for faux bus root device; add sanity
check for failed bus init
- Fix dev_has_sync_state() data race with READ_ONCE() and move it to
base.h
- Avoid spurious device_links warning when removing a device while
its supplier is unbinding
- Switch ISA bus to dynamic root device
- Fix suspicious RCU usage in kernfs_put()
- Remove devcoredump exit callback
- Constify devfreq_event_class"
* tag 'driver-core-7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/driver-core/driver-core: (81 commits)
software node: allow passing reference args to PROPERTY_ENTRY_REF()
driver core: platform: set mod_name in driver registration
coresight: pass THIS_MODULE implicitly through a macro
kernel: param: initialize module_kset in a pure_initcall
soc/tegra: cbb: Move driver registration from pure_initcall to core_initcall
firmware_loader: Fix recursive lock in device_cache_fw_images()
driver core: Use system_percpu_wq instead of system_wq
driver core: remove driver_set_override()
rpmsg: use generic driver_override infrastructure
Drivers: hv: vmbus: use generic driver_override infrastructure
cdx: use generic driver_override infrastructure
amba: use generic driver_override infrastructure
rust: devres: add 'static bound to Devres<T>
samples: rust: rust_driver_auxiliary: showcase lifetime-bound registration data
rust: auxiliary: generalize Registration over ForLt
rust: types: add `ForLt` trait for higher-ranked lifetime support
gpu: nova-core: separate driver type from driver data
samples: rust: rust_driver_pci: use HRT lifetime for Bar
rust: io: make IoMem and ExclusiveIoMem lifetime-parameterized
rust: pci: make Bar lifetime-parameterized
...
|
|
Change of_map_id() to take a pointer to struct of_phandle_args
instead of passing target device node and translated IDs separately.
Update all callers accordingly.
Add an explicit filter_np parameter to of_map_id() and of_map_msi_id()
to separate the filter input from the output. Previously, the target
parameter served dual purpose: as an input filter (if non-NULL, only
match entries targeting that node) and as an output (receiving the
matched node with a reference held). Now filter_np is the explicit
input filter and arg->np is the pure output.
Previously, of_map_id() would call of_node_put() on the matched node
when a filter was provided, making reference ownership inconsistent.
Remove this internal of_node_put() call so that of_map_id() now always
transfers ownership of the matched node reference to the caller via
arg->np. Callers are now consistently responsible for releasing this
reference with of_node_put(arg->np) when done.
Acked-by: Frank Li <Frank.Li@nxp.com>
Suggested-by: Rob Herring (Arm) <robh@kernel.org>
Suggested-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Signed-off-by: Charan Teja Kalla <charan.kalla@oss.qualcomm.com>
Signed-off-by: Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
Link: https://patch.msgid.link/20260603-parse_iommu_cells-v16-2-dc509dacb19a@oss.qualcomm.com
Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
|
|
Since we now have quite a few users parsing "iommu-map" and "msi-map"
properties, give them some wrappers to conveniently encapsulate the
appropriate sets of property names. This will also make it easier to
then change of_map_id() to correctly account for specifier cells.
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Marc Zyngier <maz@kernel.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
Link: https://patch.msgid.link/20260603-parse_iommu_cells-v16-1-dc509dacb19a@oss.qualcomm.com
Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
|
|
iommufd_hwpt_invalidate() takes a user-controlled entry_num and entry_len,
each bounded only by U32_MAX. An entry_len beyond the kernel's struct size
makes the copy helper verify the extra bytes are zero, scanning that excess
in one uninterruptible pass; a multi-gigabyte value over zeroed user memory
trips the soft-lockup watchdog.
A large entry_num is the other half, driving the backend invalidation loop
with no reschedule. The VT-d nested handler, for one, copies each entry and
flushes caches per iteration, pinning the CPU on a non-preemptible kernel.
Cap both in the ioctl. entry_len is held under PAGE_SIZE, above any request
struct, and entry_num under 1 << 19, the order of a hardware invalidation
queue and well beyond any real batch, bounding the per-call loop length.
Fixes: 8c6eabae3807 ("iommufd: Add IOMMU_HWPT_INVALIDATE")
Link: https://patch.msgid.link/r/447fa93663f7526eb361719e83fa8b649464483d.1780521606.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
'rockchip', 'verisilicon', 'riscv', 'intel/vt-d', 'amd/amd-vi' and 'core' into next
|
|
In iommu_dma_map_sg(), when handling PCI P2PDMA cases, the DMA length
of the current scatterlist segment `s` is incorrectly assigned from the
head entry `sg->length` instead of the current entry `s->length`.
This typo causes all P2PDMA segments in the scatterlist to inherit the
length of the first segment, leading to corrupted DMA lengths for multi-
segment scatterlists.
Fix this by using `s->length` instead of `sg->length`.
Fixes: a25e7962db ("PCI/P2PDMA: Refactor the p2pdma mapping helpers")
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Now that AMD uses iommupt, it is easy to make use of the PDE bit. If
the gather has no free list then no page directory entries were
changed.
Pass GN/PDE through the invalidation call chain in a u32 flags field
that is OR'd into data[2] and set it properly from the gather.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Wei Wang <wei.w.wang@hotmail.com>
Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The spec in Table 14 defines the "Entire Cache" case as having the low
12 bits as zero. Indeed the command format doesn't even have the low
12 bits. Since there is only one user now, fix the constant to have 0
in the low 12 bits instead of 1 and remove the masking.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Wei Wang <wei.w.wang@hotmail.com>
Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Finish clearing out the size/last/end switching by converting
amd_iommu_domain_flush_pages() to use last-based logic.
This algorithm is simpler than the previous. Ultimately all this wants
to do is select powers of two that are aligned to address and not
longer than the distance to last.
The new version is fully safe for size = U64_MAX and last = U64_MAX.
Finally, the gather can be passed through natively without risking an
overflow in (gather->end - gather->start + 1).
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Wei Wang <wei.w.wang@hotmail.com>
Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
This is the trivial call chain below amd_iommu_domain_flush_pages().
Cases that are doing a full invalidate will pass a last of U64_MAX.
This avoids converting between size and last, and type confusion with
size_t, unsigned long and u64 all being used in different places along
the driver's invalidation path. Consistently use u64 in the internals.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
This function is doing more work than it needs to:
- iommu_num_pages() is pointless, the fls() is going to compute the
required page size already.
- It is easier to understand as sz_lg2, which is 12 if size is 4K,
than msb_diff which is 11 if size is 4K.
- Simplify the control flow to early exit on the out of range cases.
- Use the usual last instead of end to signify an inclusive last
address.
- Use GENMASK to compute the 1's mask.
- Use GENMASK to compute the address mask for the command layout,
not PAGE_MASK.
- Directly reference the spec language that defines the 52 bit
limit.
No functional change intended.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Wei Wang <wei.w.wang@hotmail.com>
Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fix from Marek Szyprowski:
"Three more fixes for the DMA-mapping code, related to PCI P2PDMA, DMA
debug and DMA link ranges API (Li RongQing and Jason Gunthorpe)"
* tag 'dma-mapping-7.1-2026-06-11' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
iommu/dma: Do not try to iommu_map a 0 length region in swiotlb
dma-debug: fix physical address retrieval in debug_dma_sync_sg_for_device
dma-mapping: direct: fix missing mapping for THRU_HOST_BRIDGE segments
|
|
A comment in drivers/iommu/apple-dart.c incorrectly refers to
CONFIG_PCI_APPLE instead of CONFIG_PCIE_APPLE. Correct it.
Discovered while searching for CONFIG_* symbols referenced in code but
not defined in any Kconfig file.
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Reviewed-by: Sven Peter <sven@kernel.org>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommu_dma_iova_link_swiotlb() processes a mapping that is unaligned in three
parts, the head, middle and trailer. If the middle is empty because there
are no aligned pages it will call down to iommu_map() with a 0 size
which the iommupt implementation will fail as illegal.
It then tries to do an error unwind and starts from the wrong spot
corrupting the mapping so the eventual destruction triggers a WARN_ON.
Check for 0 length and avoid mapping and use offset not 0 as the starting
point to unlink.
This is frequently triggered by using some kinds of thunderbolt NVMe
drives that trigger forced SWIOTLB for unaligned memory. NVMe seems to
pass in oddly aligned buffers for the passthrough commands from smartctl
that hit this condition.
Cc: stable@vger.kernel.org
Fixes: 433a76207dcf ("dma-mapping: Implement link/unlink ranges API")
Reported-by: Mark Lord <mlord@pobox.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Link: https://lore.kernel.org/r/0-v1-8536728bc89f+469-swiotlb_warn_jgg@nvidia.com
|
|
Sashiko points out this has gotten out of order, the mutex could still be
in use through the dmabuf invalidation callbacks. Don't destroy any of the
pages content until the dmabuf is fully detached.
Fixes: 71db84a092c3 ("iommufd: Add DMABUF to iopt_pages")
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
dma_buf_unpin() requires the caller to hold the exporter's dma_resv
lock:
void dma_buf_unpin(struct dma_buf_attachment *attach)
{
...
dma_resv_assert_held(dmabuf->resv);
...
}
iopt_release_pages() calls dma_buf_unpin() without taking that lock,
so every iommufd_ioas_destroy()/iommufd_ioas_unmap() that releases
the last reference on a DMABUF-backed iopt_pages triggers a WARN.
This was hit while running tools/testing/selftests/iommu/iommufd:
WARNING: drivers/dma-buf/dma-buf.c:1137 at dma_buf_unpin+0x62/0x70
RIP: 0010:dma_buf_unpin+0x62/0x70
Call Trace:
<TASK>
dma_buf_unpin+0x62/0x70
iopt_release_pages+0xe4/0x190
iopt_unmap_iova_range+0x1c7/0x290
iopt_unmap_all+0x1a/0x30
iommufd_ioas_destroy+0x1d/0x50
iommufd_fops_release+0x93/0x150
__fput+0xfc/0x2c0
__x64_sys_close+0x3d/0x80
do_syscall_64+0x65/0x180
</TASK>
Take the dma_resv lock around dma_buf_unpin() in iopt_release_pages(),
matching the iopt_map_dmabuf() convention. dma_buf_detach() acquires the
reservation lock internally, so it must remain outside the locked region.
Fixes: 8c5f9645c389 ("iommufd: Add dma_buf_pin()")
Link: https://patch.msgid.link/r/20260526111034.4079-1-Ankit.Soni@amd.com
Reported-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Create hyperv subdirectory under drivers/iommu in anticipation of more
Hyper-V related files from upcoming PCI passthrough and PV-IOMMU patches.
Also, the current file hyperv-iommu.c actually implements irq remapping on
x86, so rename to more appropriate hv-irq-remap-x86.c and move it under
the new hyperv subdirectory. Since this file implements irq_remap_ops
exposed by drivers/iommu/irq_remapping.h, it cannot be relocated to the
irq directory. This is in sync with other backend directories like amd
and intel there.
Lastly, this file should not be tied to CONFIG_HYPERV_IOMMU, but to
CONFIG_HYPERV and CONFIG_IRQ_REMAP.
Signed-off-by: Mukesh R <mrathor@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
|
|
The cookie returned by xa_alloc() in iommufd_fault_fops_read() is per fault
group, but the inner copy_to_user() runs per fault inside the group. If a
copy fails mid-group, xa_erase clears the cookie and the group is restored
to the deliver list, yet done is not rolled back. The function returns the
partial byte count, with the successfully copied faults sitting at offsets
below done carrying the now-erased cookie. The next read() then re-fetches
the group, allocates a fresh cookie, and re-delivers every fault including
the ones already copied; userspace sees duplicates carrying the new cookie,
and a stale cookie that can never be responded to.
Use a local group_done variable that tracks the per-group progress inside
the inner loop, and only commit done = group_done after the inner loop has
finished successfully. On a copy_to_user failure the outer break skips the
commit, so done remains at its prior start-of-group baseline; the partial
bytes already written past done are undefined to userspace per the read(2)
contract, and the next read re-delivers the whole group atomically.
Fixes: 07838f7fd529 ("iommufd: Add iommufd fault object")
Link: https://patch.msgid.link/r/360cab4d4aeccb0bae275a970e2b3c340a71e0e0.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
On a copy_to_user() failure inside the inner list_for_each_entry, only the
inner loop breaks; the outer while re-fetches the just-restored fault group
and retries the failing copy_to_user() forever, spinning the reader at 100%
CPU with fault->mutex held.
Check rc after the inner loop and break the outer while as well.
Fixes: 07838f7fd529 ("iommufd: Add iommufd fault object")
Link: https://patch.msgid.link/r/336a9b6e44fe66a24199d3be777c405c85c98622.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The read count must be large enough to hold one fault or a group's faults.
iommufd_fault_fops_read() does not validate the count, but returns 0 as if
the read had succeeded while leaving the pending fault in the queue.
Return -EINVAL in the undersize cases.
Fixes: 07838f7fd529 ("iommufd: Add iommufd fault object")
Link: https://patch.msgid.link/r/85c118a606fbedc5c132a1f5ec223a5ba23b92d2.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
When the kzalloc_obj() fails in iommufd_veventq_deliver_fetch(), it returns
NULL, falsely advertising to userspace that the queue is empty.
Propagate the -ENOMEM properly to the caller.
Fixes: e36ba5ab808e ("iommufd: Add IOMMUFD_OBJ_VEVENTQ and IOMMUFD_CMD_VEVENTQ_ALLOC")
Link: https://patch.msgid.link/r/25d29feac909e36f78c145fa99ef2d4cb7a415da.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The read count must be large enough to hold a vEVENT header. For a normal
vEVENT, it must also hold the trailing data following the header.
iommufd_veventq_fops_read() does not validate the count, but returns 0 as
if the read had succeeded while leaving the pending event in the queue.
Return -EINVAL in both undersize cases.
Fixes: e36ba5ab808e ("iommufd: Add IOMMUFD_OBJ_VEVENTQ and IOMMUFD_CMD_VEVENTQ_ALLOC")
Link: https://patch.msgid.link/r/e1111adcc8a8882fbfd84accd6674dc846dc5689.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
When the first event copy fails, rc = -EFAULT will not be reported as done
is set to the length of the copied header.
Rewind it to report rc correctly.
Fixes: e36ba5ab808e ("iommufd: Add IOMMUFD_OBJ_VEVENTQ and IOMMUFD_CMD_VEVENTQ_ALLOC")
Link: https://patch.msgid.link/r/78f8caeb6a5d667a26b870e3068cec47dd4b5be1.1780343944.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The info->node RB-tree member is zero-initialized via kzalloc. If
a device does not support ATS, the device_rbtree_insert() call is
skipped. If a subsequent probe step fails, the error path jumps to
device_rbtree_remove(), which misinterprets the zeroed node as
a tree root and corrupts the device RB-tree.
Fix this by explicitly initializing the RB-node as empty using
RB_CLEAR_NODE() during initialization and guarding the removal with
RB_EMPTY_NODE().
Fixes: 4f1492efb495 ("iommu/vt-d: Revert ATS timing change to fix boot failure")
Reported-by: sashiko-bot@kernel.org
Closes: https://lore.kernel.org/all/20260525205628.CD4431F000E9@smtp.kernel.org/
Suggested-by: Baolu Lu <baolu.lu@linux.intel.com>
Signed-off-by: Pranjal Shrivastava <praan@google.com>
Link: https://lore.kernel.org/r/20260531170254.60493-2-praan@google.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
In some environments, multiple PCIe segments exist, and PCIe device
information needs to be differentiated and identified based on the
segment. When an IOMMU fault event occurs, the IOMMU and device segment
information should be output in detail in dmar_fault_do_one.
Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
Link: https://lore.kernel.org/r/20260528022943.1697564-1-guanghuifeng@linux.alibaba.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Rename pasid_pte_config_nestd() into pasid_pte_config_nested(). Do it to
match other function names ending with _nested().
Signed-off-by: Michał Grzelak <michal.grzelak@intel.com>
Link: https://lore.kernel.org/r/20260509174503.831134-1-michal.grzelak@intel.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
device_pasid_table_teardown() zeroes the 128-bit scalable-mode context
entry with context_clear_entry() while the Present bit is still set. This
creates a window where the hardware can fetch a torn entry, with some
fields already zeroed while Present is still set, leading to unpredictable
behavior or spurious faults. The context-cache invalidation is issued only
after the entry has been zeroed, and intel_pasid_free_table() then frees
the PASID directory pages, so the IOMMU can keep walking a stale Present=1
entry that points at freed memory.
While x86 provides strong write ordering, the compiler may reorder the two
64-bit writes to the entry, and the hardware fetch is not guaranteed to be
atomic with respect to multiple CPU writes.
Commit c1e4f1dccbe9d ("iommu/vt-d: Clear Present bit before tearing down
context entry") fixed this exact pattern in domain_context_clear_one() and
the copied-context path, but device_pasid_table_teardown() was not
converted.
Align it with the "Guidance to Software for Invalidations" in the VT-d
spec, Section 6.5.3.3, using the same ownership handshake as the sibling
fix: clear only the Present bit, flush it to the IOMMU, perform the
context-cache invalidation, and only then zero the rest of the entry.
Fixes: 81e921fd32161 ("iommu/vt-d: Fix NULL domain on device release")
Signed-off-by: Michael Bommarito <michael.bommarito@gmail.com>
Assisted-by: Claude:claude-opus-4-7
Link: https://lore.kernel.org/r/20260528025557.3209367-1-michael.bommarito@gmail.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The Intel IOMMU driver allows SVA on devices even if they do not support
PCI/PRI. Commit 39c20c4e83b9 ("iommu/vt-d: Only handle IOPF for SVA when
PRI is supported") modified the SVA bind path to allow this configuration
by skipping IOPF enablement when PRI is missing. However, it failed to
update the unbind path.
This creates an imbalance: the unbind path attempts to disable IOPF for
a device that never had it enabled, triggering a WARNING in
intel_iommu_disable_iopf():
WARNING: drivers/iommu/intel/iommu.c:3475 at intel_iommu_disable_iopf+0x4f/0x90d
Call Trace:
<TASK>
blocking_domain_set_dev_pasid+0x50/0x70
iommu_detach_device_pasid+0x89/0xc0
iommu_sva_unbind_device+0x73/0x150
xe_vm_close_and_put+0x4d2/0x1200 [xe]
Fix this by bypassing IOPF operations for SVA domains on non-PRI hardware
in both the bind and unbind paths.
Fixes: 39c20c4e83b9 ("iommu/vt-d: Only handle IOPF for SVA when PRI is supported")
Cc: stable@vger.kernel.org
Reported-by: Nareshkumar Gollakoti <naresh.kumar.g@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Link: https://lore.kernel.org/r/20260519052917.3729796-1-baolu.lu@linux.intel.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
We have observed multiple full invalidations occurring during device
detach when we are done using the vfio-device.
blocked_domain_attach_device()
-> detach_device()
-> amd_iommu_domain_flush_all()
-> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS)
while (size != 0) {
-> __domain_flush_pages( flush_size /* power of 2 flush_size */)
-> domain_flush_pages_v1()
-> build_inv_iommu_pages()
-> build_inv_address()
}
build_inv_address() will trigger a full invalidation if the chunk
size > (1 << 51). Consequently, the guest will issue multiple full
invalidations for a single call to amd_iommu_domain_flush_all()
Without this patch, we will see 10 time instead of 1 time full
invalidations for every amd_iommu_domain_flush_all().
Cc: stable@vger.kernel.org
Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
Suggested-by: Josef Bacik <josef@toxicpanda.com>
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Weinan Liu <wnliu@google.com>
Reviewed-by: Wei Wang <wei.w.wang@hotmail.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Disable the Bit 31 of the AUTO_GATING iommu register, as it causes
hangups with the RGA3 (Raster Graphics Acceleration 3) peripheral.
The RGA3 register description of the TRM already states that the bit
must be set to 1. The vendor kernel sets the bit unconditionally to
1 to fix VOP (Video Output Processor) screen black issues. This patch
squashes the 2 vendor kernel commits with the following commit messages:
Master fetch data and cpu update page table may work in parallel, may
have the following procedure:
master cpu
fetch dte update page tabl
| |
(make dte invalid) <- zap iotlb entry
| |
fetch dte again
(make dte invalid) <- zap iotlb entry
| |
fetch dte again
(make dte invalid) <- zap iotlb entry
| |
fetch dte again
(make iommu block) <- zap iotlb entry
New iommu version has the above bug, if fetch dte consecutively four
times, then it will be blocked. Fortunately, we can set bit 31 of
register MMU_AUTO_GATING to 1 to make it work as old version which does
not have this issue.
This issue only appears on RV1126 so far, so make a workaround dedicated
to "rockchip,rv1126" machine type.
iommu/rockchip: fix vop blocked and screen black on RK356X and RK3588
RK3568 and RK3588 has the same issue as RV1126/RV1109 that caused by
dte fetch time limit, So we can set BIT(31) of register 0x24 default
to 1 as a workaround.
Signed-off-by: Simon Xue <xxm@rock-chips.com>
Signed-off-by: Sven Püschel <s.pueschel@pengutronix.de>
Acked-by: Heiko Stuebner <heiko@sntech.de>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommufd_veventq_alloc() accepts any !0 veventq_depth from userspace, with
an upper bound at U32_MAX.
This leaves a vulnerability where userspace can allocate excessively large
queues to exhaust kernel memory reserves.
Cap the veventq_depth (maximum number of entries) to 1 << 19, matching the
maximum number of entries in the SMMUv3 EVTQ (the largest use case today).
Fixes: e36ba5ab808e ("iommufd: Add IOMMUFD_OBJ_VEVENTQ and IOMMUFD_CMD_VEVENTQ_ALLOC")
Link: https://patch.msgid.link/r/8426cbaa5e8294472ec7f076ef427cc473be5985.1779408671.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The veventq memory allocation happens inside the spinlock. Given its depth
is decided by the user space, this leaves a vulnerability, where userspace
can allocate large queues to exhaust atomic memory reserves.
Move the allocation outside the spinlock and use GFP_NOWAIT, which can fail
fast under memory pressure without dipping into the GFP_ATOMIC reserves or
direct-reclaiming from the threaded IRQ handler. On allocation failure,
queue the lost_events_header (so userspace learns of the drop) and return
-ENOMEM so the caller learns of the kernel-side memory pressure.
This is intentionally distinct from the queue-overflow path, which also
queues the lost_events_header but returns 0: a full queue is an expected
userspace-pacing condition rather than a kernel error.
A subsequent change will cap the upper bound of the veventq_depth.
Fixes: e36ba5ab808e ("iommufd: Add IOMMUFD_OBJ_VEVENTQ and IOMMUFD_CMD_VEVENTQ_ALLOC")
Link: https://patch.msgid.link/r/5ff36b5d80f7f6299f851be532a5195c1d2f1dae.1779408671.git.nicolinc@nvidia.com
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
