linux-toradex.git/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c, branch master Linux kernel for Apalis and Colibri modules drm/amdgpu: only tear down an active UALink accelerator 2026-11-04T17:08:34+00:00 Mukul Joshi mukul.joshi@amd.com 2026-10-21T19:45:32+00:00 05f51b3dedc1ea52422f65a2d948ee82feb9c6d5 AMDGPU_UALINK_ACCEL_STATE_ERROR is ordered after ACTIVE, so the "state < ACTIVE" check in deactivate_accelerator() lets an accelerator in the error state through to the full teardown. The NPA VM and workqueue are only created when the accelerator is activated, and an accelerator in the error state either never got there (invalid pPod or vPod configuration) or was already deactivated on the config update error path. Tearing it down again calls destroy_workqueue() on a NULL or freed npa_wq and amdgpu_vm_fini() on an uninitialized or freed NPA VM. This is reachable from driver unload (hw_fini), a later failing config update, removal from the vPod and a pPod setup commit. Only deactivate an accelerator that is actually active. Fixes: 0e8eb0dcea51 ("drm/amdgpu: Assign accel state based on ASP config") Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 0abb988dd7162e56e66e36c55780035be7d34b94) 
AMDGPU_UALINK_ACCEL_STATE_ERROR is ordered after ACTIVE, so the
"state < ACTIVE" check in deactivate_accelerator() lets an accelerator
in the error state through to the full teardown. The NPA VM and
workqueue are only created when the accelerator is activated, and an
accelerator in the error state either never got there (invalid pPod or
vPod configuration) or was already deactivated on the config update
error path.

Tearing it down again calls destroy_workqueue() on a NULL or freed
npa_wq and amdgpu_vm_fini() on an uninitialized or freed NPA VM. This
is reachable from driver unload (hw_fini), a later failing config
update, removal from the vPod and a pPod setup commit.

Only deactivate an accelerator that is actually active.

Fixes: 0e8eb0dcea51 ("drm/amdgpu: Assign accel state based on ASP config")
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 0abb988dd7162e56e66e36c55780035be7d34b94)
drm/amdgpu: Enable remote TLB shootdown 2026-10-16T22:01:16+00:00 Philip Yang Philip.Yang@amd.com 2026-08-28T00:06:06+00:00 948baa3fc47b01fc20d4beefd5cd176c11c1457c FW remote TLB shootdown support is ready now. Remove the early-return stub so the command is actually sent to the peer GPU. This requires nHT firmware version >= 1.2.E.1 (IFWI >= BRP0660). Signed-off-by: Philip Yang <Philip.Yang@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit f1be774d6d0128346fea1dc616fdf623c898c82c) 
FW remote TLB shootdown support is ready now. Remove the early-return stub
so the command is actually sent to the peer GPU.

This requires nHT firmware version >= 1.2.E.1 (IFWI >= BRP0660).

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit f1be774d6d0128346fea1dc616fdf623c898c82c)
drm/amdgpu: Reject UALink source-identification address mode 2026-10-16T22:01:16+00:00 Philip Yang Philip.Yang@amd.com 2026-08-26T21:22:53+00:00 2304e9596f1d6021811335c054be2c7567d6ca38 Source-identification address mode has never been tested or enabled, so reject it in __check_vpod_info() instead of running with an untested config. Both the inband and the sideband vpod config paths go through this validation. The SOURCE_IDENT handling in the rest of the driver is left in place for when the mode is enabled. Also drop r from the vpod-commit error message, __check_vpod_info() returns bool, so r is uninitialized there. v2: use switch to check address mode. Signed-off-by: Philip Yang <Philip.Yang@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit a925a168ba7e8afc06884058dfdd0cf0dc8995ca) 
Source-identification address mode has never been tested or enabled, so
reject it in __check_vpod_info() instead of running with an untested
config. Both the inband and the sideband vpod config paths go through
this validation. The SOURCE_IDENT handling in the rest of the driver is
left in place for when the mode is enabled.

Also drop r from the vpod-commit error message, __check_vpod_info()
returns bool, so r is uninitialized there.

v2: use switch to check address mode.

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit a925a168ba7e8afc06884058dfdd0cf0dc8995ca)
drm/amdgpu: Fix NPA-REVOKE racing an in-flight UALink import 2026-10-07T16:12:19+00:00 David Carlier devnexen@gmail.com 2026-09-29T19:34:32+00:00 f9c41782e42899d8d882b0d547db40c882205799 The exporter records an importer when it answers NPA-REQ, so it can send NPA-REVOKE as soon as the BO is freed, before the importer has finished building the dma-buf for that handle. The revoke handler assumes a fully imported node: it dereferences imp_xa_node->dmabuf, which is still NULL until the import completes, and drops the xarray reference the importing thread still relies on. The importer then links the node and marks it READY regardless, so the node can be freed while still on the per-remote list. Only tear down a node that is READY. A PENDING node is still being imported and nothing has been handed to user-space yet, so only mark it for teardown and send NPA-RELEASE. The importer checks for teardown under the xarray lock before linking the node and marking it READY, and unwinds otherwise. As NPA-REVOKE always follows NPA-RSP, a revoke that finds the node NOT_READY is stale, and one that finds it in teardown hits a node that is already being released, so both are ignored. Fixes: 7cc82cd90d35 ("drm/amdgpu: Implement mechanism to revoke exported memory") Assisted-by: LLM Signed-off-by: David Carlier <devnexen@gmail.com> Reviewed-by: Mukul Joshi <mukul.joshi@amd.com> Signed-off-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
The exporter records an importer when it answers NPA-REQ, so it can send
NPA-REVOKE as soon as the BO is freed, before the importer has finished
building the dma-buf for that handle. The revoke handler assumes a fully
imported node: it dereferences imp_xa_node->dmabuf, which is still NULL
until the import completes, and drops the xarray reference the importing
thread still relies on. The importer then links the node and marks it
READY regardless, so the node can be freed while still on the per-remote
list.

Only tear down a node that is READY. A PENDING node is still being
imported and nothing has been handed to user-space yet, so only mark it
for teardown and send NPA-RELEASE. The importer checks for teardown under
the xarray lock before linking the node and marking it READY, and unwinds
otherwise. As NPA-REVOKE always follows NPA-RSP, a revoke that finds the
node NOT_READY is stale, and one that finds it in teardown hits a node
that is already being released, so both are ignored.

Fixes: 7cc82cd90d35 ("drm/amdgpu: Implement mechanism to revoke exported memory")
Assisted-by: LLM
Signed-off-by: David Carlier <devnexen@gmail.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: mark A+A VF VRAM mappings as system memory 2026-10-07T16:02:46+00:00 Zhigang Luo Zhigang.Luo@amd.com 2026-08-31T19:39:03+00:00 010e510347f522f5c83c5bd1124b5de9c7f52c27 On the A+A VF the GPU reaches its HBM over the system memory path at a system physical address, so page table entries covering VRAM have to carry AMDGPU_PTE_SYSTEM and the addresses they hold have to be physical rather than MC addresses. Set the SYSTEM bit in the PDB0 entries and in amdgpu_ttm_tt_pde_flags(), and convert the VRAM address in amdgpu_gmc_get_pde_for_bo() with amdgpu_gmc_vram_mc2pa(). NPA mappings still require VSCT=0011 (PTE.S=0). Clear SYSTEM in gmc_v12_1_get_npa_flags() and amdgpu_ualink_peer_remote_init() so NPA PTEs do not inherit the VRAM SYSTEM bit. v2: - Use (sriov_vf || passthrough) and amdgpu_virt_vram_is_spm() for PDB0/TTM SYSTEM bits and mc2pa. v3: - Rely on amdgpu_virt_vram_is_spm() without redundant VF/passthrough checks. v4: - Explicitly clear AMDGPU_PTE_SYSTEM for NPA (VSCT=0011). Signed-off-by: Zhigang Luo <zhigang.luo@amd.com> Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com> Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com> Reviewed-by: Philip Yang <philip.yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
On the A+A VF the GPU reaches its HBM over the system memory path at a
system physical address, so page table entries covering VRAM have to
carry AMDGPU_PTE_SYSTEM and the addresses they hold have to be physical
rather than MC addresses.

Set the SYSTEM bit in the PDB0 entries and in amdgpu_ttm_tt_pde_flags(),
and convert the VRAM address in amdgpu_gmc_get_pde_for_bo() with
amdgpu_gmc_vram_mc2pa().

NPA mappings still require VSCT=0011 (PTE.S=0). Clear SYSTEM in
gmc_v12_1_get_npa_flags() and amdgpu_ualink_peer_remote_init() so NPA
PTEs do not inherit the VRAM SYSTEM bit.

v2:
- Use (sriov_vf || passthrough) and amdgpu_virt_vram_is_spm() for
  PDB0/TTM SYSTEM bits and mc2pa.

v3:
- Rely on amdgpu_virt_vram_is_spm() without redundant VF/passthrough
  checks.

v4:
- Explicitly clear AMDGPU_PTE_SYSTEM for NPA (VSCT=0011).

Signed-off-by: Zhigang Luo <zhigang.luo@amd.com>
Signed-off-by: Fangsheng Huang <Fangsheng.Huang@amd.com>
Signed-off-by: YuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: Philip Yang <philip.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: Fix NPA BO memory accounting stats during import invalidation 2026-10-01T16:17:51+00:00 Mukul Joshi mukul.joshi@amd.com 2026-09-02T19:13:07+00:00 222627765a4699b9d56c8f6b7f2952905663a8c0 Use amdgpu_vm_bo_move() for updating the NPA BO memory accounting stats when the NPA BO is moved out from NPA domain into an empty placement. Without this, when the app finishes, it spits out a warning: "VM memory stats for proc .. task .. is non-zero when fini" This was only affecting the memory accounting stats, we were not leaking any memory here. Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Acked-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
Use amdgpu_vm_bo_move() for updating the NPA BO memory accounting stats
when the NPA BO is moved out from NPA domain into an empty placement.
Without this, when the app finishes, it spits out a warning:
"VM memory stats for proc .. task .. is non-zero when fini"

This was only affecting the memory accounting stats, we were not leaking
any memory here.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Acked-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: re-check connection state before sending NPA-REVOKE 2026-10-01T16:17:48+00:00 Mukul Joshi mukul.joshi@amd.com 2026-09-22T20:41:57+00:00 c4d9c64c11427ba88448c8c54b89bcfb13da7a29 The export cleanup worker checks amdgpu_ualink_check_conn_ready() once when building npa_release_bitmap, but not again in the send loop. Cleanup workers run concurrently on the unbounded NPA WQ (one per exported handle), so when a peer's egress dies many workers have already seeded npa_release_bitmap from a connection that was still ESTABLISHED and all issue an LSDMA copy to the dead peer, flooding the log with repeated NPA-REVOKE send failures. Re-check the connection immediately before each send, as the tlb shootdown, force_retry_rpcs and unmap_all_npa_addr loops already do, so once the first worker marks a peer NOT_READY the rest skip it. With the re-check in place, seed npa_release_bitmap with a single bitmap_copy() of importers_bitmap instead of a separate build loop; peers that fail the re-check or the send are dropped from the bitmap in the send loop. Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
The export cleanup worker checks amdgpu_ualink_check_conn_ready() once
when building npa_release_bitmap, but not again in the send loop. Cleanup
workers run concurrently on the unbounded NPA WQ (one per exported
handle), so when a peer's egress dies many workers have already seeded
npa_release_bitmap from a connection that was still ESTABLISHED and all
issue an LSDMA copy to the dead peer, flooding the log with repeated
NPA-REVOKE send failures.

Re-check the connection immediately before each send, as the tlb
shootdown, force_retry_rpcs and unmap_all_npa_addr loops already do, so
once the first worker marks a peer NOT_READY the rest skip it. With the
re-check in place, seed npa_release_bitmap with a single bitmap_copy() of
importers_bitmap instead of a separate build loop; peers that fail the
re-check or the send are dropped from the bitmap in the send loop.

Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: Fix race on ualink_handle_lo in UALink handle export 2026-10-01T16:09:34+00:00 Mukul Joshi mukul.joshi@amd.com 2026-09-16T02:21:36+00:00 d0185861e20a93a9b398cbd85870686467443fa9 amdgpu_ualink_export_handle() tests robj->ualink_handle_lo and stores the generated handle into it without any lock; xa_lock only covers handle generation and the xarray insert. Two threads can therefore both observe a zero handle for the same BO and both allocate an exporter node, take an amdgpu_bo_ref() and insert into exp_xa, with the last store to ualink_handle_lo winning. Teardown only erases the handle recorded in the BO, so the losing node keeps its BO reference and pinned VRAM until device teardown. Each racing thread still generates its own handle and inserts its own node, but the handle is published to the BO under the BO reservation lock after re-checking that ualink_handle_lo is still zero. Exactly one thread makes that transition; the others find a non-zero value, roll back their node (erase from exp_xa, drop the DMABuf and BO references, free it) and return -EAGAIN so the caller retries and picks up the winning handle. This keeps a single exporter node per BO. Fixes: c67cb19811ec ("drm/amdgpu: Implement UALink handle export") Reported-by: Sashiko-bot <sashiko-bot@kernel.org> Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1 Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
amdgpu_ualink_export_handle() tests robj->ualink_handle_lo and stores
the generated handle into it without any lock; xa_lock only covers
handle generation and the xarray insert.

Two threads can therefore both observe a zero handle for the same BO and
both allocate an exporter node, take an amdgpu_bo_ref() and insert into
exp_xa, with the last store to ualink_handle_lo winning. Teardown only
erases the handle recorded in the BO, so the losing node keeps its BO
reference and pinned VRAM until device teardown.

Each racing thread still generates its own handle and inserts its own
node, but the handle is published to the BO under the BO reservation
lock after re-checking that ualink_handle_lo is still zero. Exactly one
thread makes that transition; the others find a non-zero value, roll
back their node (erase from exp_xa, drop the DMABuf and BO references,
free it) and return -EAGAIN so the caller retries and picks up the
winning handle. This keeps a single exporter node per BO.

Fixes: c67cb19811ec ("drm/amdgpu: Implement UALink handle export")
Reported-by: Sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: Fix BO reference leak in UALink export error path 2026-10-01T15:59:30+00:00 Mukul Joshi mukul.joshi@amd.com 2026-09-16T02:17:42+00:00 b80987c0904a60e63f37f5e053679376257109c0 amdgpu_ualink_export_handle() takes an explicit amdgpu_bo_ref() before building the exporter node, and the out: label only drops the reference from drm_gem_object_lookup(). The xarray insert failure path unrefs the BO, but the drm_gem_prime_handle_to_dmabuf() failure path does not. Userspace can hit that path at will, for example by exhausting its file descriptor table so the dmabuf export returns -EMFILE, leaking a BO reference and its pinned VRAM on every attempt. Fixes: c67cb19811ec ("drm/amdgpu: Implement UALink handle export") Reported-by: Sashiko-bot <sashiko-bot@kernel.org> Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1 Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
amdgpu_ualink_export_handle() takes an explicit amdgpu_bo_ref() before
building the exporter node, and the out: label only drops the reference
from drm_gem_object_lookup(). The xarray insert failure path unrefs the
BO, but the drm_gem_prime_handle_to_dmabuf() failure path does not.

Userspace can hit that path at will, for example by exhausting its file
descriptor table so the dmabuf export returns -EMFILE, leaking a BO
reference and its pinned VRAM on every attempt.

Fixes: c67cb19811ec ("drm/amdgpu: Implement UALink handle export")
Reported-by: Sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm/amdgpu: Reject UALink handle ioctl on devices without UALink 2026-10-01T15:31:21+00:00 Mukul Joshi mukul.joshi@amd.com 2026-09-16T02:17:02+00:00 0799087e2712fcc9efd849e3c99298ee2abac827 amdgpu_gem_ualink_handle_ioctl() dereferences adev->ualink.info before validating it, but that structure is only allocated when the UALink IP block is present, which is currently added only for GFX 12.1 only. The ioctl is registered for every amdgpu device with DRM_RENDER_ALLOW, so on any other ASIC a render node user can oops the kernel with a single call. Return -EOPNOTSUPP when the UALink software state is absent. Fixes: 15d1c306d829 ("drm/amdgpu: Add ioctl infra for exporting/importing UALink handles") Reported-by: Sashiko-bot <sashiko-bot@kernel.org> Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1 Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> 
amdgpu_gem_ualink_handle_ioctl() dereferences adev->ualink.info before
validating it, but that structure is only allocated when the UALink IP
block is present, which is currently added only for GFX 12.1 only. The
ioctl is registered for every amdgpu device with DRM_RENDER_ALLOW, so
on any other ASIC a render node user can oops the kernel with a single
call.

Return -EOPNOTSUPP when the UALink software state is absent.

Fixes: 15d1c306d829 ("drm/amdgpu: Add ioctl infra for exporting/importing UALink handles")
Reported-by: Sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260914054604.12850-1-kmehltretter@gmail.com?part=1
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>