| Age | Commit message (Collapse) | Author |
|---|
|
Add the ioctl infrastructure to support exporting and importing BOs
to facilitate NPA based memory sharing across GPUs in a rack scale
setup.
Proposed userspace:
https://github.com/ROCm/rocm-systems/blob/35959f8e1260c7cee3e51a740e320be3856ee4ff/projects/rocr-runtime/libhsakmt/src/memory.c#L971
https://github.com/ROCm/rocm-systems/blob/35959f8e1260c7cee3e51a740e320be3856ee4ff/projects/rocr-runtime/libhsakmt/src/memory.c#L1036
v2: Move the ioctl wire-up to the end of the series.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Horatio Zhang <hongkun.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Mirror the read-only ualink info attributes on each secondary compute
partition as a per-partition ualink node, so a partition-scoped consumer
(e.g. a container that only sees its partition's device node) can read
the ualink identity and state. Partition 0 shares the primary device,
which already exposes that node, so it is skipped. An inactive partition
device won't be having any attributes listed under ualink node.
The per-partition attributes are served by thin wrappers that delegate
to the existing device-level info show functions. A reference on the
info kobject is held for the node's lifetime so it cannot be freed
while a partition still uses it.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Assisted-by: Claude (claude-opus-4.7)
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Deactivate accelerator from vpod during hw_fini sequence.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Move more IP version specific related things to ualink version file.
Use soc v1.0 IH client id defintion and define mpnht interrupt source
id in ivsrcid header.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Keep state transitions under lock. Validate the ppod/vpod config or both
based on the state passed by ASP. When a config update is received, if
the GPU is already active on a vpod, local vpod gpu integrity check is
skipped to keep minimal disruption. A gpu removed from the vpod will get
the new vpod id as 0. A GPU is not expected to transition directly from a
valid/nonzero vpod id to another valid vpod id. It needs to be removed
from the existing vpod first.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add handlers for ualink notifications from ASP. Pause/Resume
notifications are not handled currently.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add explicit PPOD/VPOD states for clarity. Assign the state accordingly
as received from ASP during initialization.
UALink accel state transitions:
UNCONFIGURED -> PPOD_CONFIGURED -> VPOD_CONFIGURED -> READY -> ACTIVE
- PPOD: sysfs ppod commit/ ASP {UAL_CFG_PPOD}
- VPOD: sysfs vpod commit/ ASP {UAL_CFG_VPOD} | {UAL_CFG_STATION}
- READY: local vpod integrity + activate_accelerator()
- ACTIVE: all local gpus belonging to one vpod are configured and ready.
deactivate_accelerator(): READY -> PPOD_CONFIGURED
invalid vpod: -> ERROR
Query ualink info during late init phase. Add a local vpod integrity
check before activating devices in the vpod. Move the devices to active
state once integrity check passes.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Message handshake with NHT firmware is IP specific. Separate that into
IP specific file. Move ip block v1.0 definition also the version
specific file.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Manage ualink lifecycle as a separate IP block. For now, use GC version
to add specific ualink ip block.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Separate out sw_init and hw_init part of ualink. ualink init errors
are ignored temporarily. Keep ualink sysfs init together with sysfs init
sequence of other IPs.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Introduce a debugfs-controlled bitmap, amdgpu_ualink_drop_msg_bitmap,
where each bit corresponds to a value from
enum AMDGPU_UALINK_PROTOCOL_MESSAGES (e.g. NPA-REQ, NPA-RSP, NPA-FAIL,
NPA-REVOKE, NPA-RELEASE).
When a bit is set, the next incoming message of that type is dropped
in amdgpu_ualink_process_irq() and the bit is atomically cleared via
test_and_clear_bit(), so subsequent incoming messages are processed
normally. This allows exercising the UALink connection reset and
recovery paths (NPA-RSP timeout on the importer, retransmit/teardown
on the exporter, etc.) by injecting a single packet loss from
userspace, e.g.:
# drop one NPA-REQ
echo 0x08 > /sys/kernel/debug/dri/0/amdgpu_ualink_drop_msg_bitmap
# drop one NPA-RSP
echo 0x10 > /sys/kernel/debug/dri/0/amdgpu_ualink_drop_msg_bitmap
The bitmap lives in struct amdgpu_ualink_mgr and is exposed via
debugfs_create_ulong() next to the existing amdgpu_ualink_test entry.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Replace the stub implementation of TLB shootdown message with
the actual implementation.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add interrupt handling to process the NPA remote interrupts.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
This patch adds implementation to revoke exported memory.
The exporter GPU will send a NPA-REVOKE message to inform the
importer GPUs to stop using the exported memory. Once the importers
stop using the exporter memory, they would send back a NPA-RELEASE
message to inform the exporters to free the exporter memory.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The ualink handle import process involves NPA protocol
message exchange between the exporting and the importing
GPUs in the rack scale setup. The process is as follows:
1. On the importing GPU, check if connection is already setup
with the exporting GPU. If not, then exchange HELLO/HELLO_ACK
messages to setup the connection.
2. Once the connection is setup, then exchange NPA-REQ/NPA-RSP
messages with the exporter GPU to get the NPA address and size
of the BO associated with the ualink handle.
3. On the exporter GPU, upon receipt of NPA-REQ message, do basic
validation to check the ualink handle is valid. If not, send a
NPA-FAIL message back. If its valid, then using the address
allocator, allocate a NPA address, map it into the NPA VM and
send back the address and size in the NPA-RSP to the importing GPU.
4. On the importer GPU, upon receipt of NPA-RSP message, allocate a NPA
BO at the NPA address received in the NPA-RSP message.
5. Finally, generate the corresponding DMABuf for the NPA BO and return
the dmabuf to user-space.
6. We are using refcount to keep track of the importer/exporter xarray
entries. The cleanup functions for these are added in the subsequent
patches.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add functions for managing connection state between GPUs
before import/export of ualink handles occurs.
These functions handle the lifecycle of connections between
GPUs,including handling GPU resets, in the NPA-based memory
sharing model.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
This patch adds the implementation for exporting a ualink
handle associated with a BO. If the BO doesn't have a
ualink handle generated yet, we first allocate a unique
ualink handle, add it to a lookup table and return the
handle to user-space.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Create a drm client for storing the GEM handles corresponding
to NPA BOs. We always want to keep one GEM handle alive for
the NPA BOs during the lifetime of the BO.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Populate the NPA VM so remote GPUs can access local ring buffers and
write pointer pages at their reserved NPA addresses. Teardown waits for
DMA fences and issues a heavyweight TLB flush.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Extend the UALink manager with data structures for cross-GPU interrupt
and TLB shootdown: per-ring state, per-peer channel pairs, and the
top-level remote container.
Declare the public init, teardown, and reset APIs.
Define ring buffer capacity, firmware command and status codes, SMN
mailbox register addresses, and the metadata and writeback buffer layouts
shared between driver and MPNHT firmware.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Initialize a NPA VM for managing NPA addresses on the exporter
GPU.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add a address allocator, which uses DRM MM, for allocating and
deallocating NPA addresses for BOs exported/imported in a rack-scale
setup.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add a new address space for NPA address management.
This is needed for sharing buffer objects across GPUs
with each running their own OS. The NPA address space
size can change across different HW generations so the
size is initialized during early init of the driver boot
up process.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Query the initial UALink configuration info from PSP at driver
initialization time.
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
This patch introduces AMDGPU UALink manager core infrastructure
which will be responsible for sharing buffer objects across
multiple GPUs, with each GPU running their own OS, in a rack scale
setup. The core infrastructure of the ualink manager will include:
- Store BOs exported from a GPU.
- Store BOs imported by a GPU.
- Store connection state with different GPUs.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
<device>/ualink/stations: Directory for staging UALink station/port
configuration attributes:
- station_flags (currently only sets the number of port per station)
- lane_en_bitmap (hexadecimal bitmaps of lanes enabled per station)
- write "true" to commit to apply settings
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
<device>/ualink/config: Directory for staging virtual pod configuration
attributes. Write "true" to commit to apply settings
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
<device>/ualink/setup: Directory for staging physical pod setup
attributes. Write "true" to commit to apply settings
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
<device>/ualink/info: Directory contains current UALink attributes
- link_type: UALoE of UALink
- accel_state: unconfigured, configured, ready, active, error
- accel_id: Accelerator ID
- ppod_id: Physical pod ID (UUID)
- ppod_size: Number of accelerators in physical pod
- bandwidth: Estimated total bandwidth between pairs of accelerators
- latency: Estimated latency between pairs of accelerators
- vpod_id: Virtual pod ID
- vpod_size: Number of accelerators in virtual pod
- vpod_active_accels: List of all accelerator IDs in vpod
- local_accels: List of local accelerator IDs in vpod
- addr_mode: source-aliasing or source-identification
Signed-off-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Mukul Joshi <mukul.joshi@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
