From 165e77353831a85caa0444d16f29bd6b111dd2c5 Mon Sep 17 00:00:00 2001 From: Sean Christopherson Date: Fri, 13 Feb 2026 17:26:57 -0800 Subject: KVM: x86/tdx: Do VMXON and TDX-Module initialization during subsys init Now that VMXON can be done without bouncing through KVM, do TDX-Module initialization during subsys init (specifically before module_init() so that it runs before KVM when both are built-in). Aside from the obvious benefits of separating core TDX code from KVM, this will allow tagging a pile of TDX functions and globals as being __init and __ro_after_init. Reviewed-by: Dan Williams Reviewed-by: Chao Gao Acked-by: Dave Hansen Tested-by: Chao Gao Tested-by: Sagi Shahar Link: https://patch.msgid.link/20260214012702.2368778-12-seanjc@google.com Signed-off-by: Sean Christopherson --- Documentation/arch/x86/tdx.rst | 36 +++++------------------------------- 1 file changed, 5 insertions(+), 31 deletions(-) (limited to 'Documentation') diff --git a/Documentation/arch/x86/tdx.rst b/Documentation/arch/x86/tdx.rst index 61670e7df2f7..ff6b110291bc 100644 --- a/Documentation/arch/x86/tdx.rst +++ b/Documentation/arch/x86/tdx.rst @@ -60,44 +60,18 @@ Besides initializing the TDX module, a per-cpu initialization SEAMCALL must be done on one cpu before any other SEAMCALLs can be made on that cpu. -The kernel provides two functions, tdx_enable() and tdx_cpu_enable() to -allow the user of TDX to enable the TDX module and enable TDX on local -cpu respectively. - -Making SEAMCALL requires VMXON has been done on that CPU. Currently only -KVM implements VMXON. For now both tdx_enable() and tdx_cpu_enable() -don't do VMXON internally (not trivial), but depends on the caller to -guarantee that. - -To enable TDX, the caller of TDX should: 1) temporarily disable CPU -hotplug; 2) do VMXON and tdx_enable_cpu() on all online cpus; 3) call -tdx_enable(). For example:: - - cpus_read_lock(); - on_each_cpu(vmxon_and_tdx_cpu_enable()); - ret = tdx_enable(); - cpus_read_unlock(); - if (ret) - goto no_tdx; - // TDX is ready to use - -And the caller of TDX must guarantee the tdx_cpu_enable() has been -successfully done on any cpu before it wants to run any other SEAMCALL. -A typical usage is do both VMXON and tdx_cpu_enable() in CPU hotplug -online callback, and refuse to online if tdx_cpu_enable() fails. - User can consult dmesg to see whether the TDX module has been initialized. If the TDX module is initialized successfully, dmesg shows something like below:: [..] virt/tdx: 262668 KBs allocated for PAMT - [..] virt/tdx: module initialized + [..] virt/tdx: TDX-Module initialized If the TDX module failed to initialize, dmesg also shows it failed to initialize:: - [..] virt/tdx: module initialization failed ... + [..] virt/tdx: TDX-Module initialization failed ... TDX Interaction to Other Kernel Components ------------------------------------------ @@ -129,9 +103,9 @@ CPU Hotplug ~~~~~~~~~~~ TDX module requires the per-cpu initialization SEAMCALL must be done on -one cpu before any other SEAMCALLs can be made on that cpu. The kernel -provides tdx_cpu_enable() to let the user of TDX to do it when the user -wants to use a new cpu for TDX task. +one cpu before any other SEAMCALLs can be made on that cpu. The kernel, +via the CPU hotplug framework, performs the necessary initialization when +a CPU is first brought online. TDX doesn't support physical (ACPI) CPU hotplug. During machine boot, TDX verifies all boot-time present logical CPUs are TDX compatible before -- cgit v1.2.3 From b88d05a893cb7c8a48d03ff93d4aca95a6165377 Mon Sep 17 00:00:00 2001 From: Sascha Bischoff Date: Thu, 19 Mar 2026 15:56:43 +0000 Subject: KVM: arm64: gic-v5: Support GICv5 interrupts with KVM_IRQ_LINE Interrupts under GICv5 look quite different to those from older Arm GICs. Specifically, the type is encoded in the top bits of the interrupt ID. Extend KVM_IRQ_LINE to cope with GICv5 PPIs and SPIs. The requires subtly changing the KVM_IRQ_LINE API for GICv5 guests. For older Arm GICs, PPIs had to be in the range of 16-31, and SPIs had to be 32-1019, but this no longer holds true for GICv5. Instead, for a GICv5 guest support PPIs in the range of 0-127, and SPIs in the range 0-65535. The documentation is updated accordingly. The SPI range doesn't cover the full SPI range that a GICv5 system can potentially cope with (GICv5 provides up to 24-bits of SPI ID space, and we only have 16 bits to work with in KVM_IRQ_LINE). However, 65k SPIs is more than would be reasonably expected on systems for years to come. In order to use vgic_is_v5(), the kvm/arm_vgic.h header is added to kvm/arm.c. Note: As the GICv5 KVM implementation currently doesn't support injecting SPIs attempts to do so will fail. This restriction will by lifted as the GICv5 KVM support evolves. Co-authored-by: Timothy Hayes Signed-off-by: Timothy Hayes Signed-off-by: Sascha Bischoff Reviewed-by: Jonathan Cameron Link: https://patch.msgid.link/20260319154937.3619520-28-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/api.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 032516783e96..03d87d9b97d9 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -907,10 +907,12 @@ The irq_type field has the following values: - KVM_ARM_IRQ_TYPE_CPU: out-of-kernel GIC: irq_id 0 is IRQ, irq_id 1 is FIQ - KVM_ARM_IRQ_TYPE_SPI: - in-kernel GIC: SPI, irq_id between 32 and 1019 (incl.) + in-kernel GICv2/GICv3: SPI, irq_id between 32 and 1019 (incl.) (the vcpu_index field is ignored) + in-kernel GICv5: SPI, irq_id between 0 and 65535 (incl.) - KVM_ARM_IRQ_TYPE_PPI: - in-kernel GIC: PPI, irq_id between 16 and 31 (incl.) + in-kernel GICv2/GICv3: PPI, irq_id between 16 and 31 (incl.) + in-kernel GICv5: PPI, irq_id between 0 and 127 (incl.) (The irq_id field thus corresponds nicely to the IRQ ID in the ARM GIC specs) -- cgit v1.2.3 From 7c31c06e2d2d75859d773ba940e56d1db2bd1fcd Mon Sep 17 00:00:00 2001 From: Sascha Bischoff Date: Thu, 19 Mar 2026 15:58:01 +0000 Subject: KVM: arm64: gic-v5: Mandate architected PPI for PMU emulation on GICv5 Make it mandatory to use the architected PPI when running a GICv5 guest. Attempts to set anything other than the architected PPI (23) are rejected. Additionally, KVM_ARM_VCPU_PMU_V3_INIT is relaxed to no longer require KVM_ARM_VCPU_PMU_V3_IRQ to be called for GICv5-based guests. In this case, the architectued PPI is automatically used. Documentation is bumped accordingly. Signed-off-by: Sascha Bischoff Reviewed-by: Jonathan Cameron Reviewed-by: Joey Gouly Link: https://patch.msgid.link/20260319154937.3619520-33-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/devices/vcpu.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/virt/kvm/devices/vcpu.rst b/Documentation/virt/kvm/devices/vcpu.rst index 60bf205cb373..5e3805820010 100644 --- a/Documentation/virt/kvm/devices/vcpu.rst +++ b/Documentation/virt/kvm/devices/vcpu.rst @@ -37,7 +37,8 @@ Returns: A value describing the PMUv3 (Performance Monitor Unit v3) overflow interrupt number for this vcpu. This interrupt could be a PPI or SPI, but the interrupt type must be same for each vcpu. As a PPI, the interrupt number is the same for -all vcpus, while as an SPI it must be a separate number per vcpu. +all vcpus, while as an SPI it must be a separate number per vcpu. For +GICv5-based guests, the architected PPI (23) must be used. 1.2 ATTRIBUTE: KVM_ARM_VCPU_PMU_V3_INIT --------------------------------------- @@ -50,7 +51,7 @@ Returns: -EEXIST Interrupt number already used -ENODEV PMUv3 not supported or GIC not initialized -ENXIO PMUv3 not supported, missing VCPU feature or interrupt - number not set + number not set (non-GICv5 guests, only) -EBUSY PMUv3 already initialized ======= ====================================================== -- cgit v1.2.3 From eb3c4d2c9a4d76b775a9dbd5ac056d1abf0083a1 Mon Sep 17 00:00:00 2001 From: Sascha Bischoff Date: Thu, 19 Mar 2026 15:59:35 +0000 Subject: Documentation: KVM: Introduce documentation for VGICv5 Now that it is possible to create a VGICv5 device, provide initial documentation for it. At this stage, there is little to document. Signed-off-by: Sascha Bischoff Reviewed-by: Jonathan Cameron Link: https://patch.msgid.link/20260319154937.3619520-39-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/devices/arm-vgic-v5.rst | 37 ++++++++++++++++++++++++++ Documentation/virt/kvm/devices/index.rst | 1 + 2 files changed, 38 insertions(+) create mode 100644 Documentation/virt/kvm/devices/arm-vgic-v5.rst (limited to 'Documentation') diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst new file mode 100644 index 000000000000..9904cb888277 --- /dev/null +++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst @@ -0,0 +1,37 @@ +.. SPDX-License-Identifier: GPL-2.0 + +==================================================== +ARM Virtual Generic Interrupt Controller v5 (VGICv5) +==================================================== + + +Device types supported: + - KVM_DEV_TYPE_ARM_VGIC_V5 ARM Generic Interrupt Controller v5.0 + +Only one VGIC instance may be instantiated through this API. The created VGIC +will act as the VM interrupt controller, requiring emulated user-space devices +to inject interrupts to the VGIC instead of directly to CPUs. + +Creating a guest GICv5 device requires a host GICv5 host. The current VGICv5 +device only supports PPI interrupts. These can either be injected from emulated +in-kernel devices (such as the Arch Timer, or PMU), or via the KVM_IRQ_LINE +ioctl. + +Groups: + KVM_DEV_ARM_VGIC_GRP_CTRL + Attributes: + + KVM_DEV_ARM_VGIC_CTRL_INIT + request the initialization of the VGIC, no additional parameter in + kvm_device_attr.addr. Must be called after all VCPUs have been created. + + Errors: + + ======= ======================================================== + -ENXIO VGIC not properly configured as required prior to calling + this attribute + -ENODEV no online VCPU + -ENOMEM memory shortage when allocating vgic internal data + -EFAULT Invalid guest ram access + -EBUSY One or more VCPUS are running + ======= ======================================================== diff --git a/Documentation/virt/kvm/devices/index.rst b/Documentation/virt/kvm/devices/index.rst index 192cda7405c8..70845aba38f4 100644 --- a/Documentation/virt/kvm/devices/index.rst +++ b/Documentation/virt/kvm/devices/index.rst @@ -10,6 +10,7 @@ Devices arm-vgic-its arm-vgic arm-vgic-v3 + arm-vgic-v5 mpic s390_flic vcpu -- cgit v1.2.3 From d51c978b7d3e143381f871d28d8a0437d446b51b Mon Sep 17 00:00:00 2001 From: Sascha Bischoff Date: Thu, 19 Mar 2026 15:59:50 +0000 Subject: KVM: arm64: gic-v5: Communicate userspace-driveable PPIs via a UAPI GICv5 systems will likely not support the full set of PPIs. The presence of any virtual PPI is tied to the presence of the physical PPI. Therefore, the available PPIs will be limited by the physical host. Userspace cannot drive any PPIs that are not implemented. Moreover, it is not desirable to expose all PPIs to the guest in the first place, even if they are supported in hardware. Some devices, such as the arch timer, are implemented in KVM, and hence those PPIs shouldn't be driven by userspace, either. Provided a new UAPI: KVM_DEV_ARM_VGIC_GRP_CTRL => KVM_DEV_ARM_VGIC_USERPSPACE_PPIs This allows userspace to query which PPIs it is able to drive via KVM_IRQ_LINE. Additionally, introduce a check in kvm_vm_ioctl_irq_line() to reject any PPIs not in the userspace mask. Signed-off-by: Sascha Bischoff Reviewed-by: Jonathan Cameron Link: https://patch.msgid.link/20260319154937.3619520-40-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/devices/arm-vgic-v5.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'Documentation') diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst index 9904cb888277..29335ea823fc 100644 --- a/Documentation/virt/kvm/devices/arm-vgic-v5.rst +++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst @@ -25,6 +25,19 @@ Groups: request the initialization of the VGIC, no additional parameter in kvm_device_attr.addr. Must be called after all VCPUs have been created. + KVM_DEV_ARM_VGIC_USERPSPACE_PPIs + request the mask of userspace-drivable PPIs. Only a subset of the PPIs can + be directly driven from userspace with GICv5, and the returned mask + informs userspace of which it is allowed to drive via KVM_IRQ_LINE. + + Userspace must allocate and point to __u64[2] of data in + kvm_device_attr.addr. When this call returns, the provided memory will be + populated with the userspace PPI mask. The lower __u64 contains the mask + for the lower 64 PPIS, with the remaining 64 being in the second __u64. + + This is a read-only attribute, and cannot be set. Attempts to set it are + rejected. + Errors: ======= ======================================================== -- cgit v1.2.3 From 287c6981f12a008bafc46f18a3e48540a1172a52 Mon Sep 17 00:00:00 2001 From: Will Deacon Date: Mon, 30 Mar 2026 15:48:33 +0100 Subject: KVM: arm64: Add some initial documentation for pKVM Add some initial documentation for pKVM to help people understand what is supported, the limitations of protected VMs when compared to non-protected VMs and also what is left to do. Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba Tested-by: Mostafa Saleh Signed-off-by: Will Deacon Link: https://patch.msgid.link/20260330144841.26181-33-will@kernel.org Signed-off-by: Marc Zyngier --- Documentation/admin-guide/kernel-parameters.txt | 4 +- Documentation/virt/kvm/arm/index.rst | 1 + Documentation/virt/kvm/arm/pkvm.rst | 106 ++++++++++++++++++++++++ 3 files changed, 109 insertions(+), 2 deletions(-) create mode 100644 Documentation/virt/kvm/arm/pkvm.rst (limited to 'Documentation') diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 03a550630644..44854a67bc63 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3247,8 +3247,8 @@ Kernel parameters for the host. To force nVHE on VHE hardware, add "arm64_sw.hvhe=0 id_aa64mmfr1.vh=0" to the command-line. - "nested" is experimental and should be used with - extreme caution. + "nested" and "protected" are experimental and should be + used with extreme caution. kvm-arm.vgic_v3_group0_trap= [KVM,ARM,EARLY] Trap guest accesses to GICv3 group-0 diff --git a/Documentation/virt/kvm/arm/index.rst b/Documentation/virt/kvm/arm/index.rst index ec09881de4cf..0856b4942e05 100644 --- a/Documentation/virt/kvm/arm/index.rst +++ b/Documentation/virt/kvm/arm/index.rst @@ -10,6 +10,7 @@ ARM fw-pseudo-registers hyp-abi hypercalls + pkvm pvtime ptp_kvm vcpu-features diff --git a/Documentation/virt/kvm/arm/pkvm.rst b/Documentation/virt/kvm/arm/pkvm.rst new file mode 100644 index 000000000000..514992a79a83 --- /dev/null +++ b/Documentation/virt/kvm/arm/pkvm.rst @@ -0,0 +1,106 @@ +.. SPDX-License-Identifier: GPL-2.0 + +==================== +Protected KVM (pKVM) +==================== + +**NOTE**: pKVM is currently an experimental, development feature and +subject to breaking changes as new isolation features are implemented. +Please reach out to the developers at kvmarm@lists.linux.dev if you have +any questions. + +Overview +======== + +Booting a host kernel with '``kvm-arm.mode=protected``' enables +"Protected KVM" (pKVM). During boot, pKVM installs a stage-2 identity +map page-table for the host and uses it to isolate the hypervisor +running at EL2 from the rest of the host running at EL1/0. + +pKVM permits creation of protected virtual machines (pVMs) by passing +the ``KVM_VM_TYPE_ARM_PROTECTED`` machine type identifier to the +``KVM_CREATE_VM`` ioctl(). The hypervisor isolates pVMs from the host by +unmapping pages from the stage-2 identity map as they are accessed by a +pVM. Hypercalls are provided for a pVM to share specific regions of its +IPA space back with the host, allowing for communication with the VMM. +A Linux guest must be configured with ``CONFIG_ARM_PKVM_GUEST=y`` in +order to issue these hypercalls. + +See hypercalls.rst for more details. + +Isolation mechanisms +==================== + +pKVM relies on a number of mechanisms to isolate PVMs from the host: + +CPU memory isolation +-------------------- + +Status: Isolation of anonymous memory and metadata pages. + +Metadata pages (e.g. page-table pages and '``struct kvm_vcpu``' pages) +are donated from the host to the hypervisor during pVM creation and +are consequently unmapped from the stage-2 identity map until the pVM is +destroyed. + +Similarly to regular KVM, pages are lazily mapped into the guest in +response to stage-2 page faults handled by the host. However, when +running a pVM, these pages are first pinned and then unmapped from the +stage-2 identity map as part of the donation procedure. This gives rise +to some user-visible differences when compared to non-protected VMs, +largely due to the lack of MMU notifiers: + +* Memslots cannot be moved or deleted once the pVM has started running. +* Read-only memslots and dirty logging are not supported. +* With the exception of swap, file-backed pages cannot be mapped into a + pVM. +* Donated pages are accounted against ``RLIMIT_MLOCK`` and so the VMM + must have a sufficient resource limit or be granted ``CAP_IPC_LOCK``. + The lack of a runtime reclaim mechanism means that memory locked for + a pVM will remain locked until the pVM is destroyed. +* Changes to the VMM address space (e.g. a ``MAP_FIXED`` mmap() over a + mapping associated with a memslot) are not reflected in the guest and + may lead to loss of coherency. +* Accessing pVM memory that has not been shared back will result in the + delivery of a SIGSEGV. +* If a system call accesses pVM memory that has not been shared back + then it will either return ``-EFAULT`` or forcefully reclaim the + memory pages. Reclaimed memory is zeroed by the hypervisor and a + subsequent attempt to access it in the pVM will return ``-EFAULT`` + from the ``VCPU_RUN`` ioctl(). + +CPU state isolation +------------------- + +Status: **Unimplemented.** + +DMA isolation using an IOMMU +---------------------------- + +Status: **Unimplemented.** + +Proxying of Trustzone services +------------------------------ + +Status: FF-A and PSCI calls from the host are proxied by the pKVM +hypervisor. + +The FF-A proxy ensures that the host cannot share pVM or hypervisor +memory with Trustzone as part of a "confused deputy" attack. + +The PSCI proxy ensures that CPUs always have the stage-2 identity map +installed when they are executing in the host. + +Protected VM firmware (pvmfw) +----------------------------- + +Status: **Unimplemented.** + +Resources +========= + +Quentin Perret's KVM Forum 2022 talk entitled "Protected KVM on arm64: A +technical deep dive" remains a good resource for learning more about +pKVM, despite some of the details having changed in the meantime: + +https://www.youtube.com/watch?v=9npebeVFbFw -- cgit v1.2.3 From 4aebd7d5c72f805ef59985958ad76b8dbce60d8f Mon Sep 17 00:00:00 2001 From: Hendrik Brueckner Date: Wed, 1 Apr 2026 17:12:21 +0200 Subject: KVM: s390: Add KVM capability for ESA mode guests Now that all the bits are properly addressed, provide a mechanism for testing ESA mode guests in nested configurations. Signed-off-by: Hendrik Brueckner [farman@us.ibm.com: Updated commit message] Reviewed-by: Janosch Frank Signed-off-by: Eric Farman Signed-off-by: Janosch Frank --- Documentation/virt/kvm/api.rst | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'Documentation') diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 6f85e1b321dd..682ae9278943 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -9428,6 +9428,14 @@ KVM exits with the register state of either the L1 or L2 guest depending on which executed at the time of an exit. Userspace must take care to differentiate between these cases. +8.47 KVM_CAP_S390_VSIE_ESAMODE +------------------------------ + +:Architectures: s390 + +The presence of this capability indicates that the nested KVM guest can +start in ESA mode. + 9. Known KVM API problems ========================= -- cgit v1.2.3