From e0ea34158ee8c4f7536cd781010339ff28c0d24c Mon Sep 17 00:00:00 2001 From: David Arinzon Date: Tue, 17 Jun 2025 14:05:37 +0300 Subject: net: ena: Add PHC support in the ENA driver The ENA driver will be extended to support the new PHC feature using ptp_clock interface [1]. this will provide timestamp reference for user space to allow measuring time offset between the PHC and the system clock in order to achieve nanosecond accuracy. [1] - https://www.kernel.org/doc/html/latest/driver-api/ptp.html Signed-off-by: Amit Bernstein Signed-off-by: David Arinzon Link: https://patch.msgid.link/20250617110545.5659-2-darinzon@amazon.com Signed-off-by: Jakub Kicinski --- Documentation/networking/device_drivers/ethernet/amazon/ena.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'Documentation') diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst index 4561e8ab9e08..98dc62174a40 100644 --- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst +++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst @@ -56,6 +56,7 @@ ena_netdev.[ch] Main Linux kernel driver. ena_ethtool.c ethtool callbacks. ena_xdp.[ch] XDP files ena_pci_id_tbl.h Supported device IDs. +ena_phc.[ch] PTP hardware clock infrastructure (see `PHC`_ for more info) ================= ====================================================== Management Interface: -- cgit v1.2.3 From 15115b1a255404795158fa92a1cba87a9acff15d Mon Sep 17 00:00:00 2001 From: David Arinzon Date: Tue, 17 Jun 2025 14:05:39 +0300 Subject: net: ena: Add device reload capability through devlink Adding basic devlink capability support of reloading the driver. This capability is required to support driver init type devlink params (DEVLINK_PARAM_CMODE_DRIVERINIT). Such params require reloading of the driver (destroy/restore sequence). The reloading is done by the devlink framework using the hooks provided by the driver. Signed-off-by: David Arinzon Link: https://patch.msgid.link/20250617110545.5659-4-darinzon@amazon.com Signed-off-by: Jakub Kicinski --- .../networking/device_drivers/ethernet/amazon/ena.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'Documentation') diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst index 98dc62174a40..112a3994bc85 100644 --- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst +++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst @@ -57,6 +57,7 @@ ena_ethtool.c ethtool callbacks. ena_xdp.[ch] XDP files ena_pci_id_tbl.h Supported device IDs. ena_phc.[ch] PTP hardware clock infrastructure (see `PHC`_ for more info) +ena_devlink.[ch] devlink files. ================= ====================================================== Management Interface: @@ -269,6 +270,18 @@ RSS - The user can provide a hash key, hash function, and configure the indirection table through `ethtool(8)`. +DEVLINK SUPPORT +=============== +.. _`devlink`: https://www.kernel.org/doc/html/latest/networking/devlink/index.html + +`devlink`_ supports reloading the driver and initiating re-negotiation with the ENA device + +.. code-block:: shell + + sudo devlink dev reload pci/ + # for example: + sudo devlink dev reload pci/0000:00:06.0 + DATA PATH ========= -- cgit v1.2.3 From cea465a96a294e7bc2537f27a737cfa7c6234b3d Mon Sep 17 00:00:00 2001 From: David Arinzon Date: Tue, 17 Jun 2025 14:05:41 +0300 Subject: devlink: Add new "enable_phc" generic device param Add a new device generic parameter to enable/disable the PHC (PTP Hardware Clock) functionality in the device associated with the devlink instance. Signed-off-by: David Arinzon Reviewed-by: Jiri Pirko Link: https://patch.msgid.link/20250617110545.5659-6-darinzon@amazon.com Signed-off-by: Jakub Kicinski --- Documentation/networking/devlink/devlink-params.rst | 3 +++ 1 file changed, 3 insertions(+) (limited to 'Documentation') diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index 4e01dc32bc08..3da8f4ef2417 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -137,3 +137,6 @@ own name. * - ``event_eq_size`` - u32 - Control the size of asynchronous control events EQ. + * - ``enable_phc`` + - Boolean + - Enable PHC (PTP Hardware Clock) functionality in the device. -- cgit v1.2.3 From 60e28350b1ca127fe22dd99d5ff2a1922450e912 Mon Sep 17 00:00:00 2001 From: David Arinzon Date: Tue, 17 Jun 2025 14:05:43 +0300 Subject: net: ena: Add debugfs support to the ENA driver Adding the base directory of debugfs to the driver. In order for the folder to be unique per driver instantiation, the chosen name is the device name. This commit contains the initialization and the base folder. The creation of the base folder may fail, but is considered non-fatal. Signed-off-by: David Arinzon Link: https://patch.msgid.link/20250617110545.5659-8-darinzon@amazon.com Signed-off-by: Jakub Kicinski --- Documentation/networking/device_drivers/ethernet/amazon/ena.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'Documentation') diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst index 112a3994bc85..7b7b8891cec9 100644 --- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst +++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst @@ -58,6 +58,7 @@ ena_xdp.[ch] XDP files ena_pci_id_tbl.h Supported device IDs. ena_phc.[ch] PTP hardware clock infrastructure (see `PHC`_ for more info) ena_devlink.[ch] devlink files. +ena_debugfs.[ch] debugfs files. ================= ====================================================== Management Interface: -- cgit v1.2.3 From c9223021433d9b2d4ca32cf9e582e6908f08c3cb Mon Sep 17 00:00:00 2001 From: David Arinzon Date: Tue, 17 Jun 2025 14:05:45 +0300 Subject: net: ena: Add PHC documentation Provide the relevant information and guidelines about the feature support in the ENA driver. Signed-off-by: Amit Bernstein Signed-off-by: David Arinzon Link: https://patch.msgid.link/20250617110545.5659-10-darinzon@amazon.com Signed-off-by: Jakub Kicinski --- .../device_drivers/ethernet/amazon/ena.rst | 93 ++++++++++++++++++++++ 1 file changed, 93 insertions(+) (limited to 'Documentation') diff --git a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst index 7b7b8891cec9..14784a0a6a8a 100644 --- a/Documentation/networking/device_drivers/ethernet/amazon/ena.rst +++ b/Documentation/networking/device_drivers/ethernet/amazon/ena.rst @@ -224,6 +224,99 @@ descriptor it was received on would be recycled. When a packet smaller than RX copybreak bytes is received, it is copied into a new memory buffer and the RX descriptor is returned to HW. +.. _`PHC`: + +PTP Hardware Clock (PHC) +======================== +.. _`ptp-userspace-api`: https://docs.kernel.org/driver-api/ptp.html#ptp-hardware-clock-user-space-api +.. _`testptp`: https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests/ptp/testptp.c + +ENA Linux driver supports PTP hardware clock providing timestamp reference to achieve nanosecond resolution. + +**PHC support** + +PHC depends on the PTP module, which needs to be either loaded as a module or compiled into the kernel. + +Verify if the PTP module is present: + +.. code-block:: shell + + grep -w '^CONFIG_PTP_1588_CLOCK=[ym]' /boot/config-`uname -r` + +- If no output is provided, the ENA driver cannot be loaded with PHC support. + +**PHC activation** + +The feature is turned off by default, in order to turn the feature on, the ENA driver +can be loaded in the following way: + +- devlink: + +.. code-block:: shell + + sudo devlink dev param set pci/ name enable_phc value true cmode driverinit + sudo devlink dev reload pci/ + # for example: + sudo devlink dev param set pci/0000:00:06.0 name enable_phc value true cmode driverinit + sudo devlink dev reload pci/0000:00:06.0 + +All available PTP clock sources can be tracked here: + +.. code-block:: shell + + ls /sys/class/ptp + +PHC support and capabilities can be verified using ethtool: + +.. code-block:: shell + + ethtool -T + +**PHC timestamp** + +To retrieve PHC timestamp, use `ptp-userspace-api`_, usage example using `testptp`_: + +.. code-block:: shell + + testptp -d /dev/ptp$(ethtool -T | awk '/PTP Hardware Clock:/ {print $NF}') -k 1 + +PHC get time requests should be within reasonable bounds, +avoid excessive utilization to ensure optimal performance and efficiency. +The ENA device restricts the frequency of PHC get time requests to a maximum +of 125 requests per second. If this limit is surpassed, the get time request +will fail, leading to an increment in the phc_err_ts statistic. + +**PHC statistics** + +PHC can be monitored using debugfs (if mounted): + +.. code-block:: shell + + sudo cat /sys/kernel/debug//phc_stats + + # for example: + sudo cat /sys/kernel/debug/0000:00:06.0/phc_stats + +PHC errors must remain below 1% of all PHC requests to maintain the desired level of accuracy and reliability + +================= ====================================================== +**phc_cnt** | Number of successful retrieved timestamps (below expire timeout). +**phc_exp** | Number of expired retrieved timestamps (above expire timeout). +**phc_skp** | Number of skipped get time attempts (during block period). +**phc_err_dv** | Number of failed get time attempts due to device errors (entering into block state). +**phc_err_ts** | Number of failed get time attempts due to timestamp errors (entering into block state), + | This occurs if driver exceeded the request limit or device received an invalid timestamp. +================= ====================================================== + +PHC timeouts: + +================= ====================================================== +**expire** | Max time for a valid timestamp retrieval, passing this threshold will fail + | the get time request and block new requests until block timeout. +**block** | Blocking period starts once get time request expires or fails, + | all get time requests during block period will be skipped. +================= ====================================================== + Statistics ========== -- cgit v1.2.3