perf test attr: Allow tests to require a core PMU

On machines without a usable core PMU, like an aarch64 box with no DT
node for the ARM PMU, the expectations that pin the legacy hardware
encoding can't be checked: without a core PMU at all 'cycles' doesn't
even resolve, and with one that can't be opened 'perf record' takes the
software fallback and records attributes they can never match:

  expected type=0, got 1
  expected config=0, got 1
  FAILED '.../attr/test-record-group-sampling1' - match failure

And since attr.py stops at the first failure, every expectation that
would come after that one ends up not being run at all.

Add a 'requires' limitation, modelled after the existing 'arch', 'auxv'
and 'kernel_since/until' ones, with 'core-pmu' as its first value, that
is checked by asking perf itself to open 'cycles' via its python
binding: a parse failure or an ENOENT means there is no PMU that can
count it, hence nothing to check and the test gets skipped, while any
other error means we can't tell, so the test keeps running as it does
today, where there is no binding to ask either.

When there is no core PMU, asking the binding fails and prints a parse
error, once for every test that asks, so the answer is kept: it is asked
once per run.  A value other than 'core-pmu' is warned about, not
silently ignored, so a typo can't turn the skip into a no-op.

An old distro python package may have a 'perf' module without
parse_events(), i.e. there is nothing to ask with, so that
AttributeError gets the same treatment as the ImportError: let the
expectations speak.

The expectations that have a strict 'type=0', plus the group tests,
where the fallback drops a member and leaves nothing to compare the
group leader attributes with, now state that they require a core PMU.
They are marked here, before the commit that has the group tests ask for
the legacy encodings, so the requisite comes with that dependency.

Reviewed-by: Ian Rogers <irogers@google.com>
Tested-by: Ian Rogers <irogers@google.com>
Assisted-by: LLM
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

15 files changed, 77 insertions, 0 deletions

diff --git a/tools/perf/tests/shell/attr/test-record-group-sampling b/tools/perf/tests/shell/attr/test-record-group-sampling
index 86a940d7895d..b43d9fe3af75 100644
--- a/tools/perf/tests/shell/attr/test-record-group-sampling
+++ b/tools/perf/tests/shell/attr/test-record-group-sampling
@@ -3,6 +3,7 @@ command = record
 args    = --no-bpf-event -e '{cycles,cache-misses}:S' kill >/dev/null 2>&1
 ret     = 1
 kernel_until = 6.12
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-record-group-sampling1 b/tools/perf/tests/shell/attr/test-record-group-sampling1
index 4748ab7bf684..3def72161820 100644
--- a/tools/perf/tests/shell/attr/test-record-group-sampling1
+++ b/tools/perf/tests/shell/attr/test-record-group-sampling1
@@ -3,6 +3,7 @@ command = record
 args    = --no-bpf-event -e '{cycles,cache-misses}:S' kill >/dev/null 2>&1
 ret     = 1
 kernel_since = 6.12
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-record-group-sampling2 b/tools/perf/tests/shell/attr/test-record-group-sampling2
index e0432244a0eb..227c18bb221c 100644
--- a/tools/perf/tests/shell/attr/test-record-group-sampling2
+++ b/tools/perf/tests/shell/attr/test-record-group-sampling2
@@ -3,6 +3,7 @@ command = record
 args    = --no-bpf-event -c 10000 -e '{cycles,cache-misses}:S' kill >/dev/null 2>&1
 ret     = 1
 kernel_since = 6.12
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-record-group1 b/tools/perf/tests/shell/attr/test-record-group1
index eeb1db392bc9..a57c7ee8aaa4 100644
--- a/tools/perf/tests/shell/attr/test-record-group1
+++ b/tools/perf/tests/shell/attr/test-record-group1
@@ -2,6 +2,7 @@
 command = record
 args    = --no-bpf-event -e '{cycles,instructions}' kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-record-group2 b/tools/perf/tests/shell/attr/test-record-group2
index 891d41a7bddf..fb6e664eeacc 100644
--- a/tools/perf/tests/shell/attr/test-record-group2
+++ b/tools/perf/tests/shell/attr/test-record-group2
@@ -3,6 +3,7 @@ command = record
 args    = --no-bpf-event -e '{cycles/period=1234000/,instructions/period=6789000/}:S' kill >/dev/null 2>&1
 ret     = 1
 kernel_until = 6.12
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-record-group3 b/tools/perf/tests/shell/attr/test-record-group3
index 249be884959e..7b90b4411d8f 100644
--- a/tools/perf/tests/shell/attr/test-record-group3
+++ b/tools/perf/tests/shell/attr/test-record-group3
@@ -3,6 +3,7 @@ command = record
 args    = --no-bpf-event -e '{cycles/period=1234000/,instructions/period=6789000/}:S' kill >/dev/null 2>&1
 ret     = 1
 kernel_since = 6.12
+requires = core-pmu
 
 [event-1:base-record]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-stat-C0 b/tools/perf/tests/shell/attr/test-stat-C0
index a2c76d10b2bb..9d242e93e3a9 100644
--- a/tools/perf/tests/shell/attr/test-stat-C0
+++ b/tools/perf/tests/shell/attr/test-stat-C0
@@ -2,6 +2,7 @@
 command = stat
 args    = -e cycles -C 0 kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 [event:base-stat]
 # events are disabled by default when attached to cpu
diff --git a/tools/perf/tests/shell/attr/test-stat-basic b/tools/perf/tests/shell/attr/test-stat-basic
index 69867d049fda..277897bd2626 100644
--- a/tools/perf/tests/shell/attr/test-stat-basic
+++ b/tools/perf/tests/shell/attr/test-stat-basic
@@ -2,6 +2,7 @@
 command = stat
 args    = -e cycles kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 [event:base-stat]
 optional=1
diff --git a/tools/perf/tests/shell/attr/test-stat-default b/tools/perf/tests/shell/attr/test-stat-default
index 8dd27c1fb661..74691188e859 100644
--- a/tools/perf/tests/shell/attr/test-stat-default
+++ b/tools/perf/tests/shell/attr/test-stat-default
@@ -2,6 +2,7 @@
 command = stat
 args    = kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 # PERF_TYPE_SOFTWARE / PERF_COUNT_SW_TASK_CLOCK
 [event1:base-stat]
diff --git a/tools/perf/tests/shell/attr/test-stat-detailed-1 b/tools/perf/tests/shell/attr/test-stat-detailed-1
index 12a2ebf4e64a..d9a6236054f2 100644
--- a/tools/perf/tests/shell/attr/test-stat-detailed-1
+++ b/tools/perf/tests/shell/attr/test-stat-detailed-1
@@ -2,6 +2,7 @@
 command = stat
 args    = -d kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 
 # PERF_TYPE_SOFTWARE / PERF_COUNT_SW_TASK_CLOCK
diff --git a/tools/perf/tests/shell/attr/test-stat-detailed-2 b/tools/perf/tests/shell/attr/test-stat-detailed-2
index 66ea25b7d38f..582a30289ccf 100644
--- a/tools/perf/tests/shell/attr/test-stat-detailed-2
+++ b/tools/perf/tests/shell/attr/test-stat-detailed-2
@@ -2,6 +2,7 @@
 command = stat
 args    = -dd kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 
 # PERF_TYPE_SOFTWARE / PERF_COUNT_SW_TASK_CLOCK
diff --git a/tools/perf/tests/shell/attr/test-stat-detailed-3 b/tools/perf/tests/shell/attr/test-stat-detailed-3
index 4a27bbfb9f87..2d0fca8b5020 100644
--- a/tools/perf/tests/shell/attr/test-stat-detailed-3
+++ b/tools/perf/tests/shell/attr/test-stat-detailed-3
@@ -2,6 +2,7 @@
 command = stat
 args    = -ddd kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 
 # PERF_TYPE_SOFTWARE / PERF_COUNT_SW_TASK_CLOCK
diff --git a/tools/perf/tests/shell/attr/test-stat-group1 b/tools/perf/tests/shell/attr/test-stat-group1
index 1746751123dc..3d8b44ba42d6 100644
--- a/tools/perf/tests/shell/attr/test-stat-group1
+++ b/tools/perf/tests/shell/attr/test-stat-group1
@@ -2,6 +2,7 @@
 command = stat
 args    = -e '{cycles,instructions}' kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 [event-1:base-stat]
 fd=1
diff --git a/tools/perf/tests/shell/attr/test-stat-no-inherit b/tools/perf/tests/shell/attr/test-stat-no-inherit
index 924fbb9300d1..38f746855b1b 100644
--- a/tools/perf/tests/shell/attr/test-stat-no-inherit
+++ b/tools/perf/tests/shell/attr/test-stat-no-inherit
@@ -2,6 +2,7 @@
 command = stat
 args    = -i -e cycles kill >/dev/null 2>&1
 ret     = 1
+requires = core-pmu
 
 [event:base-stat]
 inherit=0
diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
index 68f06df9565a..2d7dd26a5737 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 
 import configparser
+import errno
 import os
 import sys
 import glob
@@ -28,6 +29,50 @@ def data_equal(a, b):
 
     return False
 
+CORE_PMU_AVAILABLE = None
+
+def core_pmu_available():
+    """
+    Is there a core PMU on this machine? Without one hardware events aren't
+    supported and get replaced by the software fallback, so there's no way to
+    check the expectations that pin their legacy encodings.  The answer can't
+    change while the tests run, so probe the binding only once.
+    """
+    global CORE_PMU_AVAILABLE
+
+    if CORE_PMU_AVAILABLE is None:
+        CORE_PMU_AVAILABLE = core_pmu_probe()
+
+    return CORE_PMU_AVAILABLE
+
+def core_pmu_probe():
+    """
+    Ask perf itself, through its python binding, to parse and open 'cycles':
+    a parse failure or an ENOENT means no PMU can count it, any other error
+    means we can't tell whether there is one.
+    """
+    try:
+        import perf
+        parse_events = perf.parse_events
+    except (ImportError, AttributeError):
+        # Nothing to ask about it, let the expectations speak.
+        return True
+
+    try:
+        evlist = parse_events('cycles')
+    except OSError:
+        # No PMU that can count it, the same as the ENOENT below.
+        return False
+
+    try:
+        evlist.open()
+    except OSError as err:
+        # Anything else is a permission problem or alike, not our business here.
+        return err.errno != errno.ENOENT
+    evlist.close()
+
+    return True
+
 class Fail(Exception):
     def __init__(self, test, msg):
         self.msg = msg
@@ -138,6 +183,9 @@ def parse_version(version):
 #     'arch'    - architecture specific test (optional)
 #                 comma separated list, ! at the beginning
 #                 negates it.
+#     'requires' - Comma separated list of things the test needs, it is
+#                 skipped when any of them isn't there, currently:
+#                   core-pmu - the machine can count hardware events.
 #     'auxv'    - Truthy statement that is evaled in the scope of the auxv map. When false,
 #                 the test is skipped. For example 'auxv["AT_HWCAP"] == 10'. (optional)
 #     'kernel_since' - Inclusive kernel version from which the test will start running. Only the
@@ -173,6 +221,7 @@ class Test(object):
             self.arch  = ''
 
         self.auxv = parser.get('config', 'auxv', fallback=None)
+        self.requires = parser.get('config', 'requires', fallback='')
         self.kernel_since = parse_version(parser.get('config', 'kernel_since', fallback=None))
         self.kernel_until = parse_version(parser.get('config', 'kernel_until', fallback=None))
         self.expect   = {}
@@ -242,6 +291,17 @@ class Test(object):
                 return False
         return True
 
+    def skip_test_requires(self):
+        for requirement in filter(None, map(str.strip, self.requires.split(','))):
+            if requirement == 'core-pmu':
+                if not core_pmu_available():
+                    log.warning("test limitation 'requires %s'", requirement)
+                    return True
+            else:
+                log.warning("unknown requirement '%s', ignored", requirement)
+
+        return False
+
     def restore_sample_rate(self, value=10000):
         try:
             # Check value of sample_rate
@@ -296,6 +356,9 @@ class Test(object):
         if self.skip_test_kernel_until():
             raise Notest(self, "new kernel skip")
 
+        if self.skip_test_requires():
+            raise Notest(self, "requirements skip")
+
         self.restore_sample_rate()
         cmd = "PERF_TEST_ATTR=%s %s %s -o %s/perf.data %s" % (tempdir,
               self.perf, self.command, tempdir, self.args)