linux-toradex.git/tools/perf/util/parse-events.l, branch v6.3

perf stat: Add user_time and system_time events

2022-04-20T16:44:56+00:00

It bothered me that during benchmarking using 'perf stat' (to collect
for example CPU cache events) I could not simultaneously retrieve the
times spend in user or kernel mode in a machine readable format.

When running 'perf stat' the output for humans contains the times
reported by rusage and wait4.

  $ perf stat -e cache-misses:u -- true

   Performance counter stats for 'true':

             4,206      cache-misses:u

       0.001113619 seconds time elapsed

       0.001175000 seconds user
       0.000000000 seconds sys

But 'perf stat's machine-readable format does not provide this information.

  $ perf stat -x, -e cache-misses:u -- true
  4282,,cache-misses:u,492859,100.00,,

I found no way to retrieve this information using the available events
while using machine-readable output.

This patch adds two new tool internal events 'user_time' and
'system_time', similarly to the already present 'duration_time' event.

Both events use the already collected rusage information obtained by
wait4 and tracked in the global ru_stats.

Examples presenting cache-misses and rusage information in both human
and machine-readable form:

  $ perf stat -e duration_time,user_time,system_time,cache-misses -- grep -q -r duration_time .

   Performance counter stats for 'grep -q -r duration_time .':

        67,422,542 ns   duration_time:u
        50,517,000 ns   user_time:u
        16,839,000 ns   system_time:u
            30,937      cache-misses:u

       0.067422542 seconds time elapsed

       0.050517000 seconds user
       0.016839000 seconds sys

  $ perf stat -x, -e duration_time,user_time,system_time,cache-misses -- grep -q -r duration_time .
  72134524,ns,duration_time:u,72134524,100.00,,
  65225000,ns,user_time:u,65225000,100.00,,
  6865000,ns,system_time:u,6865000,100.00,,
  38705,,cache-misses:u,71189328,100.00,,

Signed-off-by: Florian Fischer 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Ian Rogers 
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Xing Zhengjun 
Link: https://lore.kernel.org/r/20220420102354.468173-3-florian.fischer@muhq.space
Signed-off-by: Arnaldo Carvalho de Melo

perf parse-events: Support event alias in form foo-bar-baz

2022-01-22T20:20:12+00:00

Event aliasing for events whose name in the form foo-bar-baz is not
supported, while foo-bar, foo_bar_baz, and other combinations are, i.e.
two hyphens are not supported.

The HiSilicon D06 platform has events in such form:

  $ ./perf list sdir-home-migrate

  List of pre-defined events (to be used in -e):

  uncore hha:
    sdir-home-migrate
   [Unit: hisi_sccl,hha]

  $ sudo ./perf stat -e sdir-home-migrate
  event syntax error: 'sdir-home-migrate'
                          \___ parser error
  Run 'perf list' for a list of valid events

   Usage: perf stat [] []

   -e, --event event selector. use 'perf list' to list available events

To support, add an extra PMU event symbol type for "baz", and add a new
rule in the bison file.

Signed-off-by: John Garry 
Acked-by: Ian Rogers 
Cc: Alexander Shishkin 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Kajol Jain 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Qi Liu 
Cc: Shaokun Zhang 
Cc: linuxarm@huawei.com
Link: https://lore.kernel.org/r/1642432215-234089-2-git-send-email-john.garry@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo

perf metric: Encode and use metric-id as qualifier

2021-10-20T13:57:05+00:00

For a metric like IPC a group of events like {instructions,cycles}:W
would be formed.

If the events names were changed in parsing then the metric expression
parser would fail to find them.

This change makes the event encoding be something like:

  {instructions/metric-id=instructions/, cycles/metric-id=cycles/}

and then uses the evsel's stable metric-id value to locate the events.

This fixes the case that an event is restricted to user because of the
paranoia setting:

  $ echo 2 > /proc/sys/kernel/perf_event_paranoid
  $ perf stat -M IPC /bin/true
   Performance counter stats for '/bin/true':

             150,298      inst_retired.any:u        #      0.77 IPC
             187,095      cpu_clk_unhalted.thread:u

         0.002042731 seconds time elapsed

         0.000000000 seconds user
         0.002377000 seconds sys

Adding the metric-id as a qualifier has a complication in that
qualifiers will become embedded in qualifiers.

For example, msr/tsc/ could become msr/tsc,metric-id=msr/tsc// which
will fail parse-events.

To solve this problem the metric is encoded and decoded for the
metric-id with ! standing in for an encoded value.

Previously ! wasn't parsed.

With this msr/tsc/ becomes msr/tsc,metric-id=msr!3tsc!3/

The metric expression parser is changed so that @ isn't changed to /,
instead this is done when the ID is encoded for parse events.

metricgroup__add_metric_non_group() and metricgroup__add_metric_weak_group()
need to inject the metric-id qualifier, so to avoid repetition they are
merged into a single metricgroup__build_event_string with error codes
more rigorously checked.

stat-shadow's prepare_metric() uses the metric-id to match the metricgroup
code.

As "metric-id=..." is added to all events, it is adding during testing
with the fake PMU.

This complicates pmu_str_check code as PE_PMU_EVENT_FAKE won't match as
part of a configuration.

The testing fake PMU case is fixed so that if a known qualifier with an
! is parsed then it isn't reported as a fake PMU.

This is sufficient to pass all testing but it and the original mechanism
are somewhat brittle.

Signed-off-by: Ian Rogers 
Acked-by: Andi Kleen 
Cc: Adrian Hunter 
Cc: Alexander Antonov 
Cc: Alexander Shishkin 
Cc: Andrew Kilroy 
Cc: Andrew Morton 
Cc: Changbin Du 
Cc: Denys Zagorui 
Cc: Fabian Hemmer 
Cc: Felix Fietkau 
Cc: Heiko Carstens 
Cc: Ingo Molnar 
Cc: Jacob Keller 
Cc: Jiapeng Chong 
Cc: Jin Yao 
Cc: Jiri Olsa 
Cc: Joakim Zhang 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kees Kook 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Nicholas Fraser 
Cc: Nick Desaulniers 
Cc: Paul Clarke 
Cc: Peter Zijlstra 
Cc: Riccardo Mancini 
Cc: Sami Tolvanen 
Cc: ShihCheng Tu 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: Sumanth Korikkar 
Cc: Thomas Richter 
Cc: Wan Jiabing 
Cc: Zhen Lei 
Link: https://lore.kernel.org/r/20211015172132.1162559-17-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf parse-events: Add new "metric-id" term

2021-10-20T13:54:44+00:00

Add a new "metric-id" term to events so that metric parsing can set an
ID that can be reliably looked up.

Metric parsing currently will turn a metric like "instructions/cycles"
into a parse events string of "{instructions,cycles}:W".

However, parse-events may change "instructions" into "instructions:u" if
perf_event_paranoid=2.

When this happens expr__resolve_id currently fails as stat-shadow adds
the ID "instructions:u" to match with the counter value and the metric
tries to look up the ID just "instructions".

A later patch will use the new term.

An example of the current problem:

  $ echo -1 > /proc/sys/kernel/perf_event_paranoid
  $ perf stat -M IPC /bin/true
   Performance counter stats for '/bin/true':

           1,217,161      inst_retired.any          #     0.97 IPC
           1,250,389      cpu_clk_unhalted.thread

         0.002064773 seconds time elapsed

         0.002378000 seconds user
         0.000000000 seconds sys

  $ echo 2 > /proc/sys/kernel/perf_event_paranoid
  $ perf stat -M IPC /bin/true
   Performance counter stats for '/bin/true':

             150,298      inst_retired.any:u        #      nan IPC
             187,095      cpu_clk_unhalted.thread:u

         0.002042731 seconds time elapsed

         0.000000000 seconds user
         0.002377000 seconds sys

Note: nan IPC is printed as an effect of "perf metric: Use NAN for
missing event IDs." but earlier versions of perf just fail with a parse
error and display no value.

Signed-off-by: Ian Rogers 
Acked-by: Andi Kleen 
Cc: Adrian Hunter 
Cc: Alexander Antonov 
Cc: Alexander Shishkin 
Cc: Andrew Kilroy 
Cc: Andrew Morton 
Cc: Changbin Du 
Cc: Denys Zagorui 
Cc: Fabian Hemmer 
Cc: Felix Fietkau 
Cc: Heiko Carstens 
Cc: Ingo Molnar 
Cc: Jacob Keller 
Cc: Jiapeng Chong 
Cc: Jin Yao 
Cc: Jiri Olsa 
Cc: Joakim Zhang 
Cc: John Garry 
Cc: Kajol Jain 
Cc: Kan Liang 
Cc: Kees Kook 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Nicholas Fraser 
Cc: Nick Desaulniers 
Cc: Paul Clarke 
Cc: Peter Zijlstra 
Cc: Riccardo Mancini 
Cc: Sami Tolvanen 
Cc: ShihCheng Tu 
Cc: Song Liu 
Cc: Stephane Eranian 
Cc: Sumanth Korikkar 
Cc: Thomas Richter 
Cc: Wan Jiabing 
Cc: Zhen Lei 
Link: https://lore.kernel.org/r/20211015172132.1162559-15-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf parse-events: Remove unnecessary #includes

2021-09-21T23:55:48+00:00

Minor cleanup motivated by trying to separately fuzz test parse-events.

Signed-off-by: Ian Rogers 
Acked-by: Namhyung Kim 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Jin Yao 
Cc: Jiri Olsa 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lore.kernel.org/lkml/20210127184629.516169-1-irogers@google.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add 'cgroup-switches' software event

2021-05-19T17:23:23+00:00

It counts how often cgroups are changed actually during the context
switches.

  # perf stat -a -e context-switches,cgroup-switches -a sleep 1

   Performance counter stats for 'system wide':

              11,267      context-switches
              10,950      cgroup-switches

         1.015634369 seconds time elapsed

Committer notes:

The kernel patches landed in v5.13, but this entry wasn't filled in
perf's parse-events tables, which was leading to a segfault when running
'perf list' on a kernel with that feature, as reported by Thomas
Richter.

Also removed the part touching tools/include/uapi/linux/perf_event.h as
it was updated in the usual sync with the kernel UAPI headers, in a
previous, already upstream, patch.

Signed-off-by: Namhyung Kim 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Heiko Carstens 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: Thomas Richter 
Link: http://lore.kernel.org/lkml/20210210083327.22726-3-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Introduce ':b' modifier

2021-04-29T13:30:58+00:00

Introduce 'b' modifier to event parser, which means use BPF program to
manage this event. This is the same as --bpf-counters option, but only
applies to this event. For example,

  perf stat -e cycles:b,cs               # use bpf for cycles, but not cs
  perf stat -e cycles,cs --bpf-counters  # use bpf for both cycles and cs

Suggested-by: Jiri Olsa 
Signed-off-by: Song Liu 
Cc: Namhyung Kim 
Cc: Song Liu 
Link: https://lore.kernel.org/r/20210425214333.1090950-5-song@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Support the auxiliary event

2021-02-08T19:25:00+00:00

On the Intel Sapphire Rapids server, an auxiliary event has to be
enabled simultaneously with the load latency event to retrieve complete
Memory Info.

Add X86 specific perf_mem_events__name() to handle the auxiliary event.

- Users are only interested in the samples of the mem-loads event.
  Sample read the auxiliary event.

- The auxiliary event must be in front of the load latency event in a
  group. Assume the second event to sample if the auxiliary event is the
  leader.

- Add a weak is_mem_loads_aux_event() to check the auxiliary event for
  X86. For other ARCHs, it always return false.

Parse the unique event name, mem-loads-aux, for the auxiliary event.

Committer notes:

According to 61b985e3e775a3a7 ("perf/x86/intel: Add perf core PMU
support for Sapphire Rapids"), ENODATA is only returned by
sys_perf_event_open() when used with these auxiliary events, with this
in evsel__open_strerror():

       case ENODATA:
               return scnprintf(msg, size, "Cannot collect data source with the load latency event alone. "
                                "Please add an auxiliary event in front of the load latency event.");

This is Ok at this point in time, but fragile long term, I pointed this
out in the e-mail thread, requesting a follow up patch to check if
ENODATA is really for this specific case.

Fixed up sizeof(MEM_LOADS_AUX_NAME) bug pointed out by Namhyung.

Signed-off-by: Kan Liang 
Cc: Andi Kleen 
Cc: Jin Yao 
Cc: Jiri Olsa 
Cc: Madhavan Srinivasan 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lore.kernel.org/lkml/20210205152648.GC920417@kernel.org
Link: http://lore.kernel.org/lkml/1612296553-21962-3-git-send-email-kan.liang@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add support for exclusive groups/events

2020-10-14T15:24:28+00:00

Peter suggested that using the exclusive mode in perf could avoid some
problems with bad scheduling of groups. Exclusive is implemented in the
kernel, but wasn't exposed by the perf tool, so hard to use without
custom low level API users.

Add support for marking groups or events with :e for exclusive in the
perf tool.  The implementation is basically the same as the existing
pinned attribute.

Committer testing:

  # perf test "parse event"
   6: Parse event definition strings                                  : Ok
  # perf test -v "parse event" |& grep :u*e
  running test 56 'instructions:uep'
  running test 57 '{cycles,cache-misses,branch-misses}:e'
  #
  #
  # grep "model name" -m1 /proc/cpuinfo
  model name	: AMD Ryzen 9 3900X 12-Core Processor
  #
  # perf stat -a -e '{cycles,cache-misses,branch-misses}:e' sleep 1

   Performance counter stats for 'system wide':

             cycles                                                        (0.00%)
             cache-misses                                                  (0.00%)
             branch-misses                                                 (0.00%)

         1.001269893 seconds time elapsed

  Some events weren't counted. Try disabling the NMI watchdog:
  	echo 0 > /proc/sys/kernel/nmi_watchdog
  	perf stat ...
  	echo 1 > /proc/sys/kernel/nmi_watchdog
  # echo 0 > /proc/sys/kernel/nmi_watchdog
  # perf stat -a -e '{cycles,cache-misses,branch-misses}:e' sleep 1

   Performance counter stats for 'system wide':

       1,298,663,141      cycles
          30,962,215      cache-misses
           5,325,150      branch-misses

         1.001474934 seconds time elapsed

  #
  # The output for asking for precise events on AMD needs to improve, it
  # supposedly works only for system wide or per CPU
  #
  # perf stat -a -e '{cycles,cache-misses,branch-misses}:uep' sleep 1
  Error:
  The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (cycles).
  /bin/dmesg | grep -i perf may provide additional information.

  # perf stat -a -e '{cycles,cache-misses,branch-misses}:ue' sleep 1

   Performance counter stats for 'system wide':

         746,363,126      cycles
          16,881,611      cache-misses
           2,871,259      branch-misses

         1.001636066 seconds time elapsed

  #

Signed-off-by: Andi Kleen 
Acked-by: Jiri Olsa 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Andi Kleen 
Cc: Peter Zijlstra 
Link: http://lore.kernel.org/lkml/20201014144255.22699-1-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Fix term parsing for raw syntax

2020-07-30T10:01:48+00:00

Jin Yao reported issue with possible conflict between raw events and
term values in pmu event syntax.

Currently following syntax is resolved as raw event with 0xead value:

  uncore_imc_free_running/read/

instead of using 'read' term from uncore_imc_free_running pmu, because
'read' is correct raw event syntax with 0xead value.

To solve this issue we do following:

  - check existing terms during rXXXX syntax processing
    and make them priority in case of conflict

  - allow pmu/r0x1234/ syntax to be able to specify conflicting
    raw event (implemented in previous patch)

Also add automated tests for this and perf_pmu__parse_cleanup call to
parse_events_terms, so the test gets properly cleaned up.

Fixes: 3a6c51e4d66c ("perf parser: Add support to specify rXXX event with pmu")
Reported-by: Jin Yao 
Signed-off-by: Jiri Olsa 
Tested-by: Jin Yao 
Acked-by: Ian Rogers 
Cc: Alexander Shishkin 
Cc: Andi Kleen 
Cc: Michael Petlan 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Cc: Thomas Richter 
Link: http://lore.kernel.org/lkml/20200726075244.1191481-2-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo