linux-toradex.git/tools/perf/Documentation, branch master

perf c2c: document function view in perf-c2c man page

2026-08-17T23:04:06+00:00

Describe the function view hierarchy (read-side function -> contending
writer function -> shared cachelines), the per-level indentation, and the
keys, with a worked example.

Document that reliable function attribution requires `iaddr` in
`--coalesce`, that the reader and writer may be the same function, and why
the coalesced function view cannot distinguish same-thread from
different-thread accesses in that case. Also document that verbose mode
includes code addresses in function rows.

Signed-off-by: Jiebin Sun 
Reviewed-by: Tianyou Li 
Reviewed-by: Wangyang Guo 
Reviewed-by: Ian Rogers 
Cc: Dapeng Mi 
Cc: James Clark 
Cc: Thomas Falcon 
Signed-off-by: Namhyung Kim

perf sched latency: Add histogram and time interval options

2026-08-07T17:56:23+00:00

While 'perf sched latency' reports task runtime and delay statistics
(average and maximum delay), it does not provide a visual representation
of how task wait times are distributed across latency ranges between
snapshots (start and finish of the analysis window).

The --histogram option collects CPU wait latencies (time between when
a task becomes runnable and when it gets scheduled onto a CPU) into 22
latency buckets, displaying an ASCII bar chart distribution.

The --hist-mode option configures the bucketing scheme:
  - log (default). Logarithmic latency buckets ranging from
    sub-microsecond (< 1 us) up to >= 1.05 seconds

  - linear. Equal-width linear latency buckets
    (i.e., 100 us steps up to >= 2.1 ms)

The --time option allows filtering trace event processing to a
specific time interval [start,stop].

Example histogram output excerpt:

    ❯ sudo perf sched latency --histogram --CPU 0

     CPU Wait Latency Distribution Histogram (between snapshots) (total samples: 36114)
     -------------------------------------------------------------------
      Latency Range    |      Count |    Pct | Histogram Graph
     -------------------------------------------------------------------
      < 1 us           |         17 |   0.0% | #
      2 - 4 us         |        673 |   1.9% | #
      4 - 8 us         |       6237 |  17.3% | ######
      8 - 16 us        |       3224 |   8.9% | ###
      16 - 32 us       |       1388 |   3.8% | #
      32 - 64 us       |        709 |   2.0% | #
      64 - 128 us      |        690 |   1.9% | #
      128 - 256 us     |        789 |   2.2% | #
      256 - 512 us     |        541 |   1.5% | #
      512 - 1024 us    |       2256 |   6.2% | ##
      1 - 2 ms         |       3577 |   9.9% | ###
      2 - 4 ms         |      13259 |  36.7% | ##############
      4 - 8 ms         |       2523 |   7.0% | ##
      8 - 16 ms        |        222 |   0.6% | #
      16 - 32 ms       |         10 |   0.0% | #
      >= 1.05 s        |          3 |   0.0% | #
     -------------------------------------------------------------------

Reviewed-by: Ian Rogers 
Signed-off-by: Aaron Tomlin 
Signed-off-by: Namhyung Kim

perf trace: Add --bitmask-list command-line option

2026-07-24T05:59:42+00:00

Introduce a new '--bitmask-list' command-line option for 'perf trace'.
When this option is specified, the formatting of cpumasks is delegated
to bitmap_scnprintf(), enabling cpumasks to be displayed as a condensed,
human-readable list (e.g., "0,2-5,7") instead of the default hexadecimal
representation. An example is provided below:

❯ sudo ./perf trace --show-cpu --bitmask-list --event ipi:ipi_send_cpumask --max-event 5
     0.000 [000] Xorg/1434 ipi:ipi_send_cpumask(cpumask: 2-3,6, callsite: 0xffffffff9994f8e4, callback: 0xffffffff9994fdd0)
   694.527 [002] chrome/2894 ipi:ipi_send_cpumask(cpumask: 1,3-5, callsite: 0xffffffff9994f8e4, callback: 0xffffffff9994fdd0)
  2666.608 [003] Chrome_ChildIO/2948 ipi:ipi_send_cpumask(cpumask: 4,7, callsite: 0xffffffff9994f8e4, callback: 0xffffffff9994fdd0)
  2673.638 [000] Chrome_IOThrea/2920 ipi:ipi_send_cpumask(cpumask: 2-5, callsite: 0xffffffff9994f8e4, callback: 0xffffffff9994fdd0)
  2714.228 [005] chrome/3375 ipi:ipi_send_cpumask(cpumask: 0-4,6-7, callsite: 0xffffffff9994f8e4, callback: 0xffffffff9994fdd0)

Signed-off-by: Aaron Tomlin 
Signed-off-by: Namhyung Kim

perf stat: Add --hide-zero-events option to suppress zero-count events

2026-07-20T04:49:30+00:00

When monitoring a large number of events (e.g., with wildcards such as
--event 'syscalls:sys_enter_*'), many matched events will return a count
of zero. This clutters the output, making it difficult to spot the
active events.

Add a new option --hide-zero-events to suppress printing events that
have a count of zero.

To prevent formatting and diagnostic issues, the zero-skipping logic
implements the following rules:

    1. In metric-only mode (i.e., --metric-only), columns must remain
       aligned in the output grid. We evaluate config->metric_only first
       to avoid skipping zero-valued columns, preventing values from
       shifting left and aligning under incorrect headers

    2. For explicitly requested events, we ensure they are not silently
       hidden if they are unsupported. We only hide a zero-count event
       if counter->supported is true, ensuring that unsupported explicit
       events still report ""

Signed-off-by: Aaron Tomlin 
Reviewed-by: Ian Rogers 
Signed-off-by: Namhyung Kim

perf doc: Fix mmap failure checks in topdown example

2026-07-15T18:16:41+00:00

Use MAP_FAILED instead of NULL to detect mmap errors, and fix the
slots_p variable name typo in the sample code.

Signed-off-by: Hongfu Li 
Signed-off-by: Namhyung Kim

perf test: Add Arm CoreSight callchain test

2026-07-03T23:52:00+00:00

Add a CoreSight shell test for synthesized callchains.

The test uses the new callchain workload to generate trace and decodes
it with synthesis callchain. It then verifies that the instruction
samples show the expected callchain push and pop.

Use control FIFOs so tracing starts only around the workload, which
keeps the trace data small. The test is limited to with the cs_etm
event available and root permission.

After:

  perf test 138 -vvv
  138: CoreSight synthesized callchain:
  ---- start ----
  test child forked, pid 35581
  Callchain flow matched:
    l1=4642868 l2=4642880 l3=4642895 l4=4642919 l5=4670494 l6=4670500 l7=4670520
  ---- end(0) ----
  138: CoreSight synthesized callchain                                                                           : Ok

Assisted-by: Codex:GPT-5.5
Reviewed-by: James Clark 
Signed-off-by: Leo Yan 
Signed-off-by: Namhyung Kim

perf tools: Document recent additions to the perf.data file header

2026-06-15T19:54:39+00:00

Add documentation for recently added HEADER_E_MACHINE and
HEADER_CLN_SIZE data to the perf.data file. Also fix a typo
at the end of the header section.

Reviewed-by: Ian Rogers 
Signed-off-by: Thomas Falcon 
Cc: Adrian Hunter 
Cc: Alexander Shishkin 
Cc: Dapeng Mi 
Cc: Ingo Molnar 
Cc: James Clark 
Cc: Jiri Olsa 
Cc: Mark Rutland 
Cc: Namhyung Kim 
Cc: Peter Zijlstra 
Signed-off-by: Arnaldo Carvalho de Melo

perf test: Add named_threads workload

2026-06-10T21:55:49+00:00

Add a workload that runs X threads that run a unique function named
"named_threads_thread[x]" which performs a multiplication in a loop for
Y loops. Each thread sets its name to "thread[x]".

This can be used to test that processor trace decoding handles
concurrent threads correctly and the correct symbols and thread names
are assigned to samples.

Signed-off-by: James Clark 
Cc: Amir Ayupov 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Jonathan Corbet 
Cc: Leo Yan 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Paschalis Mpeis 
Cc: Shuah Khan 
Cc: Suzuki Poulouse 
Signed-off-by: Arnaldo Carvalho de Melo

perf test: Add deterministic workload

2026-06-10T21:55:48+00:00

Add a workload that does the same thing every time for testing CPU trace
decoding.

Reviewed-by: Leo Yan 
Signed-off-by: James Clark 
Cc: Amir Ayupov 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Jonathan Corbet 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Paschalis Mpeis 
Cc: Shuah Khan 
Cc: Suzuki Poulouse 
Signed-off-by: Arnaldo Carvalho de Melo

perf test: Add a workload that forces context switches

2026-06-10T21:55:48+00:00

This workload launches two processes that block when reading and writing
to each other forcing the other process to be scheduled for each
read/write pair.

Signed-off-by: James Clark 
Cc: Amir Ayupov 
Cc: Ian Rogers 
Cc: Jiri Olsa 
Cc: Jonathan Corbet 
Cc: Leo Yan 
Cc: Mike Leach 
Cc: Namhyung Kim 
Cc: Paschalis Mpeis 
Cc: Shuah Khan 
Cc: Suzuki Poulouse 
Signed-off-by: Arnaldo Carvalho de Melo