Instead of using zalloc(nr_entries * sizeof_entry) that is what calloc() does. In some places where linux/zalloc.h isn't needed, remove it, add when needed and was getting it indirectly. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The hashmap__new() function never returns NULL, it returns error
pointers. Fix the error checking to match.
Additionally, set ftrace->profile_hash to NULL on error, and return the
exact error code from hashmap__new().
Fixes: 0f223813edd051a5 ("perf ftrace: Add 'profile' command")
Suggested-by: Ian Rogers <irogers@google.com>
Signed-off-by: Chen Ni <nichen@iscas.ac.cn>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Clark <james.clark@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This change adds support for new funcgraph tracer options funcgraph-args,
funcgraph-retval, funcgraph-retval-hex and funcgraph-retaddr.
The new added options are:
- args : Show function arguments.
- retval : Show function return value.
- retval-hex : Show function return value in hexadecimal format.
- retaddr : Show function return address.
# ./perf ftrace -G vfs_write --graph-opts retval,retaddr
# tracer: function_graph
#
# CPU DURATION FUNCTION CALLS
# | | | | | | |
5) | mutex_unlock() { /* <-rb_simple_write+0xda/0x150 */
5) 0.188 us | local_clock(); /* <-lock_release+0x2ad/0x440 ret=0x3bf2a3cf90e */
5) | rt_mutex_slowunlock() { /* <-rb_simple_write+0xda/0x150 */
5) | _raw_spin_lock_irqsave() { /* <-rt_mutex_slowunlock+0x4f/0x200 */
5) 0.123 us | preempt_count_add(); /* <-_raw_spin_lock_irqsave+0x23/0x90 ret=0x0 */
5) 0.128 us | local_clock(); /* <-__lock_acquire.isra.0+0x17a/0x740 ret=0x3bf2a3cfc8b */
5) 0.086 us | do_raw_spin_trylock(); /* <-_raw_spin_lock_irqsave+0x4a/0x90 ret=0x1 */
5) 0.845 us | } /* _raw_spin_lock_irqsave ret=0x292 */
5) | _raw_spin_unlock_irqrestore() { /* <-rt_mutex_slowunlock+0x191/0x200 */
5) 0.097 us | local_clock(); /* <-lock_release+0x2ad/0x440 ret=0x3bf2a3cff1f */
5) 0.086 us | do_raw_spin_unlock(); /* <-_raw_spin_unlock_irqrestore+0x23/0x60 ret=0x1 */
5) 0.104 us | preempt_count_sub(); /* <-_raw_spin_unlock_irqrestore+0x35/0x60 ret=0x0 */
5) 0.726 us | } /* _raw_spin_unlock_irqrestore ret=0x80000000 */
5) 1.881 us | } /* rt_mutex_slowunlock ret=0x0 */
5) 2.931 us | } /* mutex_unlock ret=0x0 */
Signed-off-by: Changbin Du <changbin.du@huawei.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250613114048.132336-1-changbin.du@huawei.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
In addition to the function latency, it can measure events latencies.
Some kernel tracepoints are paired and it's menningful to measure how
long it takes between the two events. The latency is tracked for the
same thread.
Currently it only uses BPF to do the work but it can be lifted later.
Instead of having separate a BPF program for each tracepoint, it only
uses generic 'event_begin' and 'event_end' programs to attach to any
(raw) tracepoints.
$ sudo perf ftrace latency -a -b --hide-empty \
-e i915_request_wait_begin,i915_request_wait_end -- sleep 1
# DURATION | COUNT | GRAPH |
256 - 512 us | 4 | ###### |
2 - 4 ms | 2 | ### |
4 - 8 ms | 12 | ################### |
8 - 16 ms | 10 | ################ |
# statistics (in usec)
total time: 194915
avg time: 6961
max time: 12855
min time: 373
count: 28
Reviewed-by: Ian Rogers <irogers@google.com>
Link: https://lore.kernel.org/r/20250714052143.342851-1-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Gathering threads with a uid by scanning /proc is inherently racy leading to perf_event_open failures that quit perf. All users of the functionality now use BPF filters, so remove uid and uid_str from target. Signed-off-by: Ian Rogers <irogers@google.com> Link: https://lore.kernel.org/r/20250604174545.2853620-10-irogers@google.com Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Executing 'perf ftrace' commands 'ftrace', 'profile' and 'latency' leave tracing disabled as can seen in this output: # echo 1 > /sys/kernel/debug/tracing/tracing_on # cat /sys/kernel/debug/tracing/tracing_on 1 # perf ftrace trace --graph-opts depth=5 sleep 0.1 > /dev/null # cat /sys/kernel/debug/tracing/tracing_on 0 # The 'tracing_on' file is not restored to its value before the command. To fix that this patch uses the .../tracing/instances/XXX subdirectory feature. Each 'perf ftrace' invocation creates its own session/process specific subdirectory and does not change the global state in the .../tracing directory itself. Use rmdir(../tracing/instances/dir) to stop process/session specific tracing and delete all process/session specific setings. Reported-by: Alexander Egorenkov <egorenar@linux.ibm.com> Suggested-by: Arnaldo Carvalho de Melo <acme@kernel.org> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Sumanth Korikkar <sumanthk@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Link: https://lore.kernel.org/r/20250520093726.2009696-1-tmricht@linux.ibm.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Especially while using several buckets, it isn't uncommon to have some
of them empty and reading the histogram may be a bit more complex:
# perf ftrace latency -a -T mutex_lock --bucket-range 5 --max-latency 200
# DURATION | COUNT | GRAPH |
0 - 5 us | 14816 | ###################################### |
5 - 10 us | 1228 | ### |
10 - 15 us | 438 | # |
15 - 20 us | 106 | |
20 - 25 us | 21 | |
25 - 30 us | 11 | |
30 - 35 us | 1 | |
35 - 40 us | 2 | |
40 - 45 us | 4 | |
45 - 50 us | 0 | |
50 - 55 us | 1 | |
55 - 60 us | 0 | |
60 - 65 us | 1 | |
65 - 70 us | 1 | |
70 - 75 us | 1 | |
75 - 80 us | 2 | |
80 - 85 us | 0 | |
85 - 90 us | 1 | |
90 - 95 us | 0 | |
95 - 100 us | 1 | |
100 - 105 us | 0 | |
105 - 110 us | 0 | |
110 - 115 us | 0 | |
115 - 120 us | 0 | |
120 - 125 us | 1 | |
125 - 130 us | 0 | |
130 - 135 us | 0 | |
135 - 140 us | 1 | |
140 - 145 us | 0 | |
145 - 150 us | 0 | |
150 - 155 us | 0 | |
155 - 160 us | 0 | |
160 - 165 us | 0 | |
165 - 170 us | 0 | |
170 - 175 us | 0 | |
175 - 180 us | 0 | |
180 - 185 us | 0 | |
185 - 190 us | 0 | |
190 - 195 us | 0 | |
195 - 200 us | 0 | |
200 - ... us | 2 | |
Allow the optional flag --hide-empty to remove buckets with no element
and produce a more compact graph. This feature could be misleading since
there is no clear indication for missing buckets, for this reason it's
disabled by default.
# perf ftrace latency -a -T mutex_lock --bucket-range 5 --max-latency --hide-empty 200
# DURATION | COUNT | GRAPH |
0 - 5 us | 14816 | ###################################### |
5 - 10 us | 1228 | ### |
10 - 15 us | 438 | # |
15 - 20 us | 106 | |
20 - 25 us | 21 | |
25 - 30 us | 11 | |
30 - 35 us | 1 | |
35 - 40 us | 2 | |
40 - 45 us | 4 | |
50 - 55 us | 1 | |
60 - 65 us | 1 | |
65 - 70 us | 1 | |
70 - 75 us | 1 | |
75 - 80 us | 2 | |
85 - 90 us | 1 | |
95 - 100 us | 1 | |
120 - 125 us | 1 | |
135 - 140 us | 1 | |
200 - ... us | 2 | |
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Link: https://lore.kernel.org/r/20250207080446.77630-2-gmonaco@redhat.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
The max-latency value can make the histogram smaller, but not larger, we have a maximum of 22 buckets and specifying a max-latency that would require more buckets has no effect. Dynamically allocate the buckets and compute the bucket number from the max latency as (max-min) / range + 2 If the maximum is not specified, we still set the bucket number to 22 and compute the maximum accordingly. Fail if the maximum is smaller than min+range, this way we make sure we always have 3 buckets: those below min, those above max and one in the middle. Since max-latency is not available in log2 mode, always use 22 buckets. Signed-off-by: Gabriele Monaco <gmonaco@redhat.com> Link: https://lore.kernel.org/r/20250207080446.77630-1-gmonaco@redhat.com Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When min_latency is not given, it prints 0 - 0. It should be 0 - 1.
Before:
$ sudo ./perf ftrace latency -a -T do_futex sleep 1
# DURATION | COUNT | GRAPH |
0 - 0 us | 321 | ########### |
...
After:
$ sudo ./perf ftrace latency -a -T do_futex sleep 1
# DURATION | COUNT | GRAPH |
0 - 1 us | 699 | ############ |
...
Fixes: 08b875b6bf608589 ("perf ftrace latency: Introduce --min-latency to narrow down into a latency range")
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Gabriele Monaco <gmonaco@redhat.com
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250108210015.1188531-2-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It's an optional feature and remains 0 when bucket range is not given.
And it makes the histogram goes to the last entry always because any
latency (num) is greater than or equal to 0.
Before:
$ sudo ./perf ftrace latency -a -T do_futex sleep 1
# DURATION | COUNT | GRAPH |
0 - 0 us | 0 | |
1 - 2 us | 0 | |
2 - 4 us | 0 | |
4 - 8 us | 0 | |
8 - 16 us | 0 | |
16 - 32 us | 0 | |
32 - 64 us | 0 | |
64 - 128 us | 0 | |
128 - 256 us | 0 | |
256 - 512 us | 0 | |
512 - 1024 us | 0 | |
1 - 2 ms | 0 | |
2 - 4 ms | 0 | |
4 - 8 ms | 0 | |
8 - 16 ms | 0 | |
16 - 32 ms | 0 | |
32 - 64 ms | 0 | |
64 - 128 ms | 0 | |
128 - 256 ms | 0 | |
256 - 512 ms | 0 | |
512 - 1024 ms | 0 | |
1 - ... s | 1353 | ############################################## |
After:
$ sudo ./perf ftrace latency -a -T do_futex sleep 1
# DURATION | COUNT | GRAPH |
0 - 0 us | 321 | ########### |
1 - 2 us | 132 | #### |
2 - 4 us | 202 | ####### |
4 - 8 us | 188 | ###### |
8 - 16 us | 16 | |
16 - 32 us | 12 | |
32 - 64 us | 30 | # |
64 - 128 us | 98 | ### |
128 - 256 us | 53 | # |
256 - 512 us | 57 | ## |
512 - 1024 us | 9 | |
1 - 2 ms | 9 | |
2 - 4 ms | 1 | |
4 - 8 ms | 98 | ### |
8 - 16 ms | 5 | |
16 - 32 ms | 7 | |
32 - 64 ms | 32 | # |
64 - 128 ms | 10 | |
128 - 256 ms | 10 | |
256 - 512 ms | 2 | |
512 - 1024 ms | 0 | |
1 - ... s | 0 | |
Fixes: 690a052a6d85c530 ("perf ftrace latency: Add --max-latency option")
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Gabriele Monaco <gmonaco@redhat.com
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250108210015.1188531-1-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>