linux-toradex.git/tools/perf/scripts, branch v4.4-rc6

perf tools: Add more documentation to export-to-postgresql.py script

2015-09-28T19:53:07+00:00

Add some comments to the script and some 'views' to the created database
that better illustrate the database structure and how it can be used.

Signed-off-by: Adrian Hunter 
Cc: Jiri Olsa 
Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf scripts python: Add new compaction-times script

2015-08-28T17:53:40+00:00

This patch creates a new script (compaction-times) to report time
spent in mm compaction. It is possible to report times in nanoseconds
(default) or microseconds (-u).

The option -p will break down results by process id, -pv will further
decompose by each compaction entry/exit.

For each compaction entry/exit what is reported is controlled by the
options:

  -t   report only timing
  -m   report migration stats
  -ms  report migration scanner stats
  -fs  report free scanner stats

The default is to report all.

Entries may be further filtered by pid, pid-range or comm (regex).

The script is useful when analysing workloads that compact memory. The
most common example will be THP allocations on systems with a lot of
uptime that has fragmented memory.

This is an example of using the script to analyse a thpscale from
mmtests which deliberately fragments memory and allocates THP in 4
separate threads

  # Recording step, one of the following;
  $ perf record -e 'compaction:mm_compaction_*' ./workload
  # or:
  $ perf script record compaction-times

  # Reporting: basic
  total: 2444505743ns migration: moved=357738 failed=39275
  free_scanner: scanned=2705578 isolated=387875
  migration_scanner: scanned=414426 isolated=397013

  # Reporting: Per task stall times
  $ perf script report compaction-times -- -t -p
  total: 2444505743ns
  6384[thpscale]: 740800017ns
  6385[thpscale]: 274119512ns
  6386[thpscale]: 832961337ns
  6383[thpscale]: 596624877ns

  # Reporting: Per-compaction attempts for task 6385
  $ perf script report compaction-times -- -m -pv 6385
  total: 274119512ns migration: moved=14893 failed=24285
  6385[thpscale]: 274119512ns migration: moved=14893 failed=24285
  6385[thpscale].1: 3033277ns migration: moved=511 failed=1
  6385[thpscale].2: 9592094ns migration: moved=1524 failed=12
  6385[thpscale].3: 2495587ns migration: moved=512 failed=0
  6385[thpscale].4: 2561766ns migration: moved=512 failed=0
  6385[thpscale].5: 2523521ns migration: moved=512 failed=0
  ..... output continues ...

Changes since v1:
- report stats for isolate_migratepages and isolate_freepages
  (Vlastimil Babka)
- refactor code to achieve above
- add help text
- output to stdout/stderr explicitly

Signed-off-by: Tony Jones 
Cc: Mel Gorman 
Cc: Vlastimil Babka 
Link: http://lkml.kernel.org/r/1439840932-8933-1-git-send-email-tonyj@suse.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add example call-graph script

2015-08-21T15:32:40+00:00

Add a script to produce a call-graph from data exported to a postgresql
database and derived from a processor trace event like intel_pt or intel_bts.

Refer to comments in the scripts call-graph-from-postgresql.py and
export-to-postgresql.py for more details on how to set up the environment,
install the required packages, etc.

Committer note:

From the scripts, for convenience while reading 'git log':

  An example of using this script with Intel PT:

  $ perf record -e intel_pt//u ls
  $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls
  2015-05-29 12:49:23.464364 Creating database...
  2015-05-29 12:49:26.281717 Writing to intermediate files...
  2015-05-29 12:49:27.190383 Copying to database...
  2015-05-29 12:49:28.140451 Removing intermediate files...
  2015-05-29 12:49:28.147451 Adding primary keys
  2015-05-29 12:49:28.655683 Adding foreign keys
  2015-05-29 12:49:29.365350 Done
  $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example
  # The result is a GUI window with a tree representing a context-sensitive
  # call-graph.  Expanding a couple of levels of the tree and adjusting column
  # widths to suit will display something like:

                                         Call Graph: pt_example
  Call Path                        |Object     |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%)
  v- ls
     v- 2638:2638
         v- _start                  ld-2.19.so    1   10074071  100.0        211135          100.0
           |- unknown               unknown       1      13198    0.1             1            0.0
           >- _dl_start             ld-2.19.so    1    1400980   13.9         19637            9.3
           >- _d_linit_internal     ld-2.19.so    1     448152    4.4         11094            5.3
           v-__libc_start_main@plt  ls            1    8211741   81.5        180397           85.4
              >- _dl_fixup          ld-2.19.so    1       7607    0.1           108            0.1
              >- __cxa_atexit       libc-2.19.so  1      11737    0.1            10            0.0
              >- __libc_csu_init    ls            1      10354    0.1            10            0.0
              |- _setjmp            libc-2.19.so  1          0    0.0             4            0.0
              v- main               ls            1    8182043   99.6        180254           99.9

Signed-off-by: Adrian Hunter 
Tested-by: Arnaldo Carvalho de Melo 
Cc: Jiri Olsa 
Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com
[ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ]
Signed-off-by: Arnaldo Carvalho de Melo

perf build: Add scripts objects building

2015-02-12T14:49:53+00:00

Move the scripts objects building under build framework to be included
in the libperf build object.

Signed-off-by: Jiri Olsa 
Tested-by: Sukadev Bhattiprolu 
Tested-by: Will Deacon 
Cc: Alexis Berlemont 
Cc: Borislav Petkov 
Cc: Corey Ashford 
Cc: David Ahern 
Cc: Frederic Weisbecker 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lkml.kernel.org/n/tip-ry8pd41ahwpq9h46i8te33c7@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo

perf scripting perl: Force to use stdbool

2015-01-21T13:05:00+00:00

When building perf for arm64 I hit a warning (and be treated as an
error) like below:

 aarch64-oe-linux-gcc -o .../scripts/perl/Perf-Trace-Util/Context.o -c -Wbad-function-cast \
         ... scripts/perl/Perf-Trace-Util/Context.c

 In file included from .../usr/lib64/perl/5.14.3/CORE/perl.h:2464:0,
                  from Context.xs:23:
 /.../usr/lib64/perl/5.14.3/CORE/handy.h:108:0: error: "bool" redefined [-Werror]
  #  define bool char
  ^
 In file included from /.../usr/src/kernel/tools/include/linux/types.h:4:0,
                  from /.../usr/src/kernel/arch/arm64/include/uapi/asm/sigcontext.h:19,
		  from /.../usr/include/bits/sigcontext.h:27,
		  from /.../usr/include/signal.h:340,
		  from /.../usr/include/sys/param.h:28,
		  from /.../usr/lib64/perl/5.14.3/CORE/perl.h:678,
		  from Context.xs:23:
  /.../usr/lib/aarch64-oe-linux/gcc/aarch64-oe-linux/4.9.2/include/stdbool.h:33:0: note: this is the location of the previous definition
    #define bool _Bool

Looks like the failure is caused by arm64 uapi/asm/sigcontext.h, which
includes linux/types.h while other archs not.

Current perl consider this problem:

http://perl5.git.perl.org/perl.git/commit/bd31be4baa3ee68abdb92c0db3200efe0fad903b

However there are users which use old version of perl.

This patch includes stdbool.h before Context.xs and define HAS_BOOL to
prevent perl'e headers define its own 'bool'. Code is learn from perl's
git tree.

Signed-off-by: Wang Nan 
Acked-by: Jiri Olsa 
Acked-by: Namhyung Kim 
Cc: Li Zefan 
Link: http://lkml.kernel.org/r/1421671397-4659-1-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add call information to Python export

2014-11-03T21:10:06+00:00

Add the ability to export detailed information about paired calls and
returns to Python db export and the export-to-postgresql.py script.

Signed-off-by: Adrian Hunter 
Cc: David Ahern 
Cc: Frederic Weisbecker 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add branch_type and in_tx to Python export

2014-11-03T21:07:34+00:00

Add branch_type and in_tx to Python db export and the
export-to-postgresql.py script.

Signed-off-by: Adrian Hunter 
Cc: David Ahern 
Cc: Frederic Weisbecker 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script: Add Python script to export to postgresql

2014-10-29T12:32:49+00:00

Add a Python script to export to a postgresql database.

The script requires the Python pyside module and the Qt PostgreSQL
driver.  The packages needed are probably named "python-pyside" and
"libqt4-sql-psql"

The caller of the script must be able to create postgresql databases.

The script takes the database name as a parameter.  The database and
database tables are created.  Data is written to flat files which are
then imported using SQL COPY FROM.

Example:

  $ perf record ls
  ...
  $ perf script report export-to-postgresql lsdb
  2014-02-14 10:55:38.631431 Creating database...
  2014-02-14 10:55:39.291958 Writing to intermediate files...
  2014-02-14 10:55:39.350280 Copying to database...
  2014-02-14 10:55:39.358536 Removing intermediate files...
  2014-02-14 10:55:39.358665 Adding primary keys
  2014-02-14 10:55:39.658697 Adding foreign keys
  2014-02-14 10:55:39.667412 Done
  $ psql lsdb
  lsdb-# \d
              List of relations
   Schema |      Name       | Type  | Owner
  --------+-----------------+-------+-------
   public | comm_threads    | table | acme
   public | comms           | table | acme
   public | dsos            | table | acme
   public | machines        | table | acme
   public | samples         | table | acme
   public | samples_view    | view  | acme
   public | selected_events | table | acme
   public | symbols         | table | acme
   public | threads         | table | acme
  (9 rows)
  lsdb-# \d samples
         Table "public.samples"
      Column     |  Type   | Modifiers
  ---------------+---------+-----------
   id            | bigint  | not null
   evsel_id      | bigint  |
   machine_id    | bigint  |
   thread_id     | bigint  |
   comm_id       | bigint  |
   dso_id        | bigint  |
   symbol_id     | bigint  |
   sym_offset    | bigint  |
   ip            | bigint  |
   time          | bigint  |
   cpu           | integer |
   to_dso_id     | bigint  |
   to_symbol_id  | bigint  |
   to_sym_offset | bigint  |
   to_ip         | bigint  |
   period        | bigint  |
   weight        | bigint  |
   transaction   | bigint  |
   data_src      | bigint  |
  Indexes:
      "samples_pkey" PRIMARY KEY, btree (id)
  Foreign-key constraints:
      "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id)
      "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id)
      "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id)
      "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id)
      "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id)
      "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id)
      "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id)
      "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id)

  lsdb-# \d samples_view
                 View "public.samples_view"
        Column       |          Type           | Modifiers
  -------------------+-------------------------+-----------
   id                | bigint                  |
   time              | bigint                  |
   cpu               | integer                 |
   pid               | integer                 |
   tid               | integer                 |
   command           | character varying(16)   |
   event             | character varying(80)   |
   ip_hex            | text                    |
   symbol            | character varying(2048) |
   sym_offset        | bigint                  |
   dso_short_name    | character varying(256)  |
   to_ip_hex         | text                    |
   to_symbol         | character varying(2048) |
   to_sym_offset     | bigint                  |
   to_dso_short_name | character varying(256)  |

    lsdb=# select * from samples_view;

   id| time       |cpu | pid  | tid  |command| event  |   ip_hex      |           symbol    |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name
   --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+----------
   1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown

My notes (which may be out-of-date) on setting up postgresql so you can
create databases:

fedora:

        $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql
        $ sudo su - postgres -c initdb
        $ sudo service postgresql start
        $ sudo su - postgres
        $ createuser -s 

I used the the unix user name in createuser.

If it fails, try createuser without -s and answer the following question
to allow your user to create tables:

        Shall the new role be a superuser? (y/n) y

ubuntu:

        $ sudo apt-get install postgresql
        $ sudo su - postgres
        $ createuser 
        Shall the new role be a superuser? (y/n) y

You may want to disable automatic startup.  One way is to edit
/etc/postgresql/9.3/main/start.conf.  Another is to disable the init
script e.g. sudo update-rc.d postgresql disable

Signed-off-by: Adrian Hunter 
Cc: David Ahern 
Cc: Frederic Weisbecker 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Stephane Eranian 
Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo

perf script: Add callchain to generic and tracepoint events

2014-07-16T20:57:33+00:00

This provides valuable information for tracing performance problems.

Since this change alters the interface for the python scripts, also
adjust the script generation and the provided scripts.

Signed-off-by: Joseph Schuchart 
Acked-by: Thomas Ilsche 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Cc: Thomas Ilsche 
Link: http://lkml.kernel.org/r/53BE7E1B.10503@tu-dresden.de
Signed-off-by: Arnaldo Carvalho de Melo

perf scripts: Fallback to syscalls:* when raw_syscalls:* is not available

2014-06-25T15:26:56+00:00

Older kernels (e.g., RHEL6) do system call tracing via the
syscalls:sys_{enter,exit} tracepoints rather than using raw_syscalls:*.

Update perf python and perl scripts to fallback to syscalls:* when
raw_syscalls:* isn't available.

Signed-off-by: Daniel Bristot de Oliveira 
Cc: Ingo Molnar 
Cc: Luis Claudio R. Goncalves 
Cc: Paul Mackerras 
Cc: Peter Zijlstra 
Link: http://lkml.kernel.org/r/5a6c64081a3375bc3bc66351b14559678ef4d71e.1402507908.git.bristot@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo