linux-toradex.git/kernel/events/ring_buffer.c, branch v3.14.3

perf: Optimize ring-buffer write by depending on control dependencies

2013-12-11T14:53:22+00:00

Remove a full barrier from the ring-buffer write path by relying on
a control dependency to order a LOAD -> STORE scenario.

Cc: "Paul E. McKenney" 
Signed-off-by: Peter Zijlstra 
Link: http://lkml.kernel.org/n/tip-8alv40z6ikk57jzbaobnxrjl@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Update a stale comment

2013-11-06T11:34:23+00:00

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-9s5mze78gmlz19agt39i8rii@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Optimize perf_output_begin() -- address calculation

2013-11-06T11:34:22+00:00

Rewrite the handle address calculation code to be clearer.

Saves 8 bytes on x86_64-defconfig.

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-3trb2n2henb9m27tncef3ag7@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Optimize perf_output_begin() -- lost_event case

2013-11-06T11:34:21+00:00

Avoid touching the lost_event and sample_data cachelines twince. Its
not like we end up doing less work, but it might help to keep all
accesses to these cachelines in one place.

Due to code shuffle, this looses 4 bytes on x86_64-defconfig.

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-zfxnc58qxj0eawdoj31hhupv@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Optimize perf_output_begin()

2013-11-06T11:34:20+00:00

There's no point in re-doing the memory-barrier when we fail the
cmpxchg(). Also placing it after the space reservation loop makes it
clearer it only separates the userpage->tail read from the data
stores.

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-c19u6egfldyx86tpyc3zgkw9@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Add unlikely() to the ring-buffer code

2013-11-06T11:34:19+00:00

Add unlikely() annotations to 'slow' paths:

When having a sampling event but no output buffer; you have bigger
issues -- also the bail is still faster than actually doing the work.

When having a sampling event but a control page only buffer, you have
bigger issues -- again the bail is still faster than actually doing
work.

Optimize for the case where you're not loosing events -- again, not
doing the work is still faster but make sure that when you have to
actually do work its as fast as possible.

The typical watermark is 1/2 the buffer size, so most events will not
take this path.

Shrinks perf_output_begin() by 16 bytes on x86_64-defconfig.

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-wlg3jew3qnutm8opd0hyeuwn@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Simplify the ring-buffer code

2013-11-06T11:34:18+00:00

By using CIRC_SPACE() we can obviate the need for perf_output_space().

Shrinks the size of perf_output_begin() by 17 bytes on
x86_64-defconfig.

Signed-off-by: Peter Zijlstra 
Cc: Benjamin Herrenschmidt 
Cc: Frederic Weisbecker 
Cc: Mathieu Desnoyers 
Cc: Michael Ellerman 
Cc: Michael Neuling 
Cc: "Paul E. McKenney" 
Cc: james.hogan@imgtec.com
Cc: Vince Weaver 
Cc: Victor Kaplansky 
Cc: Oleg Nesterov 
Cc: Anton Blanchard 
Link: http://lkml.kernel.org/n/tip-vtb0xb0llebmsdlfn1v5vtfj@git.kernel.org
Signed-off-by: Ingo Molnar

perf: Fix perf ring buffer memory ordering

2013-10-29T11:01:19+00:00

The PPC64 people noticed a missing memory barrier and crufty old
comments in the perf ring buffer code. So update all the comments and
add the missing barrier.

When the architecture implements local_t using atomic_long_t there
will be double barriers issued; but short of introducing more
conditional barrier primitives this is the best we can do.

Reported-by: Victor Kaplansky 
Tested-by: Victor Kaplansky 
Signed-off-by: Peter Zijlstra 
Cc: Mathieu Desnoyers 
Cc: michael@ellerman.id.au
Cc: Paul McKenney 
Cc: Michael Neuling 
Cc: Frederic Weisbecker 
Cc: anton@samba.org
Cc: benh@kernel.crashing.org
Link: http://lkml.kernel.org/r/20131025173749.GG19466@laptop.lan
Signed-off-by: Ingo Molnar

perf: Fix vmalloc ring buffer pages handling

2013-05-01T10:34:46+00:00

If we allocate perf ring buffer with the size of single (user)
page, we will get memory corruption when releasing itin
rb_free_work function (for CONFIG_PERF_USE_VMALLOC option).

For single page sized ring buffer the page_order is -1 (because
nr_pages is 0). This needs to be recognized in the rb_free_work
function to release proper amount of pages.

Adding data_page_nr function that returns number of allocated
data pages. Customizing the rest of the code to use it.

Reported-by: Jan Stancek 
Original-patch-by: Peter Zijlstra 
Acked-by: Peter Zijlstra 
Cc: Corey Ashford 
Cc: Frederic Weisbecker 
Cc: Ingo Molnar 
Cc: Namhyung Kim 
Cc: Paul Mackerras 
Cc: Arnaldo Carvalho de Melo 
Signed-off-by: Jiri Olsa 
Link: http://lkml.kernel.org/r/20130319143509.GA1128@krava.brq.redhat.com
Signed-off-by: Ingo Molnar

perf: Fix ring_buffer perf_output_space() boundary calculation

2013-03-21T11:04:35+00:00

This patch fixes a flaw in perf_output_space(). In case the size
of the space needed is bigger than the actual buffer size, there
may be situations where the function would return true (i.e.,
there is space) when it should not. head > offset due to
rounding of the masking logic.

The problem can be tested by activating BTS on Intel processors.
A BTS record can be as big as 16 pages. The following command
fails:

  $ perf record -m 4 -c 1 -e branches:u my_test_program

You will get a buffer corruption with this. Perf report won't be
able to parse the perf.data.

The fix is to first check that the requested space is smaller
than the buffer size. If so, then the masking logic will work
fine. If not, then there is no chance the record can be saved
and it will be gracefully handled by upper code layers.

[ In v2, we also make the logic for the writable more explicit by
  renaming it to rb->overwrite because it tells whether or not the
  buffer can overwrite its tail (suggested by PeterZ). ]

Signed-off-by: Stephane Eranian 
Acked-by: Peter Zijlstra 
Cc: peterz@infradead.org
Cc: jolsa@redhat.com
Cc: fweisbec@gmail.com
Link: http://lkml.kernel.org/r/20130318133327.GA3056@quad
Signed-off-by: Ingo Molnar