Backport for 2.6.30-stable of: 469535a ring-buffer: Fix advance of reader in rb_buffer_peek() When calling rb_buffer_peek() from ring_buffer_consume() and a padding event is returned, the function rb_advance_reader() is called twice. This may lead to missing samples or under high workloads to the warning below. This patch fixes this. If a padding event is returned by rb_buffer_peek() it will be consumed by the calling function now. Also, I simplified some code in ring_buffer_consume(). ------------[ cut here ]------------ WARNING: at /dev/shm/.source/linux/kernel/trace/ring_buffer.c:2289 rb_advance_reader+0x2e/0xc5() Hardware name: Anaheim Modules linked in: Pid: 29, comm: events/2 Tainted: G W 2.6.31-rc3-oprofile-x86_64-standard-00059-g5050dc2 #1 Call Trace: [<ffffffff8106776f>] ? rb_advance_reader+0x2e/0xc5 [<ffffffff81039ffe>] warn_slowpath_common+0x77/0x8f [<ffffffff8103a025>] warn_slowpath_null+0xf/0x11 [<ffffffff8106776f>] rb_advance_reader+0x2e/0xc5 [<ffffffff81068bda>] ring_buffer_consume+0xa0/0xd2 [<ffffffff81326933>] op_cpu_buffer_read_entry+0x21/0x9e [<ffffffff810be3af>] ? __find_get_block+0x4b/0x165 [<ffffffff8132749b>] sync_buffer+0xa5/0x401 [<ffffffff810be3af>] ? __find_get_block+0x4b/0x165 [<ffffffff81326c1b>] ? wq_sync_buffer+0x0/0x78 [<ffffffff81326c76>] wq_sync_buffer+0x5b/0x78 [<ffffffff8104aa30>] worker_thread+0x113/0x1ac [<ffffffff8104dd95>] ? autoremove_wake_function+0x0/0x38 [<ffffffff8104a91d>] ? worker_thread+0x0/0x1ac [<ffffffff8104dc9a>] kthread+0x88/0x92 [<ffffffff8100bdba>] child_rip+0xa/0x20 [<ffffffff8104dc12>] ? kthread+0x0/0x92 [<ffffffff8100bdb0>] ? child_rip+0x0/0x20 ---[ end trace f561c0a58fcc89bd ]--- Cc: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Robert Richter <robert.richter@amd.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
commit bd3f02212d6a457267e0c9c02c426151c436d9d4 upstream. I noticed oprofile memleaked in linux-2.6 current tree, and tracked this ring-buffer leak. Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com> LKML-Reference: <4A7C06B9.2090302@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Impact: prevent possible memory leak
The reader page of the ring buffer is special. Although it points
into the ring buffer, it is not part of the actual buffer. It is
a page used by the reader to swap with a page in the ring buffer.
Once the swap is made, the new reader page is again outside the
buffer.
Even though the reader page points into the buffer, it is really
pointing to residual data. Note, this data is used by the reader.
reader page
|
v
(prev) +---+ (next)
+----------| |----------+
| +---+ |
v v
+---+ +---+ +---+
-->| |------->| |------->| |--->
<--| |<-------| |<-------| |<---
+---+ +---+ +---+
^ ^ ^
\ | /
------- Buffer---------
If we perform a list_del_init() on the reader page we will actually remove
the last page the reader swapped with and not the reader page itself.
This will cause that page to not be freed, and thus is a memory leak.
Luckily, the only user of the ring buffer so far is ftrace. And ftrace
will not free its ring buffer after it allocates it. There is no current
possible memory leak. But once there are other users, or if ftrace
dynamically creates and frees its ring buffer, then this would be a
memory leak.
This patch fixes the leak for future cases.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
This patch overloads RINGBUF_TYPE_PADDING to provide a way to discard events from the ring buffer, for the event-filtering mechanism introduced in a subsequent patch. I did the initial version but thanks to Steven Rostedt for adding the parts that actually made it work. ;-) Signed-off-by: Tom Zanussi <tzanussi@gmail.com> Acked-by: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
Impact: remove a section warning CONFIG_DEBUG_SECTION_MISMATCH raises the following warning on -tip: WARNING: kernel/trace/built-in.o(.text+0x5bc5): Section mismatch in reference from the function ring_buffer_alloc() to the function .cpuinit.text:rb_cpu_notify() The function ring_buffer_alloc() references the function __cpuinit rb_cpu_notify(). This is actually harmless. The code in the ring buffer don't build rb_cpu_notify and other cpu hotplug stuffs when !CONFIG_HOTPLUG_CPU so we have no risk to reference freed memory here (it would even be harmless if we unconditionally build it because register_cpu_notifier would do nothing when !CONFIG_HOTPLUG_CPU. But since ring_buffer_alloc() can be called everytime, we don't want it to be annotated with __cpuinit so we drop the __cpuinit from rb_cpu_notify. This is not a waste of memory because it is only defined and used on CONFIG_HOTPLUG_CPU. Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Steven Rostedt <rostedt@goodmis.org> LKML-Reference: <1237606416-22268-1-git-send-email-fweisbec@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
Impact: fix warning with irqsoff tracer The ring buffer allocates its buffers on pre-smp time (early_initcall). It means that, at first, only the boot cpu buffer is allocated and the ring-buffer cpumask only has the boot cpu set (cpu_online_mask). Later, the secondary cpu will show up and the ring-buffer will be notified about this event: the appropriate buffer will be allocated and the cpumask will be updated. Unfortunately, if !CONFIG_CPU_HOTPLUG, the ring-buffer will not be notified about the secondary cpus, meaning that the cpumask will have only the cpu boot set, and only one cpu buffer allocated. We fix that by using cpu_possible_mask if !CONFIG_CPU_HOTPLUG. This patch fixes the following warning with irqsoff tracer running: [ 169.317794] WARNING: at kernel/trace/trace.c:466 update_max_tr_single+0xcc/0xf3() [ 169.318002] Hardware name: AMILO Li 2727 [ 169.318002] Modules linked in: [ 169.318002] Pid: 5624, comm: bash Not tainted 2.6.29-rc8-tip-02636-g6aafa6c #11 [ 169.318002] Call Trace: [ 169.318002] [<ffffffff81036182>] warn_slowpath+0xea/0x13d [ 169.318002] [<ffffffff8100b9d6>] ? ftrace_call+0x5/0x2b [ 169.318002] [<ffffffff8100b9d6>] ? ftrace_call+0x5/0x2b [ 169.318002] [<ffffffff8100b9d1>] ? ftrace_call+0x0/0x2b [ 169.318002] [<ffffffff8101ef10>] ? ftrace_modify_code+0xa9/0x108 [ 169.318002] [<ffffffff8106e27f>] ? trace_hardirqs_off+0x25/0x27 [ 169.318002] [<ffffffff8149afe7>] ? _spin_unlock_irqrestore+0x1f/0x2d [ 169.318002] [<ffffffff81064f52>] ? ring_buffer_reset_cpu+0xf6/0xfb [ 169.318002] [<ffffffff8106637c>] ? ring_buffer_reset+0x36/0x48 [ 169.318002] [<ffffffff8106aeda>] update_max_tr_single+0xcc/0xf3 [ 169.318002] [<ffffffff8100bc17>] ? sysret_check+0x22/0x5d [ 169.318002] [<ffffffff8106e3ea>] stop_critical_timing+0x142/0x204 [ 169.318002] [<ffffffff8106e4cf>] trace_hardirqs_on_caller+0x23/0x25 [ 169.318002] [<ffffffff8149ac28>] trace_hardirqs_on_thunk+0x3a/0x3c [ 169.318002] [<ffffffff8100bc17>] ? sysret_check+0x22/0x5d [ 169.318002] ---[ end trace db76cbf775a750cf ]--- Because this tracer may try to swap two cpu ring buffers for an unregistered cpu on the ring buffer. This patch might also fix a fair loss of traces due to unallocated buffers for secondary cpus. Reported-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Acked-b: Steven Rostedt <rostedt@goodmis.org> LKML-Reference: <1237470453-5427-1-git-send-email-fweisbec@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
This patch adds a new function called ring_buffer_set_clock that allows a tracer to assign its own clock source to the buffer. Signed-off-by: Steven Rostedt <srostedt@redhat.com>
In a private email conversation I explained how the ring buffer page worked by using silly ASCII art. Ingo suggested that I add that to the comments of the code. Here it is. Requested-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Impact: speed up and remove possible races The get_online_cpus was added to the ring buffer because the original design would free the ring buffer on a CPU that was being taken off line. The final design kept the ring buffer around even when the CPU was taken off line. This is to allow a user to still read the information on that ring buffer. Most of the get_online_cpus are no longer needed since the ring buffer will not disappear from the use cases. Reported-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Steven Rostedt <srostedt@redhat.com>