linux-toradex.git/include/linux/jump_label.h, branch v3.2-rc5

jump-label: initialize jump-label subsystem much earlier

2011-10-25T18:55:15+00:00

Initialize jump_labels much, much earlier, so they're available for use
during system setup.

Signed-off-by: Jeremy Fitzhardinge 
Acked-by: Peter Zijlstra

jump_label: add arch_jump_label_transform_static() to optimise non-live code updates

2011-10-25T18:54:31+00:00

When updating a newly loaded module, the code is definitely not yet
executing on any processor, so it can be updated with no need for any
heavyweight synchronization.

This patch adds arch_jump_label_static() which is implemented as
arch_jump_label_transform() by default, but architectures can override
it if it avoids, say, a call to stop_machine().

Signed-off-by: Jeremy Fitzhardinge 
Acked-by: Jason Baron 
Acked-by: Peter Zijlstra

jump_label: if a key has already been initialized, don't nop it out

2011-10-25T18:54:15+00:00

If a key has been enabled before jump_label_init() is called, don't
nop it out.

This removes arch_jump_label_text_poke_early() (which can only nop
out a site) and uses arch_jump_label_transform() instead.

Signed-off-by: Jeremy Fitzhardinge 
Acked-by: Jason Baron 
Acked-by: Peter Zijlstra

jump_label: use proper atomic_t initializer

2011-10-25T18:53:17+00:00

ATOMIC_INIT() is the proper thing to use.

Signed-off-by: Jeremy Fitzhardinge 
Acked-by: Jason Baron 
Acked-by: Peter Zijlstra

atomic: use

2011-07-26T23:49:47+00:00

This allows us to move duplicated code in 
(atomic_inc_not_zero() for now) to 

Signed-off-by: Arun Sharma 
Reviewed-by: Eric Dumazet 
Cc: Ingo Molnar 
Cc: David Miller 
Cc: Eric Dumazet 
Acked-by: Mike Frysinger 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

jump label: Introduce static_branch() interface

2011-04-04T16:48:08+00:00

Introduce:

static __always_inline bool static_branch(struct jump_label_key *key);

instead of the old JUMP_LABEL(key, label) macro.

In this way, jump labels become really easy to use:

Define:

        struct jump_label_key jump_key;

Can be used as:

        if (static_branch(&jump_key))
                do unlikely code

enable/disale via:

        jump_label_inc(&jump_key);
        jump_label_dec(&jump_key);

that's it!

For the jump labels disabled case, the static_branch() becomes an
atomic_read(), and jump_label_inc()/dec() are simply atomic_inc(),
atomic_dec() operations. We show testing results for this change below.

Thanks to H. Peter Anvin for suggesting the 'static_branch()' construct.

Since we now require a 'struct jump_label_key *key', we can store a pointer into
the jump table addresses. In this way, we can enable/disable jump labels, in
basically constant time. This change allows us to completely remove the previous
hashtable scheme. Thanks to Peter Zijlstra for this re-write.

Testing:

I ran a series of 'tbench 20' runs 5 times (with reboots) for 3
configurations, where tracepoints were disabled.

jump label configured in
avg: 815.6

jump label *not* configured in (using atomic reads)
avg: 800.1

jump label *not* configured in (regular reads)
avg: 803.4

Signed-off-by: Peter Zijlstra 
LKML-Reference: <20110316212947.GA8792@redhat.com>
Signed-off-by: Jason Baron 
Suggested-by: H. Peter Anvin 
Tested-by: David Daney 
Acked-by: Ralf Baechle 
Acked-by: David S. Miller 
Acked-by: Mathieu Desnoyers 
Signed-off-by: Steven Rostedt

jump label: Add work around to i386 gcc asm goto bug

2010-10-29T18:45:29+00:00

On i386 (not x86_64) early implementations of gcc would have a bug
with asm goto causing it to produce code like the following:

(This was noticed by Peter Zijlstra)

   56 pushl 0
   67 nopl         jmp 0x6f
      popl
      jmp 0x8c

   6f              mov
                   test
                   je 0x8c

   8c mov
      call *(%esp)

The jump added in the asm goto skipped over the popl that matched
the pushl 0, which lead up to a quick crash of the system when
the jump was enabled. The nopl is defined in the asm goto () statement
and when tracepoints are enabled, the nop changes to a jump to the label
that was specified by the asm goto. asm goto is suppose to tell gcc that
the code in the asm might jump to an external label. Here gcc obviously
fails to make that work.

The bug report for gcc is here:

  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46226

The bug only appears on x86 when not compiled with
-maccumulate-outgoing-args. This option is always set on x86_64 and it
is also the work around for a function graph tracer i386 bug.
(See commit: 746357d6a526d6da9d89a2ec645b28406e959c2e)
This explains why the bug only showed up on i386 when function graph
tracer was not enabled.

This patch now adds a CONFIG_JUMP_LABEL option that is default
off instead of using jump labels by default. When jump labels are
enabled, the -maccumulate-outgoing-args will be used (causing a
slightly larger kernel image on i386). This option will exist
until we have a way to detect if the gcc compiler in use is safe
to use on all configurations without the work around.

Note, there exists such a test, but for now we will keep the enabling
of jump label as a manual option.

Archs that know the compiler is safe with asm goto, may choose to
select JUMP_LABEL and enable it by default.

Reported-by: Ingo Molnar 
Cause-discovered-by: Peter Zijlstra 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Jason Baron 
Cc: H. Peter Anvin 
Cc: David Daney 
Cc: Mathieu Desnoyers 
Cc: Masami Hiramatsu 
Cc: David Miller 
Cc: Richard Henderson 
LKML-Reference: <1288028746.3673.11.camel@laptop>
Signed-off-by: Steven Rostedt

jump label: Fix deadlock b/w jump_label_mutex vs. text_mutex

2010-10-28T13:17:40+00:00

register_kprobe() downs the 'text_mutex' and then calls
jump_label_text_reserved(), which downs the 'jump_label_mutex'.
However, the jump label code takes those mutexes in the reverse
order.

Fix by requiring the caller of jump_label_text_reserved() to do
the jump label locking via the newly added: jump_label_lock(),
jump_label_unlock(). Currently, kprobes is the only user
of jump_label_text_reserved().

Reported-by: Ingo Molnar 
Acked-by: Masami Hiramatsu 
Signed-off-by: Jason Baron 
LKML-Reference: <759032c48d5e30c27f0bba003d09bffa8e9f28bb.1285965957.git.jbaron@redhat.com>
Signed-off-by: Steven Rostedt

jump_label: Add COND_STMT(), reducer wrappery

2010-10-18T17:59:01+00:00

The use of the JUMP_LABEL() construct ends up creating endless silly
wrappers, create a higher level construct to reduce this clutter.

Signed-off-by: Peter Zijlstra 
Cc: Jason Baron 
Cc: Steven Rostedt 
Cc: Arnaldo Carvalho de Melo 
Cc: Frederic Weisbecker 
Cc: Paul Mackerras 
LKML-Reference: 
Signed-off-by: Ingo Molnar

jump_label: Use more consistent naming

2010-10-18T17:58:56+00:00

Now that there's still only a few users around, rename things to make
them more consistent.

Signed-off-by: Peter Zijlstra 
LKML-Reference: <20101014203625.448565169@chello.nl>
Signed-off-by: Ingo Molnar