linux-toradex.git/lib/atomic64.c, branch v4.19-rc5

atomics/generic: Define atomic64_fetch_add_unless()

2018-06-21T12:25:21+00:00

As a step towards unifying the atomic/atomic64/atomic_long APIs, this
patch converts the generic implementation of atomic64_add_unless() into
a generic implementation of atomic64_fetch_add_unless().

A wrapper in  will build atomic_add_unless() atop of
this, provided it is given a preprocessor definition.

No functional change is intended as a result of this patch.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Cc: Arnd Bergmann 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-9-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

atomics: Make conditional ops return 'bool'

2018-06-21T12:22:33+00:00

Some of the atomics return a status value, which is a boolean value
describing whether the operation was performed. To make it clear that
this is a boolean value, let's update the common fallbacks to return
bool, fixing up the return values and comments likewise.

At the same time, let's simplify the description of the operations in
their respective comments.

The instrumented atomics and generic atomic64 implementation are updated
accordingly.

Note that atomic64_dec_if_positive() doesn't follow the usual test op
pattern, and returns the would-be decremented value. This is not
changed.

Signed-off-by: Mark Rutland 
Reviewed-by: Will Deacon 
Acked-by: Peter Zijlstra (Intel) 
Cc: Boqun Feng 
Cc: Linus Torvalds 
Cc: Michael Ellerman 
Cc: Thomas Gleixner 
Link: https://lore.kernel.org/lkml/20180621121321.4761-5-mark.rutland@arm.com
Signed-off-by: Ingo Molnar

locking/atomic: Implement atomic{,64,_long}_fetch_{add,sub,and,andnot,or,xor}{,_relaxed,_acquire,_release}()

2016-06-16T08:48:32+00:00

Now that all the architectures have implemented support for these new
atomic primitives add on the generic infrastructure to expose and use
it.

Signed-off-by: Peter Zijlstra (Intel) 
Cc: Andrew Morton 
Cc: Arnd Bergmann 
Cc: Boqun Feng 
Cc: Borislav Petkov 
Cc: Davidlohr Bueso 
Cc: Frederic Weisbecker 
Cc: Linus Torvalds 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Will Deacon 
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar

atomic: Provide atomic_{or,xor,and}

2015-07-27T12:06:24+00:00

Implement atomic logic ops -- atomic_{or,xor,and}.

These will replace the atomic_{set,clear}_mask functions that are
available on some archs.

Signed-off-by: Peter Zijlstra (Intel) 
Signed-off-by: Thomas Gleixner

locking,arch: Rewrite generic atomic support

2014-08-14T10:48:14+00:00

Rewrite generic atomic support to only require cmpxchg(), generate all
other primitives from that.

Furthermore reduce the endless repetition for all these primitives to
a few CPP macros. This way we get more for less lines.

Signed-off-by: Peter Zijlstra 
Link: http://lkml.kernel.org/r/20140508135852.940119622@infradead.org
Cc: Arnd Bergmann 
Cc: David Howells 
Cc: Paul E. McKenney 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: linux-arch@vger.kernel.org
Signed-off-by: Ingo Molnar

lib: atomic64: Initialize locks statically to fix early users

2012-12-20T21:50:16+00:00

The atomic64 library uses a handful of static spin locks to implement
atomic 64-bit operations on architectures without support for atomic
64-bit instructions.

Unfortunately, the spinlocks are initialized in a pure initcall and that
is too late for the vfs namespace code which wants to use atomic64
operations before the initcall is run.

This became a problem as of commit 8823c079ba71: "vfs: Add setns support
for the mount namespace".

This leads to BUG messages such as:

  BUG: spinlock bad magic on CPU#0, swapper/0/0
   lock: atomic64_lock+0x240/0x400, .magic: 00000000, .owner: /-1, .owner_cpu: 0
    do_raw_spin_lock+0x158/0x198
    _raw_spin_lock_irqsave+0x4c/0x58
    atomic64_add_return+0x30/0x5c
    alloc_mnt_ns.clone.14+0x44/0xac
    create_mnt_ns+0xc/0x54
    mnt_init+0x120/0x1d4
    vfs_caches_init+0xe0/0x10c
    start_kernel+0x29c/0x300

coming out early on during boot when spinlock debugging is enabled.

Fix this by initializing the spinlocks statically at compile time.

Reported-and-tested-by: Vaibhav Bedia 
Tested-by: Tony Lindgren 
Cc: Eric W. Biederman 
Signed-off-by: Stephen Boyd 
Signed-off-by: Linus Torvalds

lib: reduce the use of module.h wherever possible

2012-03-07T20:04:04+00:00

For files only using THIS_MODULE and/or EXPORT_SYMBOL, map
them onto including export.h -- or if the file isn't even
using those, then just delete the include.  Fix up any implicit
include dependencies that were being masked by module.h along
the way.

Signed-off-by: Paul Gortmaker

lib: atomic64: Change the type of local lock to raw_spinlock_t

2011-09-14T11:14:11+00:00

There are still some leftovers of commit f59ca058
[locking, lib/atomic64: Annotate atomic64_lock::lock as raw]

[ tglx: Seems I picked the wrong version of that patch :( ]

Signed-off-by: Yong Zhang 
Cc: Peter Zijlstra 
Cc: Shan Hai 
Cc: Stephen Rothwell 
Link: http://lkml.kernel.org/r/20110914074924.GA16096@zhy
Signed-off-by: Thomas Gleixner

locking, lib/atomic64: Annotate atomic64_lock::lock as raw

2011-09-13T09:12:22+00:00

The spinlock protected atomic64 operations must be irq safe as they
are used in hard interrupt context and cannot be preempted on -rt:

 NIP [c068b218] rt_spin_lock_slowlock+0x78/0x3a8
  LR [c068b1e0] rt_spin_lock_slowlock+0x40/0x3a8
 Call Trace:
  [eb459b90] [c068b1e0] rt_spin_lock_slowlock+0x40/0x3a8 (unreliable)
  [eb459c20] [c068bdb0] rt_spin_lock+0x40/0x98
  [eb459c40] [c03d2a14] atomic64_read+0x48/0x84
  [eb459c60] [c001aaf4] perf_event_interrupt+0xec/0x28c
  [eb459d10] [c0010138] performance_monitor_exception+0x7c/0x150
  [eb459d30] [c0014170] ret_from_except_full+0x0/0x4c

So annotate it.

In mainline this change documents the low level nature of
the lock - otherwise there's no functional difference. Lockdep
and Sparse checking will work as usual.

Signed-off-by: Shan Hai 
Reviewed-by: Yong Zhang 
Signed-off-by: Thomas Gleixner 
Signed-off-by: Ingo Molnar

atomic: use

2011-07-26T23:49:47+00:00

This allows us to move duplicated code in 
(atomic_inc_not_zero() for now) to 

Signed-off-by: Arun Sharma 
Reviewed-by: Eric Dumazet 
Cc: Ingo Molnar 
Cc: David Miller 
Cc: Eric Dumazet 
Acked-by: Mike Frysinger 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds