linux-toradex.git/include/linux/rwsem.h, branch v4.14-rc5

locking/rwsem-xadd: Add killable versions of rwsem_down_read_failed()

2017-08-10T10:28:55+00:00

Rename rwsem_down_read_failed() in __rwsem_down_read_failed_common()
and teach it to abort waiting in case of pending signals and killable
state argument passed.

Note, that we shouldn't wake anybody up in EINTR path, as:

We check for (waiter.task) under spinlock before we go to out_nolock
path. Current task wasn't able to be woken up, so there are
a writer, owning the sem, or a writer, which is the first waiter.
In the both cases we shouldn't wake anybody. If there is a writer,
owning the sem, and we were the only waiter, remove RWSEM_WAITING_BIAS,
as there are no waiters anymore.

Signed-off-by: Kirill Tkhai 
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: arnd@arndb.de
Cc: avagin@virtuozzo.com
Cc: davem@davemloft.net
Cc: fenghua.yu@intel.com
Cc: gorcunov@virtuozzo.com
Cc: heiko.carstens@de.ibm.com
Cc: hpa@zytor.com
Cc: ink@jurassic.park.msu.ru
Cc: mattst88@gmail.com
Cc: rth@twiddle.net
Cc: schwidefsky@de.ibm.com
Cc: tony.luck@intel.com
Link: http://lkml.kernel.org/r/149789534632.9059.2901382369609922565.stgit@localhost.localdomain
Signed-off-by: Ingo Molnar

locking/rwsem: Convert sem->count to 'atomic_long_t'

2016-06-08T13:16:42+00:00

Convert the rwsem count variable to an atomic_long_t since we use it
as an atomic variable. This also allows us to remove the
rwsem_atomic_{add,update}() "abstraction" which would now be an unnecesary
level of indirection. In follow up patches, we also remove the
rwsem_atomic_{add,update}() definitions across the various architectures.

Suggested-by: Peter Zijlstra 
Signed-off-by: Jason Low 
[ Build warning fixes on various architectures. ]
Signed-off-by: Peter Zijlstra (Intel) 
Cc: Andrew Morton 
Cc: Davidlohr Bueso 
Cc: Fenghua Yu 
Cc: Heiko Carstens 
Cc: Jason Low 
Cc: Linus Torvalds 
Cc: Martin Schwidefsky 
Cc: Paul E. McKenney 
Cc: Peter Hurley 
Cc: Terry Rudd 
Cc: Thomas Gleixner 
Cc: Tim Chen 
Cc: Tony Luck 
Cc: Waiman Long 
Link: http://lkml.kernel.org/r/1465017963-4839-2-git-send-email-jason.low2@hpe.com
Signed-off-by: Ingo Molnar

add down_write_killable_nested()

2016-05-26T04:04:58+00:00

Signed-off-by: Al Viro

locking/rwsem: Provide down_write_killable()

2016-04-22T06:58:33+00:00

Now that all the architectures implement the necessary glue code
we can introduce down_write_killable(). The only difference wrt. regular
down_write() is that the slow path waits in TASK_KILLABLE state and the
interruption by the fatal signal is reported as -EINTR to the caller.

Signed-off-by: Michal Hocko 
Cc: Andrew Morton 
Cc: Chris Zankel 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: Max Filippov 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Signed-off-by: Davidlohr Bueso 
Cc: Signed-off-by: Jason Low 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: linux-alpha@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: sparclinux@vger.kernel.org
Link: http://lkml.kernel.org/r/1460041951-22347-12-git-send-email-mhocko@kernel.org
Signed-off-by: Ingo Molnar

locking/rwsem: Introduce basis for down_write_killable()

2016-04-13T08:42:20+00:00

Introduce a generic implementation necessary for down_write_killable().

This is a trivial extension of the already existing down_write() call
which can be interrupted by SIGKILL.  This patch doesn't provide
down_write_killable() yet because arches have to provide the necessary
pieces before.

rwsem_down_write_failed() which is a generic slow path for the
write lock is extended to take a task state and renamed to
__rwsem_down_write_failed_common(). The return value is either a valid
semaphore pointer or ERR_PTR(-EINTR).

rwsem_down_write_failed_killable() is exported as a new way to wait for
the lock and be killable.

For rwsem-spinlock implementation the current __down_write() it updated
in a similar way as __rwsem_down_write_failed_common() except it doesn't
need new exports just visible __down_write_killable().

Architectures which are not using the generic rwsem implementation are
supposed to provide their __down_write_killable() implementation and
use rwsem_down_write_failed_killable() for the slow path.

Signed-off-by: Michal Hocko 
Cc: Andrew Morton 
Cc: Chris Zankel 
Cc: David S. Miller 
Cc: Linus Torvalds 
Cc: Max Filippov 
Cc: Paul E. McKenney 
Cc: Peter Zijlstra 
Cc: Signed-off-by: Davidlohr Bueso 
Cc: Signed-off-by: Jason Low 
Cc: Thomas Gleixner 
Cc: Tony Luck 
Cc: linux-alpha@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-s390@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: sparclinux@vger.kernel.org
Link: http://lkml.kernel.org/r/1460041951-22347-7-git-send-email-mhocko@kernel.org
Signed-off-by: Ingo Molnar

locking/Documentation: Move locking related docs into Documentation/locking/

2014-08-13T08:32:03+00:00

Specifically:
  Documentation/locking/lockdep-design.txt
  Documentation/locking/lockstat.txt
  Documentation/locking/mutex-design.txt
  Documentation/locking/rt-mutex-design.txt
  Documentation/locking/rt-mutex.txt
  Documentation/locking/spinlocks.txt
  Documentation/locking/ww-mutex-design.txt

Signed-off-by: Davidlohr Bueso 
Acked-by: Randy Dunlap 
Signed-off-by: Peter Zijlstra 
Cc: jason.low2@hp.com
Cc: aswin@hp.com
Cc: Alexei Starovoitov 
Cc: Al Viro 
Cc: Andrew Morton 
Cc: Chris Mason 
Cc: Dan Streetman 
Cc: David Airlie 
Cc: Davidlohr Bueso 
Cc: David S. Miller 
Cc: Greg Kroah-Hartman 
Cc: Heiko Carstens 
Cc: Jason Low 
Cc: Josef Bacik 
Cc: Kees Cook 
Cc: Linus Torvalds 
Cc: Lubomir Rintel 
Cc: Masanari Iida 
Cc: Paul E. McKenney 
Cc: Randy Dunlap 
Cc: Tim Chen 
Cc: Vineet Gupta 
Cc: fengguang.wu@intel.com
Link: http://lkml.kernel.org/r/1406752916-3341-6-git-send-email-davidlohr@hp.com
Signed-off-by: Ingo Molnar

locking/rwsem: Add CONFIG_RWSEM_SPIN_ON_OWNER

2014-07-16T12:57:13+00:00

Just like with mutexes (CONFIG_MUTEX_SPIN_ON_OWNER),
encapsulate the dependencies for rwsem optimistic spinning.
No logical changes here as it continues to depend on both
SMP and the XADD algorithm variant.

Signed-off-by: Davidlohr Bueso 
Acked-by: Jason Low 
[ Also make it depend on ARCH_SUPPORTS_ATOMIC_RMW. ]
Signed-off-by: Peter Zijlstra 
Link: http://lkml.kernel.org/r/1405112406-13052-2-git-send-email-davidlohr@hp.com
Cc: aswin@hp.com
Cc: Chris Mason 
Cc: Davidlohr Bueso 
Cc: Josef Bacik 
Cc: Linus Torvalds 
Cc: Waiman Long 
Signed-off-by: Ingo Molnar 

Signed-off-by: Ingo Molnar

locking/rwsem: Reduce the size of struct rw_semaphore

2014-07-16T12:57:03+00:00

Recent optimistic spinning additions to rwsem provide significant performance
benefits on many workloads on large machines. The cost of it was increasing
the size of the rwsem structure by up to 128 bits.

However, now that the previous patches in this series bring the overhead of
struct optimistic_spin_queue to 32 bits, this patch reorders some fields in
struct rw_semaphore such that we can reduce the overhead of the rwsem structure
by 64 bits (on 64 bit systems).

The extra overhead required for rwsem optimistic spinning would now be up
to 8 additional bytes instead of up to 16 bytes. Additionally, the size of
rwsem would now be more in line with mutexes.

Signed-off-by: Jason Low 
Signed-off-by: Peter Zijlstra 
Cc: Scott Norton 
Cc: "Paul E. McKenney" 
Cc: Dave Chinner 
Cc: Waiman Long 
Cc: Davidlohr Bueso 
Cc: Rik van Riel 
Cc: Andrew Morton 
Cc: "H. Peter Anvin" 
Cc: Steven Rostedt 
Cc: Tim Chen 
Cc: Konrad Rzeszutek Wilk 
Cc: Aswin Chandramouleeswaran 
Cc: Linus Torvalds 
Cc: Chris Mason 
Cc: Josef Bacik 
Link: http://lkml.kernel.org/r/1405358872-3732-6-git-send-email-jason.low2@hp.com
Signed-off-by: Ingo Molnar

locking/spinlocks/mcs: Introduce and use init macro and function for osq locks

2014-07-16T11:28:05+00:00

Currently, we initialize the osq lock by directly setting the lock's values. It
would be preferable if we use an init macro to do the initialization like we do
with other locks.

This patch introduces and uses a macro and function for initializing the osq lock.

Signed-off-by: Jason Low 
Signed-off-by: Peter Zijlstra 
Cc: Scott Norton 
Cc: "Paul E. McKenney" 
Cc: Dave Chinner 
Cc: Waiman Long 
Cc: Davidlohr Bueso 
Cc: Rik van Riel 
Cc: Andrew Morton 
Cc: "H. Peter Anvin" 
Cc: Steven Rostedt 
Cc: Tim Chen 
Cc: Konrad Rzeszutek Wilk 
Cc: Aswin Chandramouleeswaran 
Cc: Linus Torvalds 
Cc: Chris Mason 
Cc: Josef Bacik 
Link: http://lkml.kernel.org/r/1405358872-3732-4-git-send-email-jason.low2@hp.com
Signed-off-by: Ingo Molnar

locking/spinlocks/mcs: Convert osq lock to atomic_t to reduce overhead

2014-07-16T11:28:04+00:00

The cancellable MCS spinlock is currently used to queue threads that are
doing optimistic spinning. It uses per-cpu nodes, where a thread obtaining
the lock would access and queue the local node corresponding to the CPU that
it's running on. Currently, the cancellable MCS lock is implemented by using
pointers to these nodes.

In this patch, instead of operating on pointers to the per-cpu nodes, we
store the CPU numbers in which the per-cpu nodes correspond to in atomic_t.
A similar concept is used with the qspinlock.

By operating on the CPU # of the nodes using atomic_t instead of pointers
to those nodes, this can reduce the overhead of the cancellable MCS spinlock
by 32 bits (on 64 bit systems).

Signed-off-by: Jason Low 
Signed-off-by: Peter Zijlstra 
Cc: Scott Norton 
Cc: "Paul E. McKenney" 
Cc: Dave Chinner 
Cc: Waiman Long 
Cc: Davidlohr Bueso 
Cc: Rik van Riel 
Cc: Andrew Morton 
Cc: "H. Peter Anvin" 
Cc: Steven Rostedt 
Cc: Tim Chen 
Cc: Konrad Rzeszutek Wilk 
Cc: Aswin Chandramouleeswaran 
Cc: Linus Torvalds 
Cc: Chris Mason 
Cc: Heiko Carstens 
Cc: Josef Bacik 
Link: http://lkml.kernel.org/r/1405358872-3732-3-git-send-email-jason.low2@hp.com
Signed-off-by: Ingo Molnar