linux-toradex.git/include/linux/rcupdate.h, branch v6.0-rc1

Merge branch 'ctxt.2022.07.05a' into HEAD

2022-07-22T00:46:18+00:00

ctxt.2022.07.05a: Linux-kernel memory model development branch.

Merge branches 'doc.2022.06.21a', 'fixes.2022.07.19a', 'nocb.2022.07.19a', 'poll.2022.07.21a', 'rcu-tasks.2022.06.21a' and 'torture.2022.06.21a' into HEAD

2022-07-22T00:43:16+00:00

doc.2022.06.21a: Documentation updates.
fixes.2022.07.19a: Miscellaneous fixes.
nocb.2022.07.19a: Callback-offload updates.
poll.2022.07.21a: Polled grace-period updates.
rcu-tasks.2022.06.21a: Tasks RCU updates.
torture.2022.06.21a: Torture-test updates.

rcu/context-tracking: Remove unused and/or unecessary middle functions

2022-07-05T20:33:00+00:00

Some eqs functions are now only used internally by context tracking, so
their public declarations can be removed.

Also middle functions such as rcu_user_*() and rcu_idle_*()
which now directly call to rcu_eqs_enter() and rcu_eqs_exit() can be
wiped out as well.

Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Neeraj Upadhyay 
Cc: Uladzislau Rezki 
Cc: Joel Fernandes 
Cc: Boqun Feng 
Cc: Nicolas Saenz Julienne 
Cc: Marcelo Tosatti 
Cc: Xiongfeng Wang 
Cc: Yu Liao 
Cc: Phil Auld 
Cc: Paul Gortmaker
Cc: Alex Belits 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Nicolas Saenz Julienne 
Tested-by: Nicolas Saenz Julienne

rcu/context-tracking: Move deferred nocb resched to context tracking

2022-07-05T20:32:59+00:00

To prepare for migrating the RCU eqs accounting code to context tracking,
split the last-resort deferred nocb resched from rcu_user_enter() and
move it into a separate call from context tracking.

Acked-by: Paul E. McKenney 
Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Neeraj Upadhyay 
Cc: Uladzislau Rezki 
Cc: Joel Fernandes 
Cc: Boqun Feng 
Cc: Nicolas Saenz Julienne 
Cc: Marcelo Tosatti 
Cc: Xiongfeng Wang 
Cc: Yu Liao 
Cc: Phil Auld 
Cc: Paul Gortmaker
Cc: Alex Belits 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Nicolas Saenz Julienne 
Tested-by: Nicolas Saenz Julienne

context_tracking: Take IRQ eqs entrypoints over RCU

2022-07-05T20:32:59+00:00

The RCU dynticks counter is going to be merged into the context tracking
subsystem. Prepare with moving the IRQ extended quiescent states
entrypoints to context tracking. For now those are dumb redirection to
existing RCU calls.

[ paulmck: Apply Stephen Rothwell feedback from -next. ]
[ paulmck: Apply Nathan Chancellor feedback. ]

Acked-by: Paul E. McKenney 
Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Neeraj Upadhyay 
Cc: Uladzislau Rezki 
Cc: Joel Fernandes 
Cc: Boqun Feng 
Cc: Nicolas Saenz Julienne 
Cc: Marcelo Tosatti 
Cc: Xiongfeng Wang 
Cc: Yu Liao 
Cc: Phil Auld 
Cc: Paul Gortmaker
Cc: Alex Belits 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Nicolas Saenz Julienne 
Tested-by: Nicolas Saenz Julienne

context_tracking: Take idle eqs entrypoints over RCU

2022-07-05T20:32:16+00:00

The RCU dynticks counter is going to be merged into the context tracking
subsystem. Start with moving the idle extended quiescent states
entrypoints to context tracking. For now those are dumb redirections to
existing RCU calls.

[ paulmck: Apply kernel test robot feedback. ]

Signed-off-by: Frederic Weisbecker 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: Neeraj Upadhyay 
Cc: Uladzislau Rezki 
Cc: Joel Fernandes 
Cc: Boqun Feng 
Cc: Nicolas Saenz Julienne 
Cc: Marcelo Tosatti 
Cc: Xiongfeng Wang 
Cc: Yu Liao 
Cc: Phil Auld 
Cc: Paul Gortmaker
Cc: Alex Belits 
Signed-off-by: Paul E. McKenney 
Reviewed-by: Nicolas Saenz Julienne 
Tested-by: Nicolas Saenz Julienne

rcu-tasks: Track blocked RCU Tasks Trace readers

2022-06-20T16:22:28+00:00

This commit places any task that has ever blocked within its current
RCU Tasks Trace read-side critical section on a per-CPU list within the
rcu_tasks_percpu structure.  Tasks are removed from this list when they
exit by the exit_tasks_rcu_finish_trace() function.  The purpose of this
commit is to provide the information needed to eliminate the current
scan of the full task list.

This commit offsets the INT_MIN value for ->trc_reader_nesting with the
new nesting level in order to avoid queueing tasks that are exiting
their read-side critical sections.

[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Apply feedback from syzbot+9bb26e7c5e8e4fa7e641@syzkaller.appspotmail.com ]

Signed-off-by: Paul E. McKenney 
Tested-by: syzbot 
Tested-by: "Zhang, Qiang1" 
Cc: Peter Zijlstra 
Cc: Neeraj Upadhyay 
Cc: Eric Dumazet 
Cc: Alexei Starovoitov 
Cc: Andrii Nakryiko 
Cc: Martin KaFai Lau 
Cc: KP Singh

rcu-tasks: Merge state into .b.need_qs and atomically update

2022-06-20T16:22:27+00:00

This commit gets rid of the task_struct structure's ->trc_reader_checked
field, making it instead be a bit within the task_struct structure's
existing ->trc_reader_special.b.need_qs field.  This commit also
atomically loads, stores, and checks the resulting combination of the
reader-checked and need-quiescent state flags.  This will in turn allow
significant simplification of the rcu_tasks_trace_postgp() function
as well as elimination of the trc_n_readers_need_end counter in later
commits.  These changes will in turn simplify later elimination of the
RCU Tasks Trace scan of the task list, which will make RCU Tasks Trace
grace periods less CPU-intensive.

[ paulmck: Apply kernel test robot feedback. ]

Signed-off-by: Paul E. McKenney 
Cc: Neeraj Upadhyay 
Cc: Eric Dumazet 
Cc: Alexei Starovoitov 
Cc: Andrii Nakryiko 
Cc: Martin KaFai Lau 
Cc: KP Singh

rcu: Provide a get_completed_synchronize_rcu() function

2022-06-20T16:21:51+00:00

It is currently up to the caller to handle stale return values from
get_state_synchronize_rcu().  If poll_state_synchronize_rcu() returned
true once, a grace period has elapsed, regardless of the fact that counter
wrap might cause some future poll_state_synchronize_rcu() invocation to
return false.  For example, the caller might store a separate flag that
indicates whether some previous call to poll_state_synchronize_rcu()
determined that the relevant grace period had already ended.

This approach works, but it requires extra storage and is easy to get
wrong.  This commit therefore introduces a get_completed_synchronize_rcu()
that returns a cookie that causes poll_state_synchronize_rcu() to always
return true.  This already-completed cookie can be stored in place of the
cookie that previously caused poll_state_synchronize_rcu() to return true.
It can also be used to flag a given structure as not having been exposed
to readers, and thus not requiring a grace period to elapse.

This commit is in preparation for polled expedited grace periods.

Link: https://lore.kernel.org/all/20220121142454.1994916-1-bfoster@redhat.com/
Link: https://docs.google.com/document/d/1RNKWW9jQyfjxw2E8dsXVTdvZYh0HnYeSHDKog9jhdN8/edit?usp=sharing
Cc: Brian Foster 
Cc: Dave Chinner 
Cc: Al Viro 
Cc: Ian Kent 
Signed-off-by: Paul E. McKenney

rcu-tasks: Make Tasks RCU account for userspace execution

2022-04-12T00:06:42+00:00

The main Tasks RCU quiescent state is voluntary context switch.  However,
userspace execution is also a valid quiescent state, and is a valuable one
for userspace applications that spin repeatedly executing light-weight
non-sleeping system calls.  Currently, such an application can delay a
Tasks RCU grace period for many tens of seconds.

This commit therefore enlists the aid of the scheduler-clock interrupt to
provide a Tasks RCU quiescent state when it interrupted a task executing
in userspace.

[ paulmck: Apply feedback from kernel test robot. ]

Cc: Martin KaFai Lau 
Cc: Neil Spring 
Signed-off-by: Paul E. McKenney