linux-toradex.git/include/linux/sched.h, branch v2.6.32.22

sched: Pre-compute cpumask_weight(sched_domain_span(sd))

2010-09-20T20:18:11+00:00

commit 669c55e9f99b90e46eaa0f98a67ec53d46dc969a upstream

Dave reported that his large SPARC machines spend lots of time in
hweight64(), try and optimize some of those needless cpumask_weight()
invocations (esp. with the large offstack cpumasks these are very
expensive indeed).

Reported-by: David Miller 
Signed-off-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Fix TASK_WAKING vs fork deadlock

2010-09-20T20:18:09+00:00

commit 0017d735092844118bef006696a750a0e4ef6ebd upstream

Oleg noticed a few races with the TASK_WAKING usage on fork.

 - since TASK_WAKING is basically a spinlock, it should be IRQ safe
 - since we set TASK_WAKING (*) without holding rq->lock it could
   be there still is a rq->lock holder, thereby not actually
   providing full serialization.

(*) in fact we clear PF_STARTING, which in effect enables TASK_WAKING.

Cure the second issue by not setting TASK_WAKING in sched_fork(), but
only temporarily in wake_up_new_task() while calling select_task_rq().

Cure the first by holding rq->lock around the select_task_rq() call,
this will disable IRQs, this however requires that we push down the
rq->lock release into select_task_rq_fair()'s cgroup stuff.

Because select_task_rq_fair() still needs to drop the rq->lock we
cannot fully get rid of TASK_WAKING.

Reported-by: Oleg Nesterov 
Signed-off-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: _cpu_down(): Don't play with current->cpus_allowed

2010-09-20T20:18:08+00:00

commit 6a1bdc1b577ebcb65f6603c57f8347309bc4ab13 upstream

_cpu_down() changes the current task's affinity and then recovers it at
the end. The problems are well known: we can't restore old_allowed if it
was bound to the now-dead-cpu, and we can race with the userspace which
can change cpu-affinity during unplug.

_cpu_down() should not play with current->cpus_allowed at all. Instead,
take_cpu_down() can migrate the caller of _cpu_down() after __cpu_disable()
removes the dying cpu from cpu_online_mask.

Signed-off-by: Oleg Nesterov 
Acked-by: Rafael J. Wysocki 
Signed-off-by: Peter Zijlstra 
LKML-Reference: <20100315091023.GA9148@redhat.com>
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Extend enqueue_task to allow head queueing

2010-09-20T20:18:03+00:00

commit ea87bb7853168434f4a82426dd1ea8421f9e604d upstream

The ability of enqueueing a task to the head of a SCHED_FIFO priority
list is required to fix some violations of POSIX scheduling policy.

Extend the related functions with a "head" argument.

Signed-off-by: Thomas Gleixner 
Acked-by: Peter Zijlstra 
Tested-by: Carsten Emde 
Tested-by: Mathias Weber 
LKML-Reference: <20100120171629.734886007@linutronix.de>
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Remove the cfs_rq dependency from set_task_cpu()

2010-09-20T20:18:02+00:00

commit 88ec22d3edb72b261f8628226cd543589a6d5e1b upstream

In order to remove the cfs_rq dependency from set_task_cpu() we
need to ensure the task is cfs_rq invariant for all callsites.

The simple approach is to substract cfs_rq->min_vruntime from
se->vruntime on dequeue, and add cfs_rq->min_vruntime on
enqueue.

However, this has the downside of breaking FAIR_SLEEPERS since
we loose the old vruntime as we only maintain the relative
position.

To solve this, we observe that we only migrate runnable tasks,
we do this using deactivate_task(.sleep=0) and
activate_task(.wakeup=0), therefore we can restrain the
min_vruntime invariance to that state.

The only other case is wakeup balancing, since we want to
maintain the old vruntime we cannot make it relative on dequeue,
but since we don't migrate inactive tasks, we can do so right
before we activate it again.

This is where we need the new pre-wakeup hook, we need to call
this while still holding the old rq->lock. We could fold it into
->select_task_rq(), but since that has multiple callsites and
would obfuscate the locking requirements, that seems like a
fudge.

This leaves the fork() case, simply make sure that ->task_fork()
leaves the ->vruntime in a relative state.

This covers all cases where set_task_cpu() gets called, and
ensures it sees a relative vruntime.

Signed-off-by: Peter Zijlstra 
Cc: Mike Galbraith 
LKML-Reference: <20091216170518.191697025@chello.nl>
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Add pre and post wakeup hooks

2010-09-20T20:18:01+00:00

commit efbbd05a595343a413964ad85a2ad359b7b7efbd upstream

As will be apparent in the next patch, we need a pre wakeup hook
for sched_fair task migration, hence rename the post wakeup hook
and one pre wakeup.

Signed-off-by: Peter Zijlstra 
Cc: Mike Galbraith 
LKML-Reference: <20091216170518.114746117@chello.nl>
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Remove forced2_migrations stats

2010-09-20T20:17:59+00:00

commit b9889ed1ddeca5a3f3569c8de7354e9e97d803ae upstream

This build warning:

 kernel/sched.c: In function 'set_task_cpu':
 kernel/sched.c:2070: warning: unused variable 'old_rq'

Made me realize that the forced2_migrations stat looks pretty
pointless (and a misnomer) - remove it.

Cc: Peter Zijlstra 
Cc: Mike Galbraith 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Sanitize fork() handling

2010-09-20T20:17:59+00:00

commit cd29fe6f2637cc2ccbda5ac65f5332d6bf5fa3c6 upstream

Currently we try to do task placement in wake_up_new_task() after we do
the load-balance pass in sched_fork(). This yields complicated semantics
in that we have to deal with tasks on different RQs and the
set_task_cpu() calls in copy_process() and sched_fork()

Rename ->task_new() to ->task_fork() and call it from sched_fork()
before the balancing, this gives the policy a clear point to place the
task.

Signed-off-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Remove unused cpu_nr_migrations()

2010-09-20T20:17:58+00:00

commit 9824a2b728b63e7ff586b9fd9293c819be79f0f3 upstream

cpu_nr_migrations() is not used, remove it.

Signed-off-by: Hiroshi Shimamoto 
Cc: Peter Zijlstra 
LKML-Reference: <4AF12A66.6020609@ct.jp.nec.com>
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman

sched: Protect sched_rr_get_param() access to task->sched_class

2010-09-20T20:17:57+00:00

commit dba091b9e3522b9d32fc9975e48d3b69633b45f0 upstream

sched_rr_get_param calls
task->sched_class->get_rr_interval(task) without protection
against a concurrent sched_setscheduler() call which modifies
task->sched_class.

Serialize the access with task_rq_lock(task) and hand the rq
pointer into get_rr_interval() as it's needed at least in the
sched_fair implementation.

Signed-off-by: Thomas Gleixner 
Acked-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Mike Galbraith 
Signed-off-by: Greg Kroah-Hartman