linux-toradex.git/kernel/sched.c, branch v2.6.34.11

ftrace: Fix memory leak with function graph and cpu hotplug

2011-06-26T16:47:01+00:00

commit 868baf07b1a259f5f3803c1dc2777b6c358f83cf upstream.

When the fuction graph tracer starts, it needs to make a special
stack for each task to save the real return values of the tasks.
All running tasks have this stack created, as well as any new
tasks.

On CPU hot plug, the new idle task will allocate a stack as well
when init_idle() is called. The problem is that cpu hotplug does
not create a new idle_task. Instead it uses the idle task that
existed when the cpu went down.

ftrace_graph_init_task() will add a new ret_stack to the task
that is given to it. Because a clone will make the task
have a stack of its parent it does not check if the task's
ret_stack is already NULL or not. When the CPU hotplug code
starts a CPU up again, it will allocate a new stack even
though one already existed for it.

The solution is to treat the idle_task specially. In fact, the
function_graph code already does, just not at init_idle().
Instead of using the ftrace_graph_init_task() for the idle task,
which that function expects the task to be a clone, have a
separate ftrace_graph_init_idle_task(). Also, we will create a
per_cpu ret_stack that is used by the idle task. When we call
ftrace_graph_init_idle_task() it will check if the idle task's
ret_stack is NULL, if it is, then it will assign it the per_cpu
ret_stack.

Reported-by: Benjamin Herrenschmidt 
Suggested-by: Peter Zijlstra 
Signed-off-by: Steven Rostedt 
Signed-off-by: Paul Gortmaker

sched: Fix wake_affine() vs RT tasks

2011-06-26T16:46:39+00:00

commit e51fd5e22e12b39f49b1bb60b37b300b17378a43 upstream.

Mike reports that since e9e9250b (sched: Scale down cpu_power due to RT
tasks), wake_affine() goes funny on RT tasks due to them still having a
!0 weight and wake_affine() still subtracts that from the rq weight.

Since nobody should be using se->weight for RT tasks, set the value to
zero. Also, since we now use ->cpu_power to normalize rq weights to
account for RT cpu usage, add that factor into the imbalance computation.

Reported-by: Mike Galbraith 
Tested-by: Mike Galbraith 
Signed-off-by: Peter Zijlstra 
LKML-Reference: <1275316109.27810.22969.camel@twins>
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched, cgroup: Fixup broken cgroup movement

2011-06-26T16:46:39+00:00

commit b2b5ce022acf5e9f52f7b78c5579994fdde191d4 upstream.

Dima noticed that we fail to correct the ->vruntime of sleeping tasks
when we move them between cgroups.

Reported-by: Dima Zavin 
Signed-off-by: Peter Zijlstra 
Tested-by: Mike Galbraith 
LKML-Reference: <1287150604.29097.1513.camel@twins>
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: fix RCU lockdep splat from task_group()

2011-06-26T16:46:38+00:00

commit 6506cf6ce68d78a5470a8360c965dafe8e4b78e3 upstream.

This addresses the following RCU lockdep splat:

[0.051203] CPU0: AMD QEMU Virtual CPU version 0.12.4 stepping 03
[0.052999] lockdep: fixing up alternatives.
[0.054105]
[0.054106] ===================================================
[0.054999] [ INFO: suspicious rcu_dereference_check() usage. ]
[0.054999] ---------------------------------------------------
[0.054999] kernel/sched.c:616 invoked rcu_dereference_check() without protection!
[0.054999]
[0.054999] other info that might help us debug this:
[0.054999]
[0.054999]
[0.054999] rcu_scheduler_active = 1, debug_locks = 1
[0.054999] 3 locks held by swapper/1:
[0.054999]  #0:  (cpu_add_remove_lock){+.+.+.}, at: [] cpu_up+0x42/0x6a
[0.054999]  #1:  (cpu_hotplug.lock){+.+.+.}, at: [] cpu_hotplug_begin+0x2a/0x51
[0.054999]  #2:  (&rq->lock){-.-...}, at: [] init_idle+0x2f/0x113
[0.054999]
[0.054999] stack backtrace:
[0.054999] Pid: 1, comm: swapper Not tainted 2.6.35 #1
[0.054999] Call Trace:
[0.054999]  [] lockdep_rcu_dereference+0x9b/0xa3
[0.054999]  [] task_group+0x7b/0x8a
[0.054999]  [] set_task_rq+0x13/0x40
[0.054999]  [] init_idle+0xd2/0x113
[0.054999]  [] fork_idle+0xb8/0xc7
[0.054999]  [] ? mark_held_locks+0x4d/0x6b
[0.054999]  [] do_fork_idle+0x17/0x2b
[0.054999]  [] native_cpu_up+0x1c1/0x724
[0.054999]  [] ? do_fork_idle+0x0/0x2b
[0.054999]  [] _cpu_up+0xac/0x127
[0.054999]  [] cpu_up+0x55/0x6a
[0.054999]  [] kernel_init+0xe1/0x1ff
[0.054999]  [] kernel_thread_helper+0x4/0x10
[0.054999]  [] ? restore_args+0x0/0x30
[0.054999]  [] ? kernel_init+0x0/0x1ff
[0.054999]  [] ? kernel_thread_helper+0x0/0x10
[0.056074] Booting Node   0, Processors  #1lockdep: fixing up alternatives.
[0.130045]  #2lockdep: fixing up alternatives.
[0.203089]  #3 Ok.
[0.275286] Brought up 4 CPUs
[0.276005] Total of 4 processors activated (16017.17 BogoMIPS).

The cgroup_subsys_state structures referenced by idle tasks are never
freed, because the idle tasks should be part of the root cgroup,
which is not removable.

The problem is that while we do in-fact hold rq->lock, the newly spawned
idle thread's cpu is not yet set to the correct cpu so the lockdep check
in task_group():

  lockdep_is_held(&task_rq(p)->lock)

will fail.

But this is a chicken and egg problem.  Setting the CPU's runqueue requires
that the CPU's runqueue already be set.  ;-)

So insert an RCU read-side critical section to avoid the complaint.

Signed-off-by: Peter Zijlstra 
Signed-off-by: Paul E. McKenney 
Signed-off-by: Paul Gortmaker

sched: Move sched_avg_update() to update_cpu_load()

2011-06-26T16:46:38+00:00

commit da2b71edd8a7db44fe1746261410a981f3e03632 upstream.

Currently sched_avg_update() (which updates rt_avg stats in the rq)
is getting called from scale_rt_power() (in the load balance context)
which doesn't take rq->lock.

Fix it by moving the sched_avg_update() to more appropriate
update_cpu_load() where the CFS load gets updated as well.

Signed-off-by: Suresh Siddha 
Signed-off-by: Peter Zijlstra 
LKML-Reference: <1282596171.2694.3.camel@sbsiddha-MOBL3>
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: Fix string comparison in /proc/sched_features

2011-01-06T23:08:19+00:00

commit 7740191cd909b75d75685fb08a5d1f54b8a9d28b upstream.

Fix incorrect handling of the following case:

 INTERACTIVE
 INTERACTIVE_SOMETHING_ELSE

The comparison only checks up to each element's length.

Changelog since v1:
 - Embellish using some Rostedtisms.
  [ mingo:                 ^^ == smaller and cleaner ]

Signed-off-by: Mathieu Desnoyers 
Reviewed-by: Steven Rostedt 
Cc: Peter Zijlstra 
Cc: Tony Lindgren 
LKML-Reference: <20100913214700.GB16118@Krystal>
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: Fix user time incorrectly accounted as system time on 32-bit

2011-01-06T23:07:59+00:00

commit e75e863dd5c7d96b91ebbd241da5328fc38a78cc upstream.

We have 32-bit variable overflow possibility when multiply in
task_times() and thread_group_times() functions. When the
overflow happens then the scaled utime value becomes erroneously
small and the scaled stime becomes i erroneously big.

Reported here:

 https://bugzilla.redhat.com/show_bug.cgi?id=633037
 https://bugzilla.kernel.org/show_bug.cgi?id=16559

Reported-by: Michael Chapman 
Reported-by: Ciriaco Garcia de Celis 
Signed-off-by: Stanislaw Gruszka 
Signed-off-by: Peter Zijlstra 
Cc: Hidetoshi Seto 
LKML-Reference: <20100914143513.GB8415@redhat.com>
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: Pre-compute cpumask_weight(sched_domain_span(sd))

2011-01-06T23:07:52+00:00

commit 669c55e9f99b90e46eaa0f98a67ec53d46dc969a upstream.

Dave reported that his large SPARC machines spend lots of time in
hweight64(), try and optimize some of those needless cpumask_weight()
invocations (esp. with the large offstack cpumasks these are very
expensive indeed).

Reported-by: David Miller 
Signed-off-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: Fix nr_uninterruptible count

2011-01-06T23:07:51+00:00

commit cc87f76a601d2d256118f7bab15e35254356ae21 upstream.

The cpuload calculation in calc_load_account_active() assumes
rq->nr_uninterruptible will not change on an offline cpu after
migrate_nr_uninterruptible(). However the recent migrate on wakeup
changes broke that and would result in decrementing the offline cpu's
rq->nr_uninterruptible.

Fix this by accounting the nr_uninterruptible on the waking cpu.

Signed-off-by: Peter Zijlstra 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker

sched: Optimize task_rq_lock()

2011-01-06T23:07:51+00:00

commit 65cc8e4859ff29a9ddc989c88557d6059834c2a2 upstream.

Now that we hold the rq->lock over set_task_cpu() again, we can do
away with most of the TASK_WAKING checks and reduce them again to
set_cpus_allowed_ptr().

Removes some conditionals from scheduling hot-paths.

Signed-off-by: Peter Zijlstra 
Cc: Oleg Nesterov 
LKML-Reference: 
Signed-off-by: Ingo Molnar 
Signed-off-by: Paul Gortmaker