linux-toradex.git/kernel/sched_fair.c, branch v2.6.28.7

sched: SCHED_OTHER vs SCHED_IDLE isolation

2009-02-20T22:40:20+00:00

commit 6bc912b71b6f33b041cfde93ca3f019cbaa852bc upstream.

Stronger SCHED_IDLE isolation:

 - no SCHED_IDLE buddies
 - never let SCHED_IDLE preempt on wakeup
 - always preempt SCHED_IDLE on wakeup
 - limit SLEEPER fairness for SCHED_IDLE.

Signed-off-by: Mike Galbraith 
Signed-off-by: Peter Zijlstra 
Signed-off-by: Ingo Molnar 
Signed-off-by: Greg Kroah-Hartman

sched: fix update_min_vruntime

2009-01-25T00:41:35+00:00

commit e17036dac189dd034c092a91df56aa740db7146d upstream.

Impact: fix SCHED_IDLE latency problems

OK, so we have 1 running task A (which is obviously curr and the tree is
equally obviously empty).

'A' nicely chugs along, doing its thing, carrying min_vruntime along as it
goes.

Then some whacko speed freak SCHED_IDLE task gets inserted due to SMP
balancing, which is very likely far right, in that case

update_curr
  update_min_vruntime
    cfs_rq->rb_leftmost := true (the crazy task sitting in a tree)
      vruntime = se->vruntime

and voila, min_vruntime is waaay right of where it ought to be.

OK, so why did I write it like that to begin with...

Aah, yes.

Say we've just dequeued current

schedule
  deactivate_task(prev)
    dequeue_entity
      update_min_vruntime

Then we'll set

  vruntime = cfs_rq->min_vruntime;

we find !cfs_rq->curr, but do find someone in the tree. Then we _must_
do vruntime = se->vruntime, because

 vruntime = min_vruntime(vruntime := cfs_rq->min_vruntime, se->vruntime)

will not advance vruntime, and cause lags the other way around (which we
fixed with that initial patch: 1af5f730fc1bf7c62ec9fb2d307206e18bf40a69
(sched: more accurate min_vruntime accounting).

Signed-off-by: Peter Zijlstra 
Tested-by: Mike Galbraith 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar 
Signed-off-by: Greg Kroah-Hartman

sched: release buddies on yield

2008-11-11T10:57:22+00:00

Clear buddies on yield, so that the buddy rules don't schedule them
despite them being placed right-most.

This fixed a performance regression with yield-happy binary JVMs.

Signed-off-by: Peter Zijlstra 
Signed-off-by: Ingo Molnar 
Tested-by: Lin Ming

sched: fix buddies for group scheduling

2008-11-05T09:30:15+00:00

Impact: scheduling order fix for group scheduling

For each level in the hierarchy, set the buddy to point to the right entity.
Therefore, when we do the hierarchical schedule, we have a fair chance of
ending up where we meant to.

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: backward looking buddy

2008-11-05T09:30:14+00:00

Impact: improve/change/fix wakeup-buddy scheduling

Currently we only have a forward looking buddy, that is, we prefer to
schedule to the task we last woke up, under the presumption that its
going to consume the data we just produced, and therefore will have
cache hot benefits.

This allows co-waking producer/consumer task pairs to run ahead of the
pack for a little while, keeping their cache warm. Without this, we
would interleave all pairs, utterly trashing the cache.

This patch introduces a backward looking buddy, that is, suppose that
in the above scenario, the consumer preempts the producer before it
can go to sleep, we will therefore miss the wakeup from consumer to
producer (its already running, after all), breaking the cycle and
reverting to the cache-trashing interleaved schedule pattern.

The backward buddy will try to schedule back to the task that woke us
up in case the forward buddy is not available, under the assumption
that the last task will be the one with the most cache hot task around
barring current.

This will basically allow a task to continue after it got preempted.

In order to avoid starvation, we allow either buddy to get wakeup_gran
ahead of the pack.

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: fix fair preempt check

2008-11-05T09:30:13+00:00

Impact: fix cross-class preemption

Inter-class wakeup preemptions should go on class order.

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: cleanup fair task selection

2008-11-05T09:30:13+00:00

Impact: cleanup

Clean up task selection

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: virtual time buddy preemption

2008-10-24T10:51:03+00:00

Since we moved wakeup preemption back to virtual time, it makes sense to move
the buddy stuff back as well. The purpose of the buddy scheduling is to allow
a quickly scheduling pair of tasks to run away from the group as far as a
regular busy task would be allowed under wakeup preemption.

This has the advantage that the pair can ping-pong for a while, enjoying
cache-hotness. Without buddy scheduling other tasks would interleave destroying
the cache.

Also, it saves a word in cfs_rq.

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: re-instate vruntime based wakeup preemption

2008-10-24T10:51:02+00:00

The advantage is that vruntime based wakeup preemption has a better
conceptual model. Here wakeup_gran = 0 means: preempt when 'fair'.
Therefore wakeup_gran is the granularity of unfairness we allow in order
to make progress.

Signed-off-by: Peter Zijlstra 
Acked-by: Mike Galbraith 
Signed-off-by: Ingo Molnar

sched: weaken sync hint

2008-10-24T10:51:01+00:00

Mysql+oltp and pgsql+oltp peaks are still shifted right. The below puts
the peaks back to 1 client/server pair per core.

Use the avg_overlap information to weaken the sync hint.

Signed-off-by: Mike Galbraith 
Signed-off-by: Peter Zijlstra 
Signed-off-by: Ingo Molnar