summaryrefslogtreecommitdiff
path: root/kernel
AgeCommit message (Collapse)Author
2011-12-28sched: Optimize migrate_disablePeter Zijlstra
Change from task_rq_lock() to raw_spin_lock(&rq->lock) to avoid a few atomic ops. See comment on why it should be safe. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/n/tip-cbz6hkl5r5mvwtx5s3tor2y6@git.kernel.org
2011-12-28tracing: Show padding as unsigned shortSteven Rostedt
RT added two bytes to trace migrate disable counting to the trace events and used two bytes of the padding to make the change. The structures and all were updated correctly, but the display in the event formats was not: cat /debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 51 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1; field:unsigned short common_migrate_disable; offset:8; size:2; signed:0; field:int common_padding; offset:10; size:2; signed:0; The field for common_padding has the correct size and offset, but the use of "int" might confuse some parsers (and people that are reading it). This needs to be changed to "unsigned short". Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Link: http://lkml.kernel.org/r/1321467575.4181.36.camel@frodo Cc: stable-rt@vger.kernel.org Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28ftrace-migrate-disable-tracing.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hotplug: Call cpu_unplug_begin() before DOWN_PREPAREYong Zhang
cpu_unplug_begin() should be called before CPU_DOWN_PREPARE, because at CPU_DOWN_PREPARE cpu_active is cleared and sched_domain is rebuilt. Otherwise the 'sync_unplug' thread will be running on the cpu on which it's created and not bound on the cpu which is about to go down. I found that by an incorrect warning on smp_processor_id() called by sync_unplug/1, and trace shows below: (echo 1 > /sys/device/system/cpu/cpu1/online) bash-1664 [000] 83.136620: _cpu_down: Bind sync_unplug to cpu 1 bash-1664 [000] 83.136623: sched_wait_task: comm=sync_unplug/1 pid=1724 prio=120 bash-1664 [000] 83.136624: _cpu_down: Wake sync_unplug bash-1664 [000] 83.136629: sched_wakeup: comm=sync_unplug/1 pid=1724 prio=120 success=1 target_cpu=000 Wants to be folded back.... Signed-off-by: Yong Zhang <yong.zhang0@gmail.com> Link: http://lkml.kernel.org/r/1318762607-2261-3-git-send-email-yong.zhang0@gmail.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hotplug-use-migrate-disable.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-migrate-disable.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hotplug: Reread hotplug_pcp on pin_current_cpu() retryYong Zhang
When retry happens, it's likely that the task has been migrated to another cpu (except unplug failed), but it still derefernces the original hotplug_pcp per cpu data. Update the pointer to hotplug_pcp in the retry path, so it points to the current cpu. Signed-off-by: Yong Zhang <yong.zhang0@gmail.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/20110728031600.GA338@windriver.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hotplug: sync_unplug: No "\n" in task nameYong Zhang
Otherwise the output will look a little odd. Signed-off-by: Yong Zhang <yong.zhang0@gmail.com> Link: http://lkml.kernel.org/r/1318762607-2261-2-git-send-email-yong.zhang0@gmail.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hotplug: Lightweight get online cpusThomas Gleixner
get_online_cpus() is a heavy weight function which involves a global mutex. migrate_disable() wants a simpler construct which prevents only a CPU from going doing while a task is in a migrate disabled section. Implement a per cpu lockless mechanism, which serializes only in the real unplug case on a global mutex. That serialization affects only tasks on the cpu which should be brought down. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28stomp-machine-raw-lock.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28stomp-machine-mark-stomper-thread.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28stop_machine: convert stop_machine_run() to PREEMPT_RTIngo Molnar
Instead of playing with non-preemption, introduce explicit startup serialization. This is more robust and cleaner as well. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched: ttwu: Return success when only changing the saved_state valueThomas Gleixner
When a task blocks on a rt lock, it saves the current state in p->saved_state, so a lock related wake up will not destroy the original state. When a real wakeup happens, while the task is running due to a lock wakeup already, we update p->saved_state to TASK_RUNNING, but we do not return success, which might cause another wakeup in the waitqueue code and the task remains in the waitqueue list. Return success in that case as well. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: stable-rt@vger.kernel.org
2011-12-28sched-disable-ttwu-queue.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-no-work-when-pi-blocked.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28cond-resched-softirq-fix.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-cond-resched.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched: Break out from load_balancing on rq_lock contentionPeter Zijlstra
Also limit NEW_IDLE pull Signed-off-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-might-sleep-do-not-account-rcu-depth.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-prevent-idle-boost.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-rt-mutex-wakeup.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-mmdrop-delayed.patchThomas Gleixner
Needs thread context (pgd_lock) -> ifdeffed. workqueues wont work with RT Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-limit-nr-migrate.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28sched-delay-put-task.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28posix-timers: Avoid wakeups when no timers are activeThomas Gleixner
Waking the thread even when no timers are scheduled is useless. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28posix-timers: Shorten posix_cpu_timers/<CPU> kernel thread namesArnaldo Carvalho de Melo
Shorten the softirq kernel thread names because they always overflow the limited comm length, appearing as "posix_cpu_timer" CPU# times. Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28posix-timers: thread posix-cpu-timers on -rtJohn Stultz
posix-cpu-timer code takes non -rt safe locks in hard irq context. Move it to a thread. [ 3.0 fixes from Peter Zijlstra <peterz@infradead.org> ] Signed-off-by: John Stultz <johnstul@us.ibm.com> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hrtimer-fix-reprogram-madness.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28hrtimer: Add missing debug_activate() aid [Was: Re: [ANNOUNCE] 3.0.6-rt17]Yong Zhang
On Fri, Oct 07, 2011 at 10:25:25AM -0700, Fernando Lopez-Lezcano wrote: > On 10/06/2011 06:15 PM, Thomas Gleixner wrote: > >Dear RT Folks, > > > >I'm pleased to announce the 3.0.6-rt17 release. > > Hi and thanks again. So far this one is not hanging which is very > good news. But I still see the hrtimer_fixup_activate warnings I > reported for rt16... Hi Fernando, I think below patch will smooth your concern? Thanks, Yong
2011-12-28hrtimer: Don't call the timer handler from hrtimer_startPeter Zijlstra
[<ffffffff812de4a9>] __delay+0xf/0x11 [<ffffffff812e36e9>] do_raw_spin_lock+0xd2/0x13c [<ffffffff815028ee>] _raw_spin_lock+0x60/0x73 rt_b->rt_runtime_lock [<ffffffff81068f68>] ? sched_rt_period_timer+0xad/0x281 [<ffffffff81068f68>] sched_rt_period_timer+0xad/0x281 [<ffffffff8109e5e1>] __run_hrtimer+0x1e4/0x347 [<ffffffff81068ebb>] ? enqueue_rt_entity+0x36/0x36 [<ffffffff8109f2b1>] __hrtimer_start_range_ns+0x2b5/0x40a base->cpu_base->lock (lock_hrtimer_base) [<ffffffff81068b6f>] __enqueue_rt_entity+0x26f/0x2aa rt_b->rt_runtime_lock (start_rt_bandwidth) [<ffffffff81068ead>] enqueue_rt_entity+0x28/0x36 [<ffffffff81069355>] enqueue_task_rt+0x3d/0xb0 [<ffffffff810679d6>] enqueue_task+0x5d/0x64 [<ffffffff810714fc>] task_setprio+0x210/0x29c rq->lock [<ffffffff810b56cb>] __rt_mutex_adjust_prio+0x25/0x2a p->pi_lock [<ffffffff810b5d2c>] task_blocks_on_rt_mutex+0x196/0x20f Instead make __hrtimer_start_range_ns() return -ETIME when the timer is in the past. Since body actually uses the hrtimer_start*() return value its pretty safe to wreck it. Also, it will only ever return -ETIME for timer->irqsafe || !wakeup timers. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
2011-12-28hrtimer: fixup hrtimer callback changes for preempt-rtThomas Gleixner
In preempt-rt we can not call the callbacks which take sleeping locks from the timer interrupt context. Bring back the softirq split for now, until we fixed the signal delivery problem for real. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
2011-12-28hrtimers: prepare full preemptionIngo Molnar
Make cancellation of a running callback in softirq context safe against preemption. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28printk: Don't call printk_tick in printk_needs_cpu() on RTYong Zhang
printk_tick() can't be called in atomic context when RT is enabled, otherwise below warning will show: [ 117.597095] BUG: sleeping function called from invalid context at kernel/rtmutex.c:645 [ 117.597102] in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: kworker/0:0 [ 117.597111] Pid: 0, comm: kworker/0:0 Not tainted 3.0.6-rt17-00284-gb76d419-dirty #7 [ 117.597116] Call Trace: [ 117.597131] [<c06e3b61>] ? printk+0x1d/0x24 [ 117.597142] [<c01390b6>] __might_sleep+0xe6/0x110 [ 117.597151] [<c06e634c>] rt_spin_lock+0x1c/0x30 [ 117.597158] [<c0142f26>] __wake_up+0x26/0x60 [ 117.597166] [<c014c78e>] printk_tick+0x3e/0x40 [ 117.597173] [<c014c7b4>] printk_needs_cpu+0x24/0x30 [ 117.597181] [<c017ecc8>] tick_nohz_stop_sched_tick+0x2e8/0x410 [ 117.597191] [<c017305a>] ? sched_clock_idle_wakeup_event+0x1a/0x20 [ 117.597201] [<c010182a>] cpu_idle+0x4a/0xb0 [ 117.597209] [<c06e0b97>] start_secondary+0xd3/0xd7 Now this is a really rare case and it's very unlikely that we starve an logbuf waiter that way. Signed-off-by: Yong Zhang <yong.zhang0@gmail.com> Link: http://lkml.kernel.org/r/1318762607-2261-4-git-send-email-yong.zhang0@gmail.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28timers: Avoid the switch timers base set to NULL trick on RTThomas Gleixner
On RT that code is preemptible, so we cannot assign NULL to timers base as a preempter would spin forever in lock_timer_base(). Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28timer: delay waking softirqs from the jiffy tickPeter Zijlstra
People were complaining about broken balancing with the recent -rt series. A look at /proc/sched_debug yielded: cpu#0, 2393.874 MHz .nr_running : 0 .load : 0 .cpu_load[0] : 177522 .cpu_load[1] : 177522 .cpu_load[2] : 177522 .cpu_load[3] : 177522 .cpu_load[4] : 177522 cpu#1, 2393.874 MHz .nr_running : 4 .load : 4096 .cpu_load[0] : 181618 .cpu_load[1] : 180850 .cpu_load[2] : 180274 .cpu_load[3] : 179938 .cpu_load[4] : 179758 Which indicated the cpu_load computation was hosed, the 177522 value indicates that there is one RT task runnable. Initially I thought the old problem of calculating the cpu_load from a softirq had re-surfaced, however looking at the code shows its being done from scheduler_tick(). [ we really should fix this RT/cfs interaction some day... ] A few trace_printk()s later: sirq-timer/1-19 [001] 174.289744: 19: 50:S ==> [001] 0:140:R <idle> <idle>-0 [001] 174.290724: enqueue_task_rt: adding task: 19/sirq-timer/1 with load: 177522 <idle>-0 [001] 174.290725: 0:140:R + [001] 19: 50:S sirq-timer/1 <idle>-0 [001] 174.290730: scheduler_tick: current load: 177522 <idle>-0 [001] 174.290732: scheduler_tick: current: 0/swapper <idle>-0 [001] 174.290736: 0:140:R ==> [001] 19: 50:R sirq-timer/1 sirq-timer/1-19 [001] 174.290741: dequeue_task_rt: removing task: 19/sirq-timer/1 with load: 177522 sirq-timer/1-19 [001] 174.290743: 19: 50:S ==> [001] 0:140:R <idle> We see that we always raise the timer softirq before doing the load calculation. Avoid this by re-ordering the scheduler_tick() call in update_process_times() to occur before we deal with timers. This lowers the load back to sanity and restores regular load-balancing behaviour. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28timers: mov printk_tick to soft interruptThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@elte.hu>
2011-12-28timers: fix timer hotplug on -rtIngo Molnar
Here we are in the CPU_DEAD notifier, and we must not sleep nor enable interrupts. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28timers: preempt-rt supportIngo Molnar
Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28timers: prepare for full preemptionIngo Molnar
When softirqs can be preempted we need to make sure that cancelling the timer from the active thread can not deadlock vs. a running timer callback. Add a waitqueue to resolve that. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28workqueue-avoid-the-lock-in-cpu-dying.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28relay: fix timer madnessIngo Molnar
remove timer calls (!!!) from deep within the tracing infrastructure. This was totally bogus code that can cause lockups and worse. Poll the buffer every 2 jiffies for now. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28panic-disable-random-on-rtThomas Gleixner
2011-12-28genirq-force-threading.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28genirq: disable irqpoll on -rtIngo Molnar
Creates long latencies for no value Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28genirq: Disable random call on preempt-rtThomas Gleixner
The random call introduces high latencies and is almost unused. Disable it for -rt. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28signal-fix-up-rcu-wreckage.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28core: Do not disable interrupts on RT in res_counter.cIngo Molnar
Frederic Weisbecker reported this warning: [ 45.228562] BUG: sleeping function called from invalid context at kernel/rtmutex.c:683 [ 45.228571] in_atomic(): 0, irqs_disabled(): 1, pid: 4290, name: ntpdate [ 45.228576] INFO: lockdep is turned off. [ 45.228580] irq event stamp: 0 [ 45.228583] hardirqs last enabled at (0): [<(null)>] (null) [ 45.228589] hardirqs last disabled at (0): [<ffffffff8025449d>] copy_process+0x68d/0x1500 [ 45.228602] softirqs last enabled at (0): [<ffffffff8025449d>] copy_process+0x68d/0x1500 [ 45.228609] softirqs last disabled at (0): [<(null)>] (null) [ 45.228617] Pid: 4290, comm: ntpdate Tainted: G W 2.6.29-rc4-rt1-tip #1 [ 45.228622] Call Trace: [ 45.228632] [<ffffffff8027dfb0>] ? print_irqtrace_events+0xd0/0xe0 [ 45.228639] [<ffffffff8024cd73>] __might_sleep+0x113/0x130 [ 45.228646] [<ffffffff8077c811>] rt_spin_lock+0xa1/0xb0 [ 45.228653] [<ffffffff80296a3d>] res_counter_charge+0x5d/0x130 [ 45.228660] [<ffffffff802fb67f>] __mem_cgroup_try_charge+0x7f/0x180 [ 45.228667] [<ffffffff802fc407>] mem_cgroup_charge_common+0x57/0x90 [ 45.228674] [<ffffffff80212096>] ? ftrace_call+0x5/0x2b [ 45.228680] [<ffffffff802fc49d>] mem_cgroup_newpage_charge+0x5d/0x60 [ 45.228688] [<ffffffff802d94ce>] __do_fault+0x29e/0x4c0 [ 45.228694] [<ffffffff8077c843>] ? rt_spin_unlock+0x23/0x80 [ 45.228700] [<ffffffff802db8b5>] handle_mm_fault+0x205/0x890 [ 45.228707] [<ffffffff80212096>] ? ftrace_call+0x5/0x2b [ 45.228714] [<ffffffff8023495e>] do_page_fault+0x11e/0x2a0 [ 45.228720] [<ffffffff8077e5a5>] page_fault+0x25/0x30 [ 45.228727] [<ffffffff8043e1ed>] ? __clear_user+0x3d/0x70 [ 45.228733] [<ffffffff8043e1d1>] ? __clear_user+0x21/0x70 The reason is the raw IRQ flag use of kernel/res_counter.c. The irq flags tricks there seem a bit pointless: it cannot protect the c->parent linkage because local_irq_save() is only per CPU. So replace it with _nort(). This code needs a second look. Reported-by: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28core: Do not disable interrupts on RT in kernel/users.cThomas Gleixner
Use the local_irq_*_nort variants to reduce latencies in RT. The code is serialized by the locks. No need to disable interrupts. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28rt-preempt-base-config.patchThomas Gleixner
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2011-12-28printk: 'force_early_printk' boot param to help with debuggingPeter Zijlstra
Gives me an option to screw printk and actually see what the machine says. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1314967289.1301.11.camel@twins Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-ykb97nsfmobq44xketrxs977@git.kernel.org
generated by cgit v1.2.3 (git 2.0.4) at 2025-08-06 18:41:30 +0000