linux-toradex.git/mm/percpu.c, branch v4.4.129

percpu: acquire pcpu_lock when updating pcpu_nr_empty_pop_pages

2017-03-26T10:13:20+00:00

commit 320661b08dd6f1746d5c7ab4eb435ec64b97cd45 upstream.

Update to pcpu_nr_empty_pop_pages in pcpu_alloc() is currently done
without holding pcpu_lock. This can lead to bad updates to the variable.
Add missing lock calls.

Fixes: b539b87fed37 ("percpu: implmeent pcpu_nr_empty_pop_pages and chunk->nr_populated")
Signed-off-by: Tahsin Erdogan 
Signed-off-by: Tejun Heo 
Signed-off-by: Greg Kroah-Hartman

percpu: fix synchronization between synchronous map extension and chunk destruction

2016-07-27T16:47:33+00:00

commit 6710e594f71ccaad8101bc64321152af7cd9ea28 upstream.

For non-atomic allocations, pcpu_alloc() can try to extend the area
map synchronously after dropping pcpu_lock; however, the extension
wasn't synchronized against chunk destruction and the chunk might get
freed while extension is in progress.

This patch fixes the bug by putting most of non-atomic allocations
under pcpu_alloc_mutex to synchronize against pcpu_balance_work which
is responsible for async chunk management including destruction.

Signed-off-by: Tejun Heo 
Reported-and-tested-by: Alexei Starovoitov 
Reported-by: Vlastimil Babka 
Reported-by: Sasha Levin 
Fixes: 1a4d76076cda ("percpu: implement asynchronous chunk population")
Signed-off-by: Greg Kroah-Hartman

percpu: fix synchronization between chunk->map_extend_work and chunk destruction

2016-07-27T16:47:33+00:00

commit 4f996e234dad488e5d9ba0858bc1bae12eff82c3 upstream.

Atomic allocations can trigger async map extensions which is serviced
by chunk->map_extend_work.  pcpu_balance_work which is responsible for
destroying idle chunks wasn't synchronizing properly against
chunk->map_extend_work and may end up freeing the chunk while the work
item is still in flight.

This patch fixes the bug by rolling async map extension operations
into pcpu_balance_work.

Signed-off-by: Tejun Heo 
Reported-and-tested-by: Alexei Starovoitov 
Reported-by: Vlastimil Babka 
Reported-by: Sasha Levin 
Fixes: 9c824b6a172c ("percpu: make sure chunk->map array has available space")
Signed-off-by: Greg Kroah-Hartman

mm/percpu: use offset_in_page macro

2015-11-06T03:34:48+00:00

linux/mm.h provides offset_in_page() macro.  Let's use already predefined
macro instead of (addr & ~PAGE_MASK).

Signed-off-by: Alexander Kuleshov 
Acked-by: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

percpu: clean up of schunk->map[] assignment in pcpu_setup_first_chunk

2015-07-21T15:31:00+00:00

The original assignment is a little redundent.

Signed-off-by: Baoquan He 
Acked-by: Christoph Lameter 
Signed-off-by: Tejun Heo

mm: kmemleak_alloc_percpu() should follow the gfp from per_alloc()

2015-06-25T00:49:46+00:00

Beginning at commit d52d3997f843 ("ipv6: Create percpu rt6_info"), the
following INFO splat is logged:

  ===============================
  [ INFO: suspicious RCU usage. ]
  4.1.0-rc7-next-20150612 #1 Not tainted
  -------------------------------
  kernel/sched/core.c:7318 Illegal context switch in RCU-bh read-side critical section!
  other info that might help us debug this:
  rcu_scheduler_active = 1, debug_locks = 0
   3 locks held by systemd/1:
   #0:  (rtnl_mutex){+.+.+.}, at: [] rtnetlink_rcv+0x1f/0x40
   #1:  (rcu_read_lock_bh){......}, at: [] ipv6_add_addr+0x62/0x540
   #2:  (addrconf_hash_lock){+...+.}, at: [] ipv6_add_addr+0x184/0x540
  stack backtrace:
  CPU: 0 PID: 1 Comm: systemd Not tainted 4.1.0-rc7-next-20150612 #1
  Hardware name: TOSHIBA TECRA A50-A/TECRA A50-A, BIOS Version 4.20   04/17/2014
  Call Trace:
    dump_stack+0x4c/0x6e
    lockdep_rcu_suspicious+0xe7/0x120
    ___might_sleep+0x1d5/0x1f0
    __might_sleep+0x4d/0x90
    kmem_cache_alloc+0x47/0x250
    create_object+0x39/0x2e0
    kmemleak_alloc_percpu+0x61/0xe0
    pcpu_alloc+0x370/0x630

Additional backtrace lines are truncated.  In addition, the above splat
is followed by several "BUG: sleeping function called from invalid
context at mm/slub.c:1268" outputs.  As suggested by Martin KaFai Lau,
these are the clue to the fix.  Routine kmemleak_alloc_percpu() always
uses GFP_KERNEL for its allocations, whereas it should follow the gfp
from its callers.

Reviewed-by: Catalin Marinas 
Reviewed-by: Kamalesh Babulal 
Acked-by: Martin KaFai Lau 
Signed-off-by: Larry Finger 
Cc: Martin KaFai Lau 
Cc: Catalin Marinas 
Cc: Tejun Heo 
Cc: Christoph Lameter 
Cc: 	[3.18+]
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

percpu: Fix trivial typos in comments

2015-03-24T17:41:54+00:00

Change 'tranlated' to 'translated'
Change 'mutliples' to 'multiples'

Signed-off-by: Yannick Guerrini 
Signed-off-by: Tejun Heo

percpu: use %*pb[l] to print bitmaps including cpumasks and nodemasks

2015-02-14T05:21:37+00:00

printk and friends can now format bitmaps using '%*pb[l]'.  cpumask
and nodemask also provide cpumask_pr_args() and nodemask_pr_args()
respectively which can be used to generate the two printf arguments
necessary to format the specified cpu/nodemask.

Signed-off-by: Tejun Heo 
Cc: Christoph Lameter 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

percpu: off by one in BUG_ON()

2014-10-29T14:34:34+00:00

The unit_map[] array has "nr_cpu_ids" number of elements.  It's
allocated a few lines earlier in the function.  So this test should be
>= instead of >.

Signed-off-by: Dan Carpenter 
Signed-off-by: Tejun Heo

percpu: fix how @gfp is interpreted by the percpu allocator

2014-10-08T16:01:52+00:00

When @gfp is specified, the percpu allocator is interested in whether
it contains all of GFP_KERNEL or not.  If it does, the normal
allocation path is taken; otherwise, the atomic allocation path.
Unfortunately, pcpu_alloc() was incorrectly testing for whether @gfp
contains any part of GFP_KERNEL.

Fix it by testing "(gfp & GFP_KERNEL) != GFP_KERNEL" instead of
"!(gfp & GFP_KERNEL)" to decide whether the allocation should be
atomic or not.

Signed-off-by: Tejun Heo