linux-toradex.git/mm/mempool.c, branch v4.2.2

mm/mempool.c: kasan: poison mempool elements

2015-04-15T23:35:20+00:00

Mempools keep allocated objects in reserved for situations when ordinary
allocation may not be possible to satisfy.  These objects shouldn't be
accessed before they leave the pool.

This patch poison elements when get into the pool and unpoison when they
leave it.  This will let KASan to detect use-after-free of mempool's
elements.

Signed-off-by: Andrey Ryabinin 
Tested-by: David Rientjes 
Cc: Catalin Marinas 
Cc: Dmitry Chernenkov 
Cc: Dmitry Vyukov 
Cc: Alexander Potapenko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm, mempool: poison elements backed by slab allocator

2015-04-15T23:35:18+00:00

Mempools keep elements in a reserved pool for contexts in which allocation
may not be possible.  When an element is allocated from the reserved pool,
its memory contents is the same as when it was added to the reserved pool.

Because of this, elements lack any free poisoning to detect use-after-free
errors.

This patch adds free poisoning for elements backed by the slab allocator.
This is possible because the mempool layer knows the object size of each
element.

When an element is added to the reserved pool, it is poisoned with
POISON_FREE.  When it is removed from the reserved pool, the contents are
checked for POISON_FREE.  If there is a mismatch, a warning is emitted to
the kernel log.

This is only effective for configs with CONFIG_DEBUG_SLAB or
CONFIG_SLUB_DEBUG_ON.

[fabio.estevam@freescale.com: use '%zu' for printing 'size_t' variable]
[arnd@arndb.de: add missing include]
Signed-off-by: David Rientjes 
Cc: Dave Kleikamp 
Cc: Christoph Hellwig 
Cc: Sebastian Ott 
Cc: Mikulas Patocka 
Cc: Catalin Marinas 
Signed-off-by: Fabio Estevam 
Signed-off-by: Arnd Bergmann 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm, mempool: disallow mempools based on slab caches with constructors

2015-04-15T23:35:18+00:00

All occurrences of mempools based on slab caches with object constructors
have been removed from the tree, so disallow creating them.

We can only dereference mem->ctor in mm/mempool.c without including
mm/slab.h in include/linux/mempool.h.  So simply note the restriction,
just like the comment restricting usage of __GFP_ZERO, and warn on kernels
with CONFIG_DEBUG_VM() if such a mempool is allocated from.

We don't want to incur this check on every element allocation, so use
VM_BUG_ON().

Signed-off-by: David Rientjes 
Cc: Dave Kleikamp 
Cc: Christoph Hellwig 
Cc: Sebastian Ott 
Cc: Mikulas Patocka 
Cc: Catalin Marinas 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm, mempool: do not allow atomic resizing

2015-04-14T23:49:06+00:00

Allocating a large number of elements in atomic context could quickly
deplete memory reserves, so just disallow atomic resizing entirely.

Nothing currently uses mempool_resize() with anything other than
GFP_KERNEL, so convert existing callers to drop the gfp_mask.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: David Rientjes 
Acked-by: Steffen Maier 	[zfcp]
Cc: Martin Schwidefsky 
Cc: Heiko Carstens 
Cc: Steve French 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/mempool.c: update the kmemleak stack trace for mempool allocations

2014-06-06T23:08:17+00:00

When mempool_alloc() returns an existing pool object, kmemleak_alloc()
is no longer called and the stack trace corresponds to the original
object allocation.  This patch updates the kmemleak allocation stack
trace for such objects to make it more useful for debugging.

Signed-off-by: Catalin Marinas 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/mempool: warn about __GFP_ZERO usage

2014-06-04T23:53:58+00:00

Memory obtained via mempool_alloc is not always zeroed even when
called with __GFP_ZERO. Add a note and VM_BUG_ON statement to make
that clear.

[akpm@linux-foundation.org: use VM_WARN_ON_ONCE]
Signed-off-by: Sebastian Ott 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mempool: add unlikely and likely hints

2014-04-07T23:35:55+00:00

Add unlikely and likely hints to the function mempool_free.  It lays out
the code in such a way that the common path is executed straighforward and
saves a cache line.

Signed-off-by: Mikulas Patocka 
Cc: Ingo Molnar 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/mempool.c: convert kmalloc_node(...GFP_ZERO...) to kzalloc_node(...)

2013-09-11T22:58:14+00:00

Use the helper function instead of __GFP_ZERO.

Signed-off-by: Joe Perches 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mempool: add @gfp_mask to mempool_create_node()

2012-06-25T09:53:47+00:00

mempool_create_node() currently assumes %GFP_KERNEL.  Its only user,
blk_init_free_list(), is about to be updated to use other allocation
flags - add @gfp_mask argument to the function.

Signed-off-by: Tejun Heo 
Cc: Andrew Morton 
Cc: Hugh Dickins 
Signed-off-by: Jens Axboe

mempool: fix first round failure behavior

2012-01-11T00:30:45+00:00

mempool modifies gfp_mask so that the backing allocator doesn't try too
hard or trigger warning message when there's pool to fall back on.  In
addition, for the first try, it removes __GFP_WAIT and IO, so that it
doesn't trigger reclaim or wait when allocation can be fulfilled from
pool; however, when that allocation fails and pool is empty too, it waits
for the pool to be replenished before retrying.

Allocation which could have succeeded after a bit of reclaim has to wait
on the reserved items and it's not like mempool doesn't retry with
__GFP_WAIT and IO.  It just does that *after* someone returns an element,
pointlessly delaying things.

Fix it by retrying immediately if the first round of allocation attempts
w/o __GFP_WAIT and IO fails.

[akpm@linux-foundation.org: shorten the lock hold time]
Signed-off-by: Tejun Heo 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds