linux-toradex.git/mm/bootmem.c, branch v2.6.31-rc4

kmemleak: Add callbacks to the bootmem allocator

2009-07-08T13:25:14+00:00

This patch adds kmemleak_alloc/free callbacks to the bootmem allocator.
This would allow scanning of such blocks and help avoiding a whole class
of false positives and more kmemleak annotations.

Signed-off-by: Catalin Marinas 
Cc: Ingo Molnar 
Acked-by: Pekka Enberg 
Reviewed-by: Johannes Weiner

bootmem.c: avoid c90 declaration warning

2009-06-19T23:46:04+00:00

[akpm@linux-foundation.org: cleanup]
Signed-off-by: Joe Perches 
Cc: Pekka Enberg 
Cc: Benjamin Herrenschmidt 
Cc: Tejun Heo 
Cc: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: fix slab fallback on numa

2009-06-11T16:15:54+00:00

If the user requested bootmem allocation on a specific node, we should use
kzalloc_node() for the fallback allocation.

Cc: Ingo Molnar 
Cc: Johannes Weiner 
Cc: Linus Torvalds 
Cc: Yinghai Lu 
Signed-off-by: Pekka Enberg

bootmem: use slab if bootmem is no longer available

2009-06-11T16:10:41+00:00

As a preparation for initializing the slab allocator early, make sure the
bootmem allocator does not crash and burn if someone calls it after slab is up;
otherwise we'd need a flag day for switching to early slab.

Acked-by: Johannes Weiner 
Acked-by: Linus Torvalds 
Cc: Christoph Lameter 
Cc: Ingo Molnar 
Cc: Matt Mackall 
Cc: Nick Piggin 
Cc: Yinghai Lu 
Signed-off-by: Pekka Enberg

bootmem, x86: further fixes for arch-specific bootmem wrapping

2009-03-01T07:06:56+00:00

Impact: fix new breakages introduced by previous fix

Commit c132937556f56ee4b831ef4b23f1846e05fde102 tried to clean up
bootmem arch wrapper but it wasn't quite correct.  Before the commit,
the followings were broken.

* Low level interface functions prefixed with __ ignored arch
  preference.

* reserve_bootmem(...) can't be mapped into
  reserve_bootmem_node(NODE_DATA(0)->bdata, ...) because the node is
  not preference here.  The region specified MUST fall into the
  specified region; otherwise, it will panic.

After the commit,

* If allocation fails for the arch preferred node, it should fallback
  to whatever is available.  Instead, it simply failed allocation.

There are too many internal details to allow generic wrapping and
still keep things simple for archs.  Plus, all that arch wants is a
way to prefer certain node over another.

This patch drops the generic wrapping around alloc_bootmem_core() and
add alloc_bootmem_core() instead.  If necessary, arch can define
bootmem_arch_referred_node() macro or function which takes all
allocation information and returns the preferred node.  bootmem
generic code will always try the preferred node first and then
fallback to other nodes as usual.

Breakages noted and changes reviewed by Johannes Weiner.

Signed-off-by: Tejun Heo 
Acked-by: Johannes Weiner

bootmem: clean up arch-specific bootmem wrapping

2009-02-24T02:57:20+00:00

Impact: cleaner and consistent bootmem wrapping

By setting CONFIG_HAVE_ARCH_BOOTMEM_NODE, archs can define
arch-specific wrappers for bootmem allocation.  However, this is done
a bit strangely in that only the high level convenience macros can be
changed while lower level, but still exported, interface functions
can't be wrapped.  This not only is messy but also leads to strange
situation where alloc_bootmem() does what the arch wants it to do but
the equivalent __alloc_bootmem() call doesn't although they should be
able to be used interchangeably.

This patch updates bootmem such that archs can override / wrap the
backend function - alloc_bootmem_core() instead of the highlevel
interface functions to allow simpler and consistent wrapping.  Also,
HAVE_ARCH_BOOTMEM_NODE is renamed to HAVE_ARCH_BOOTMEM.

Signed-off-by: Tejun Heo 
Cc: Johannes Weiner

bootmem: print request details before BUG_ON(them)

2009-01-06T23:59:10+00:00

Moving the request details print-out before the sanity checks that
might panic() enables us to analyse invalid requests without having
access to the line information of the stack dump.

Signed-off-by: Johannes Weiner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

misc: replace FUNCTION with func

2008-10-16T18:21:30+00:00

__FUNCTION__ is gcc-specific, use __func__

Signed-off-by: Harvey Harrison 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem: fix aligning of node-relative indexes and offsets

2008-08-20T22:40:31+00:00

Absolute alignment requirements may never be applied to node-relative
offsets.  Andreas Herrmann spotted this flaw when a bootmem allocation on
an unaligned node was itself not aligned because the combination of an
unaligned node with an aligned offset into that node is not garuanteed to
be aligned itself.

This patch introduces two helper functions that align a node-relative
index or offset with respect to the node's starting address so that the
absolute PFN or virtual address that results from combining the two
satisfies the requested alignment.

Then all the broken ALIGN()s in alloc_bootmem_core() are replaced by these
helpers.

Signed-off-by: Johannes Weiner 
Reported-by: Andreas Herrmann 
Debugged-by: Andreas Herrmann 
Reviewed-by: Andreas Herrmann 
Tested-by: Andreas Herrmann 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

bootmem allocator: alloc_bootmem_core(): page-align the end offset

2008-08-15T15:35:41+00:00

This is the minimal sequence that jams the allocator:

void *p, *q, *r;
p = alloc_bootmem(PAGE_SIZE);
q = alloc_bootmem(64);
free_bootmem(p, PAGE_SIZE);
p = alloc_bootmem(PAGE_SIZE);
r = alloc_bootmem(64);

after this sequence (assuming that the allocator was empty or page-aligned
before), pointer "q" will be equal to pointer "r".

What's hapenning inside the allocator:
p = alloc_bootmem(PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE, bitmap contains bits 10000...
q = alloc_bootmem(64);
in allocator: last_end_off == PAGE_SIZE + 64, bitmap contains 11000...
free_bootmem(p, PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE + 64, bitmap contains 01000...
p = alloc_bootmem(PAGE_SIZE);
in allocator: last_end_off == PAGE_SIZE, bitmap contains 11000...
r = alloc_bootmem(64);

and now:

it finds bit "2", as a place where to allocate (sidx)

it hits the condition

if (bdata->last_end_off && PFN_DOWN(bdata->last_end_off) + 1 == sidx))
start_off = ALIGN(bdata->last_end_off, align);

-you can see that the condition is true, so it assigns start_off =
ALIGN(bdata->last_end_off, align); (that is PAGE_SIZE) and allocates
over already allocated block.

With the patch it tries to continue at the end of previous allocation only
if the previous allocation ended in the middle of the page.

Signed-off-by: Mikulas Patocka 
Acked-by: Johannes Weiner 
Cc: David Miller 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds