linux-toradex.git/mm/page_isolation.c, branch v3.10.17

mm: fix zone_watermark_ok_safe() accounting of isolated pages

2013-01-05T00:11:46+00:00

Commit 702d1a6e0766 ("memory-hotplug: fix kswapd looping forever
problem") added an isolated pageblocks counter (nr_pageblock_isolate in
struct zone) and used it to adjust free pages counter in
zone_watermark_ok_safe() to prevent kswapd looping forever problem.

Then later, commit 2139cbe627b8 ("cma: fix counting of isolated pages")
fixed accounting of isolated pages in global free pages counter.  It
made the previous zone_watermark_ok_safe() fix unnecessary and
potentially harmful (cause now isolated pages may be accounted twice
making free pages counter incorrect).

This patch removes the special isolated pageblocks counter altogether
which fixes zone_watermark_ok_safe() free pages check.

Reported-by: Tomasz Stanislawski 
Signed-off-by: Bartlomiej Zolnierkiewicz 
Signed-off-by: Kyungmin Park 
Cc: Minchan Kim 
Cc: KOSAKI Motohiro 
Cc: Aaditya Kumar 
Cc: KAMEZAWA Hiroyuki 
Cc: Mel Gorman 
Cc: Michal Hocko 
Cc: Marek Szyprowski 
Cc: Michal Nazarewicz 
Cc: Hugh Dickins 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: skip HWPoisoned page when offlining pages

2012-12-12T01:22:22+00:00

hwpoisoned may be set when we offline a page by the sysfs interface
/sys/devices/system/memory/soft_offline_page or
/sys/devices/system/memory/hard_offline_page. If we don't clear
this flag when onlining pages, this page can't be freed, and will
not in free list. So we can't offline these pages again. So we
should skip such page when offlining pages.

Signed-off-by: Wen Congyang 
Cc: David Rientjes 
Cc: Jiang Liu 
Cc: Len Brown 
Cc: Benjamin Herrenschmidt 
Cc: Paul Mackerras 
Cc: Christoph Lameter 
Cc: Minchan Kim 
Cc: KOSAKI Motohiro 
Cc: Yasuaki Ishimatsu 
Cc: Andi Kleen 
Cc: Mel Gorman 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/page_alloc: refactor out __alloc_contig_migrate_alloc()

2012-10-09T07:22:52+00:00

__alloc_contig_migrate_alloc() can be used by memory-hotplug so refactor
it out (move + rename as a common name) into page_isolation.c.

[akpm@linux-foundation.org: checkpatch fixes]
Signed-off-by: Minchan Kim 
Cc: Kamezawa Hiroyuki 
Reviewed-by: Yasuaki Ishimatsu 
Acked-by: Michal Nazarewicz 
Cc: Marek Szyprowski 
Cc: Wen Congyang 
Acked-by: David Rientjes 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: fix pages missed by race rather than failing

2012-10-09T07:22:46+00:00

If race between allocation and isolation in memory-hotplug offline
happens, some pages could be in MIGRATE_MOVABLE of free_list although the
pageblock's migratetype of the page is MIGRATE_ISOLATE.

The race could be detected by get_freepage_migratetype in
__test_page_isolated_in_pageblock.  If it is detected, now EBUSY gets
bubbled all the way up and the hotplug operations fails.

But better idea is instead of returning and failing memory-hotremove, move
the free page to the correct list at the time it is detected.  It could
enhance memory-hotremove operation success ratio although the race is
really rare.

Suggested by Mel Gorman.

[akpm@linux-foundation.org: small cleanup]
Signed-off-by: Minchan Kim 
Cc: KAMEZAWA Hiroyuki 
Reviewed-by: Yasuaki Ishimatsu 
Acked-by: Mel Gorman 
Cc: Xishi Qiu 
Cc: Wen Congyang 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: bug fix race between isolation and allocation

2012-10-09T07:22:46+00:00

Like below, memory-hotplug makes race between page-isolation
and page-allocation so it can hit BUG_ON in __offline_isolated_pages.

	CPU A					CPU B

start_isolate_page_range
set_migratetype_isolate
spin_lock_irqsave(zone->lock)

				free_hot_cold_page(Page A)
				/* without zone->lock */
				migratetype = get_pageblock_migratetype(Page A);
				/*
				 * Page could be moved into MIGRATE_MOVABLE
				 * of per_cpu_pages
				 */
				list_add_tail(&page->lru, &pcp->lists[migratetype]);

set_pageblock_isolate
move_freepages_block
drain_all_pages

				/* Page A could be in MIGRATE_MOVABLE of free_list. */

check_pages_isolated
__test_page_isolated_in_pageblock
/*
 * We can't catch freed page which
 * is free_list[MIGRATE_MOVABLE]
 */
if (PageBuddy(page A))
	pfn += 1 << page_order(page A);

				/* So, Page A could be allocated */

__offline_isolated_pages
/*
 * BUG_ON hit or offline page
 * which is used by someone
 */
BUG_ON(!PageBuddy(page A));

This patch checks page's migratetype in freelist in
__test_page_isolated_in_pageblock.  So now
__test_page_isolated_in_pageblock can check the page caused by above race
and can fail of memory offlining.

Signed-off-by: Minchan Kim 
Acked-by: KAMEZAWA Hiroyuki 
Reviewed-by: Yasuaki Ishimatsu 
Acked-by: Mel Gorman 
Cc: Xishi Qiu 
Cc: Wen Congyang 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: page_alloc: use get_freepage_migratetype() instead of page_private()

2012-10-09T07:22:45+00:00

The page allocator uses set_page_private and page_private for handling
migratetype when it frees page.  Let's replace them with [set|get]
_freepage_migratetype to make it more clear.

Signed-off-by: Minchan Kim 
Acked-by: KAMEZAWA Hiroyuki 
Reviewed-by: Yasuaki Ishimatsu 
Acked-by: Mel Gorman 
Cc: Xishi Qiu 
Cc: Wen Congyang 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

cma: count free CMA pages

2012-10-09T07:22:44+00:00

Add NR_FREE_CMA_PAGES counter to be later used for checking watermark in
__zone_watermark_ok().  For simplicity and to avoid #ifdef hell make this
counter always available (not only when CONFIG_CMA=y).

[akpm@linux-foundation.org: use conventional migratetype naming]
Signed-off-by: Bartlomiej Zolnierkiewicz 
Signed-off-by: Kyungmin Park 
Cc: Marek Szyprowski 
Cc: Michal Nazarewicz 
Cc: Minchan Kim 
Cc: Mel Gorman 
Cc: Hugh Dickins 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

cma: fix counting of isolated pages

2012-10-09T07:22:44+00:00

Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.  Fix
it by properly decreasing/increasing NR_FREE_PAGES counter in
set_migratetype_isolate()/unset_migratetype_isolate() and removing counter
adjustment for isolated pages from free_one_page() and split_free_page().

Signed-off-by: Bartlomiej Zolnierkiewicz 
Signed-off-by: Kyungmin Park 
Cc: Marek Szyprowski 
Cc: Michal Nazarewicz 
Cc: Minchan Kim 
Cc: Mel Gorman 
Cc: Hugh Dickins 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: fix kswapd looping forever problem

2012-08-01T01:42:45+00:00

When hotplug offlining happens on zone A, it starts to mark freed page as
MIGRATE_ISOLATE type in buddy for preventing further allocation.
(MIGRATE_ISOLATE is very irony type because it's apparently on buddy but
we can't allocate them).

When the memory shortage happens during hotplug offlining, current task
starts to reclaim, then wake up kswapd.  Kswapd checks watermark, then go
sleep because current zone_watermark_ok_safe doesn't consider
MIGRATE_ISOLATE freed page count.  Current task continue to reclaim in
direct reclaim path without kswapd's helping.  The problem is that
zone->all_unreclaimable is set by only kswapd so that current task would
be looping forever like below.

__alloc_pages_slowpath
restart:
	wake_all_kswapd
rebalance:
	__alloc_pages_direct_reclaim
		do_try_to_free_pages
			if global_reclaim && !all_unreclaimable
				return 1; /* It means we did did_some_progress */
	skip __alloc_pages_may_oom
	should_alloc_retry
		goto rebalance;

If we apply KOSAKI's patch[1] which doesn't depends on kswapd about
setting zone->all_unreclaimable, we can solve this problem by killing some
task in direct reclaim path.  But it doesn't wake up kswapd, still.  It
could be a problem still if other subsystem needs GFP_ATOMIC request.  So
kswapd should consider MIGRATE_ISOLATE when it calculate free pages BEFORE
going sleep.

This patch counts the number of MIGRATE_ISOLATE page block and
zone_watermark_ok_safe will consider it if the system has such blocks
(fortunately, it's very rare so no problem in POV overhead and kswapd is
never hotpath).

Copy/modify from Mel's quote
"
Ideal solution would be "allocating" the pageblock.
It would keep the free space accounting as it is but historically,
memory hotplug didn't allocate pages because it would be difficult to
detect if a pageblock was isolated or if part of some balloon.
Allocating just full pageblocks would work around this, However,
it would play very badly with CMA.
"

[1] http://lkml.org/lkml/2012/6/14/74

[akpm@linux-foundation.org: simplify nr_zone_isolate_freepages(), rework zone_watermark_ok_safe() comment, simplify set_pageblock_isolate() and restore_pageblock_isolate()]
[akpm@linux-foundation.org: fix CONFIG_MEMORY_ISOLATION=n build]
Signed-off-by: Minchan Kim 
Suggested-by: KOSAKI Motohiro 
Tested-by: Aaditya Kumar 
Cc: KAMEZAWA Hiroyuki 
Cc: Mel Gorman 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: factor out memory isolate functions

2012-08-01T01:42:45+00:00

mm/page_alloc.c has some memory isolation functions but they are used only
when we enable CONFIG_{CMA|MEMORY_HOTPLUG|MEMORY_FAILURE}.  So let's make
it configurable by new CONFIG_MEMORY_ISOLATION so that it can reduce
binary size and we can check it simple by CONFIG_MEMORY_ISOLATION, not if
defined CONFIG_{CMA|MEMORY_HOTPLUG|MEMORY_FAILURE}.

Signed-off-by: Minchan Kim 
Cc: Andi Kleen 
Cc: Marek Szyprowski 
Acked-by: KAMEZAWA Hiroyuki 
Cc: KOSAKI Motohiro 
Cc: Mel Gorman 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds