linux-toradex.git/mm/memory_hotplug.c, branch v3.7

revert "mm: fix-up zone present pages"

2012-11-16T22:33:04+00:00

Revert commit 7f1290f2f2a4 ("mm: fix-up zone present pages")

That patch tried to fix a issue when calculating zone->present_pages,
but it caused a regression on 32bit systems with HIGHMEM.  With that
change, reset_zone_present_pages() resets all zone->present_pages to
zero, and fixup_zone_present_pages() is called to recalculate
zone->present_pages when the boot allocator frees core memory pages into
buddy allocator.  Because highmem pages are not freed by bootmem
allocator, all highmem zones' present_pages becomes zero.

Various options for improving the situation are being discussed but for
now, let's return to the 3.6 code.

Cc: Jianguo Wu 
Cc: Jiang Liu 
Cc: Petr Tesarik 
Cc: "Luck, Tony" 
Cc: Mel Gorman 
Cc: Yinghai Lu 
Cc: Minchan Kim 
Cc: Johannes Weiner 
Acked-by: David Rientjes 
Tested-by: Chris Clayton 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: suppress "Trying to free nonexistent resource " warning

2012-10-09T07:23:04+00:00

When our x86 box calls __remove_pages(), release_mem_region() shows many
warnings.  And x86 box cannot unregister iomem_resource.

  "Trying to free nonexistent resource "

release_mem_region() has been changed to be called in each
PAGES_PER_SECTION by commit de7f0cba9678 ("memory hotplug: release
memory regions in PAGES_PER_SECTION chunks").  Because powerpc registers
iomem_resource in each PAGES_PER_SECTION chunk.  But when I hot add
memory on x86 box, iomem_resource is register in each _CRS not
PAGES_PER_SECTION chunk.  So x86 box unregisters iomem_resource.

The patch fixes the problem.

Signed-off-by: Yasuaki Ishimatsu 
Cc: David Rientjes 
Cc: Jiang Liu 
Cc: Len Brown 
Cc: Benjamin Herrenschmidt 
Cc: Paul Mackerras 
Cc: Christoph Lameter 
Cc: Minchan Kim 
Cc: Andrew Morton 
Cc: KOSAKI Motohiro 
Cc: Wen Congyang 
Cc: Dave Hansen 
Cc: Nathan Fontenot 
Cc: Badari Pulavarty 
Cc: Yasunori Goto 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: update memory block's state and notify userspace

2012-10-09T07:23:02+00:00

remove_memory() will be called when hot removing a memory device.  But
even if offlining memory, we cannot notice it.  So the patch updates the
memory block's state and sends notification to userspace.

Additionally, the memory device may contain more than one memory block.
If the memory block has been offlined, __offline_pages() will fail.  So we
should try to offline one memory block at a time.

Thus remove_memory() also check each memory block's state.  So there is no
need to check the memory block's state before calling remove_memory().

Signed-off-by: Wen Congyang 
Signed-off-by: Yasuaki Ishimatsu 
Cc: David Rientjes 
Cc: Jiang Liu 
Cc: Len Brown 
Cc: Christoph Lameter 
Cc: Minchan Kim 
Cc: KOSAKI Motohiro 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: preparation to notify memory block's state at memory hot remove

2012-10-09T07:23:02+00:00

remove_memory() is called in two cases:
1. echo offline >/sys/devices/system/memory/memoryXX/state
2. hot remove a memory device

In the 1st case, the memory block's state is changed and the notification
that memory block's state changed is sent to userland after calling
remove_memory().  So user can notice memory block is changed.

But in the 2nd case, the memory block's state is not changed and the
notification is not also sent to userspcae even if calling
remove_memory().  So user cannot notice memory block is changed.

For adding the notification at memory hot remove, the patch just prepare
as follows:
1st case uses offline_pages() for offlining memory.
2nd case uses remove_memory() for offlining memory and changing memory block's
    state and notifing the information.

The patch does not implement notification to remove_memory().

Signed-off-by: Wen Congyang 
Signed-off-by: Yasuaki Ishimatsu 
Cc: David Rientjes 
Cc: Jiang Liu 
Cc: Len Brown 
Cc: Christoph Lameter 
Cc: Minchan Kim 
Cc: KOSAKI Motohiro 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: fix-up zone present pages

2012-10-09T07:22:54+00:00

I think zone->present_pages indicates pages that buddy system can management,
it should be:

	zone->present_pages = spanned pages - absent pages - bootmem pages,

but is now:
	zone->present_pages = spanned pages - absent pages - memmap pages.

spanned pages: total size, including holes.
absent pages: holes.
bootmem pages: pages used in system boot, managed by bootmem allocator.
memmap pages: pages used by page structs.

This may cause zone->present_pages less than it should be.  For example,
numa node 1 has ZONE_NORMAL and ZONE_MOVABLE, it's memmap and other
bootmem will be allocated from ZONE_MOVABLE, so ZONE_NORMAL's
present_pages should be spanned pages - absent pages, but now it also
minus memmap pages(free_area_init_core), which are actually allocated from
ZONE_MOVABLE.  When offlining all memory of a zone, this will cause
zone->present_pages less than 0, because present_pages is unsigned long
type, it is actually a very large integer, it indirectly caused
zone->watermark[WMARK_MIN] becomes a large
integer(setup_per_zone_wmarks()), than cause totalreserve_pages become a
large integer(calculate_totalreserve_pages()), and finally cause memory
allocating failure when fork process(__vm_enough_memory()).

[root@localhost ~]# dmesg
-bash: fork: Cannot allocate memory

I think the bug described in

  http://marc.info/?l=linux-mm&m=134502182714186&w=2

is also caused by wrong zone present pages.

This patch intends to fix-up zone->present_pages when memory are freed to
buddy system on x86_64 and IA64 platforms.

Signed-off-by: Jianguo Wu 
Signed-off-by: Jiang Liu 
Reported-by: Petr Tesarik 
Tested-by: Petr Tesarik 
Cc: "Luck, Tony" 
Cc: Mel Gorman 
Cc: Yinghai Lu 
Cc: Minchan Kim 
Cc: Johannes Weiner 
Cc: David Rientjes 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: don't replace lowmem pages with highmem

2012-10-09T07:22:52+00:00

The changelog for commit 6a6dccba2fdc ("mm: cma: don't replace lowmem
pages with highmem") mentioned that lowmem pages can be replaced by
highmem pages during CMA migration.  6a6dccba2fdc fixed that issue.

Quote from that changelog:

:   The filesystem layer expects pages in the block device's mapping to not
:   be in highmem (the mapping's gfp mask is set in bdget()), but CMA can
:   currently replace lowmem pages with highmem pages, leading to crashes in
:   filesystem code such as the one below:
:
:     Unable to handle kernel NULL pointer dereference at virtual address 00000400
:     pgd = c0c98000
:     [00000400] *pgd=00c91831, *pte=00000000, *ppte=00000000
:     Internal error: Oops: 817 [#1] PREEMPT SMP ARM
:     CPU: 0    Not tainted  (3.5.0-rc5+ #80)
:     PC is at __memzero+0x24/0x80
:     ...
:     Process fsstress (pid: 323, stack limit = 0xc0cbc2f0)
:     Backtrace:
:     [] (ext4_getblk+0x0/0x180) from [] (ext4_bread+0x1c/0x98)
:     [] (ext4_bread+0x0/0x98) from [] (ext4_mkdir+0x160/0x3bc)
:      r4:c15337f0
:     [] (ext4_mkdir+0x0/0x3bc) from [] (vfs_mkdir+0x8c/0x98)
:     [] (vfs_mkdir+0x0/0x98) from [] (sys_mkdirat+0x74/0xac)
:      r6:00000000 r5:c152eb40 r4:000001ff r3:c14b43f0
:     [] (sys_mkdirat+0x0/0xac) from [] (sys_mkdir+0x20/0x24)
:      r6:beccdcf0 r5:00074000 r4:beccdbbc
:     [] (sys_mkdir+0x0/0x24) from [] (ret_fast_syscall+0x0/0x30)

Memory-hotplug has same problem as CMA has so the same fix can be applied
to memory-hotplug as well.

Fix it by reusing.

Signed-off-by: Minchan Kim 
Cc: Kamezawa Hiroyuki 
Reviewed-by: Yasuaki Ishimatsu 
Acked-by: Michal Nazarewicz 
Cc: Marek Szyprowski 
Cc: Wen Congyang 
Acked-by: David Rientjes 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory-hotplug: build zonelists when offlining pages

2012-10-09T07:22:43+00:00

online_pages() does build_all_zonelists() and zone_pcp_update(), I think
offline_pages() should do it too.

When the zone has no memory to allocate, remove it from other nodes'
zonelists.  zone_batchsize() depends on zone's present pages, if zone's
present pages are changed, zone's pcp should be updated.

Signed-off-by: Xishi Qiu 
Cc: Yasuaki Ishimatsu 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

memory hotplug: fix section info double registration bug

2012-09-17T22:00:38+00:00

There may be a bug when registering section info.  For example, on my
Itanium platform, the pfn range of node0 includes the other nodes, so
other nodes' section info will be double registered, and memmap's page
count will equal to 3.

  node0: start_pfn=0x100,    spanned_pfn=0x20fb00, present_pfn=0x7f8a3, => 0x000100-0x20fc00
  node1: start_pfn=0x80000,  spanned_pfn=0x80000,  present_pfn=0x80000, => 0x080000-0x100000
  node2: start_pfn=0x100000, spanned_pfn=0x80000,  present_pfn=0x80000, => 0x100000-0x180000
  node3: start_pfn=0x180000, spanned_pfn=0x80000,  present_pfn=0x80000, => 0x180000-0x200000

  free_all_bootmem_node()
	register_page_bootmem_info_node()
		register_page_bootmem_info_section()

When hot remove memory, we can't free the memmap's page because
page_count() is 2 after put_page_bootmem().

  sparse_remove_one_section()
	free_section_usemap()
		free_map_bootmem()
			put_page_bootmem()

[akpm@linux-foundation.org: add code comment]
Signed-off-by: Xishi Qiu 
Signed-off-by: Jiang Liu 
Acked-by: Mel Gorman 
Cc: "Luck, Tony" 
Cc: Yasuaki Ishimatsu 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hotplug: free zone->pageset when a zone becomes empty

2012-08-01T01:42:44+00:00

When a zone becomes empty after memory offlining, free zone->pageset.
Otherwise it will cause memory leak when adding memory to the empty zone
again because build_all_zonelists() will allocate zone->pageset for an
empty zone.

Signed-off-by: Jiang Liu 
Signed-off-by: Wei Wang 
Cc: Mel Gorman 
Cc: Michal Hocko 
Cc: Minchan Kim 
Cc: Rusty Russell 
Cc: Yinghai Lu 
Cc: Tony Luck 
Cc: KAMEZAWA Hiroyuki 
Cc: KOSAKI Motohiro 
Cc: David Rientjes 
Cc: Keping Chen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hotplug: correctly add new zone to all other nodes' zone lists

2012-08-01T01:42:44+00:00

When online_pages() is called to add new memory to an empty zone, it
rebuilds all zone lists by calling build_all_zonelists().  But there's a
bug which prevents the new zone to be added to other nodes' zone lists.

online_pages() {
	build_all_zonelists()
	.....
	node_set_state(zone_to_nid(zone), N_HIGH_MEMORY)
}

Here the node of the zone is put into N_HIGH_MEMORY state after calling
build_all_zonelists(), but build_all_zonelists() only adds zones from
nodes in N_HIGH_MEMORY state to the fallback zone lists.
build_all_zonelists()

    ->__build_all_zonelists()
	->build_zonelists()
	    ->find_next_best_node()
		->for_each_node_state(n, N_HIGH_MEMORY)

So memory in the new zone will never be used by other nodes, and it may
cause strange behavor when system is under memory pressure.  So put node
into N_HIGH_MEMORY state before calling build_all_zonelists().

Signed-off-by: Jianguo Wu 
Signed-off-by: Jiang Liu 
Cc: Mel Gorman 
Cc: Michal Hocko 
Cc: Minchan Kim 
Cc: Rusty Russell 
Cc: Yinghai Lu 
Cc: Tony Luck 
Cc: KAMEZAWA Hiroyuki 
Cc: KOSAKI Motohiro 
Cc: David Rientjes 
Cc: Keping Chen 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds