linux-toradex.git/mm/hugetlb.c, branch v4.3.2

mm: hugetlbfs: skip shared VMAs when unmapping private pages to satisfy a fault

2015-10-02T01:42:35+00:00

SunDong reported the following on

  https://bugzilla.kernel.org/show_bug.cgi?id=103841

	I think I find a linux bug, I have the test cases is constructed. I
	can stable recurring problems in fedora22(4.0.4) kernel version,
	arch for x86_64.  I construct transparent huge page, when the parent
	and child process with MAP_SHARE, MAP_PRIVATE way to access the same
	huge page area, it has the opportunity to lead to huge page copy on
	write failure, and then it will munmap the child corresponding mmap
	area, but then the child mmap area with VM_MAYSHARE attributes, child
	process munmap this area can trigger VM_BUG_ON in set_vma_resv_flags
	functions (vma - > vm_flags & VM_MAYSHARE).

There were a number of problems with the report (e.g.  it's hugetlbfs that
triggers this, not transparent huge pages) but it was fundamentally
correct in that a VM_BUG_ON in set_vma_resv_flags() can be triggered that
looks like this

	 vma ffff8804651fd0d0 start 00007fc474e00000 end 00007fc475e00000
	 next ffff8804651fd018 prev ffff8804651fd188 mm ffff88046b1b1800
	 prot 8000000000000027 anon_vma           (null) vm_ops ffffffff8182a7a0
	 pgoff 0 file ffff88106bdb9800 private_data           (null)
	 flags: 0x84400fb(read|write|shared|mayread|maywrite|mayexec|mayshare|dontexpand|hugetlb)
	 ------------
	 kernel BUG at mm/hugetlb.c:462!
	 SMP
	 Modules linked in: xt_pkttype xt_LOG xt_limit [..]
	 CPU: 38 PID: 26839 Comm: map Not tainted 4.0.4-default #1
	 Hardware name: Dell Inc. PowerEdge R810/0TT6JF, BIOS 2.7.4 04/26/2012
	 set_vma_resv_flags+0x2d/0x30

The VM_BUG_ON is correct because private and shared mappings have
different reservation accounting but the warning clearly shows that the
VMA is shared.

When a private COW fails to allocate a new page then only the process
that created the VMA gets the page -- all the children unmap the page.
If the children access that data in the future then they get killed.

The problem is that the same file is mapped shared and private.  During
the COW, the allocation fails, the VMAs are traversed to unmap the other
private pages but a shared VMA is found and the bug is triggered.  This
patch identifies such VMAs and skips them.

Signed-off-by: Mel Gorman 
Reported-by: SunDong 
Reviewed-by: Michal Hocko 
Cc: Andrea Arcangeli 
Cc: Hugh Dickins 
Cc: Naoya Horiguchi 
Cc: David Rientjes 
Reviewed-by: Naoya Horiguchi 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: rename alloc_pages_exact_node() to __alloc_pages_node()

2015-09-08T22:35:28+00:00

alloc_pages_exact_node() was introduced in commit 6484eb3e2a81 ("page
allocator: do not check NUMA node ID when the caller knows the node is
valid") as an optimized variant of alloc_pages_node(), that doesn't
fallback to current node for nid == NUMA_NO_NODE.  Unfortunately the
name of the function can easily suggest that the allocation is
restricted to the given node and fails otherwise.  In truth, the node is
only preferred, unless __GFP_THISNODE is passed among the gfp flags.

The misleading name has lead to mistakes in the past, see for example
commits 5265047ac301 ("mm, thp: really limit transparent hugepage
allocation to local node") and b360edb43f8e ("mm, mempolicy:
migrate_to_node should only migrate to node").

Another issue with the name is that there's a family of
alloc_pages_exact*() functions where 'exact' means exact size (instead
of page order), which leads to more confusion.

To prevent further mistakes, this patch effectively renames
alloc_pages_exact_node() to __alloc_pages_node() to better convey that
it's an optimized variant of alloc_pages_node() not intended for general
usage.  Both functions get described in comments.

It has been also considered to really provide a convenience function for
allocations restricted to a node, but the major opinion seems to be that
__GFP_THISNODE already provides that functionality and we shouldn't
duplicate the API needlessly.  The number of users would be small
anyway.

Existing callers of alloc_pages_exact_node() are simply converted to
call __alloc_pages_node(), with the exception of sba_alloc_coherent()
which open-codes the check for NUMA_NO_NODE, so it is converted to use
alloc_pages_node() instead.  This means it no longer performs some
VM_BUG_ON checks, and since the current check for nid in
alloc_pages_node() uses a 'nid < 0' comparison (which includes
NUMA_NO_NODE), it may hide wrong values which would be previously
exposed.

Both differences will be rectified by the next patch.

To sum up, this patch makes no functional changes, except temporarily
hiding potentially buggy callers.  Restricting the checks in
alloc_pages_node() is left for the next patch which can in turn expose
more existing buggy callers.

Signed-off-by: Vlastimil Babka 
Acked-by: Johannes Weiner 
Acked-by: Robin Holt 
Acked-by: Michal Hocko 
Acked-by: Christoph Lameter 
Acked-by: Michael Ellerman 
Cc: Mel Gorman 
Cc: David Rientjes 
Cc: Greg Thelen 
Cc: Aneesh Kumar K.V 
Cc: Pekka Enberg 
Cc: Joonsoo Kim 
Cc: Naoya Horiguchi 
Cc: Tony Luck 
Cc: Fenghua Yu 
Cc: Arnd Bergmann 
Cc: Benjamin Herrenschmidt 
Cc: Paul Mackerras 
Cc: Gleb Natapov 
Cc: Paolo Bonzini 
Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Cliff Whickman 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

hugetlbfs: add hugetlbfs_fallocate()

2015-09-08T22:35:28+00:00

This is based on the shmem version, but it has diverged quite a bit.  We
have no swap to worry about, nor the new file sealing.  Add
synchronication via the fault mutex table to coordinate page faults,
fallocate allocation and fallocate hole punch.

What this allows us to do is move physical memory in and out of a
hugetlbfs file without having it mapped.  This also gives us the ability
to support MADV_REMOVE since it is currently implemented using
fallocate().  MADV_REMOVE lets madvise() remove pages from the middle of
a hugetlbfs file, which wasn't possible before.

hugetlbfs fallocate only operates on whole huge pages.

Based on code by Dave Hansen.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

hugetlbfs: New huge_add_to_page_cache helper routine

2015-09-08T22:35:28+00:00

Currently, there is only a single place where hugetlbfs pages are added
to the page cache.  The new fallocate code be adding a second one, so
break the functionality out into its own helper.

Signed-off-by: Dave Hansen 
Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hugetlb: alloc_huge_page handle areas hole punched by fallocate

2015-09-08T22:35:28+00:00

Areas hole punched by fallocate will not have entries in the
region/reserve map.  However, shared mappings with min_size subpool
reservations may still have reserved pages.  alloc_huge_page needs to
handle this special case and do the proper accounting.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hugetlb: vma_has_reserves() needs to handle fallocate hole punch

2015-09-08T22:35:28+00:00

In vma_has_reserves(), the current assumption is that reserves are
always present for shared mappings.  However, this will not be the case
with fallocate hole punch.  When punching a hole, the present page will
be deleted as well as the region/reserve map entry (and hence any
reservation).  vma_has_reserves is passed "chg" which indicates whether
or not a region/reserve map is present.  Use this to determine if
reserves are actually present or were removed via hole punch.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

hugetlbfs: truncate_hugepages() takes a range of pages

2015-09-08T22:35:28+00:00

Modify truncate_hugepages() to take a range of pages (start, end)
instead of simply start.  If an end value of LLONG_MAX is passed, the
current "truncate" functionality is maintained.  Existing callers are
modified to pass LLONG_MAX as end of range.  By keying off end ==
LLONG_MAX, the routine behaves differently for truncate and hole punch.
Page removal is now synchronized with page allocation via faults by
using the fault mutex table.  The hole punch case can experience the
rare region_del error and must handle accordingly.

Add the routine hugetlb_fix_reserve_counts to fix up reserve counts in
the case where region_del returns an error.

Since the routine handles more than just the truncate case, it is
renamed to remove_inode_hugepages().  To be consistent, the routine
truncate_huge_page() is renamed remove_huge_page().

Downstream of remove_inode_hugepages(), the routine
hugetlb_unreserve_pages() is also modified to take a range of pages.
hugetlb_unreserve_pages is modified to detect an error from region_del and
pass it back to the caller.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hugetlb: expose hugetlb fault mutex for use by fallocate

2015-09-08T22:35:28+00:00

hugetlb page faults are currently synchronized by the table of mutexes
(htlb_fault_mutex_table).  fallocate code will need to synchronize with
the page fault code when it allocates or deletes pages.  Expose
interfaces so that fallocate operations can be synchronized with page
faults.  Minor name changes to be more consistent with other global
hugetlb symbols.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hugetlb: add region_del() to delete a specific range of entries

2015-09-08T22:35:28+00:00

fallocate hole punch will want to remove a specific range of pages.  The
existing region_truncate() routine deletes all region/reserve map
entries after a specified offset.  region_del() will provide this same
functionality if the end of region is specified as LONG_MAX.  Hence,
region_del() can replace region_truncate().

Unlike region_truncate(), region_del() can return an error in the rare
case where it can not allocate memory for a region descriptor.  This
ONLY happens in the case where an existing region must be split.
Current callers passing LONG_MAX as end of range will never experience
this error and do not need to deal with error handling.  Future callers
of region_del() (such as fallocate hole punch) will need to handle this
error.

Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/hugetlb: add cache of descriptors to resv_map for region_add

2015-09-08T22:35:28+00:00

hugetlbfs is used today by applications that want a high degree of
control over huge page usage.  Often, large hugetlbfs files are used to
map a large number huge pages into the application processes.  The
applications know when page ranges within these large files will no
longer be used, and ideally would like to release them back to the
subpool or global pools for other uses.  The fallocate() system call
provides an interface for preallocation and hole punching within files.
This patch set adds fallocate functionality to hugetlbfs.

fallocate hole punch will want to remove a specific range of pages.
When pages are removed, their associated entries in the region/reserve
map will also be removed.  This will break an assumption in the
region_chg/region_add calling sequence.  If a new region descriptor must
be allocated, it is done as part of the region_chg processing.  In this
way, region_add can not fail because it does not need to attempt an
allocation.

To prepare for fallocate hole punch, create a "cache" of descriptors
that can be used by region_add if necessary.  region_chg will ensure
there are sufficient entries in the cache.  It will be necessary to
track the number of in progress add operations to know a sufficient
number of descriptors reside in the cache.  A new routine region_abort
is added to adjust this in progress count when add operations are
aborted.  vma_abort_reservation is also added for callers creating
reservations with vma_needs_reservation/vma_commit_reservation.

[akpm@linux-foundation.org: fix typo in comment, use more cols]
Signed-off-by: Mike Kravetz 
Reviewed-by: Naoya Horiguchi 
Acked-by: Hillf Danton 
Cc: Dave Hansen 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: Davidlohr Bueso 
Cc: Aneesh Kumar 
Cc: Christoph Hellwig 
Cc: Michal Hocko 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds