linux-toradex.git/mm/mlock.c, branch v2.6.28.7

mm: fix error case in mlock downgrade reversion

2009-02-12T17:50:36+00:00

commit d5b562330ec766292a3ac54ae5e0673610bd5b3d upstream.

Commit 27421e211a39784694b597dbf35848b88363c248, Manually revert
"mlock: downgrade mmap sem while populating mlocked regions", has
introduced its own regression: __mlock_vma_pages_range() may report
an error (for example, -EFAULT from trying to lock down pages from
beyond EOF), but mlock_vma_pages_range() must hide that from its
callers as before.

Reported-by: Sami Farin 
Signed-off-by: Hugh Dickins 
Signed-off-by: Linus Torvalds

Manually revert "mlock: downgrade mmap sem while populating mlocked regions"

2009-02-06T21:47:16+00:00

commit 27421e211a39784694b597dbf35848b88363c248 upstream.

This essentially reverts commit 8edb08caf68184fb170f4f69c7445929e199eaea.

It downgraded our mmap semaphore to a read-lock while mlocking pages, in
order to allow other threads (and external accesses like "ps" et al) to
walk the vma lists and take page faults etc.  Which is a nice idea, but
the implementation does not work.

Because we cannot upgrade the lock back to a write lock without
releasing the mmap semaphore, the code had to release the lock entirely
and then re-take it as a writelock.  However, that meant that the caller
possibly lost the vma chain that it was following, since now another
thread could come in and mmap/munmap the range.

The code tried to work around that by just looking up the vma again and
erroring out if that happened, but quite frankly, that was just a buggy
hack that doesn't actually protect against anything (the other thread
could just have replaced the vma with another one instead of totally
unmapping it).

The only way to downgrade to a read map _reliably_ is to do it at the
end, which is likely the right thing to do: do all the 'vma' operations
with the write-lock held, then downgrade to a read after completing them
all, and then do the "populate the newly mlocked regions" while holding
just the read lock.  And then just drop the read-lock and return to user
space.

The (perhaps somewhat simpler) alternative is to just make all the
callers of mlock_vma_pages_range() know that the mmap lock got dropped,
and just re-grab the mmap semaphore if it needs to mlock more than one
vma region.

So we can do this "downgrade mmap sem while populating mlocked regions"
thing right, but the way it was done here was absolutely not correct.
Thus the revert, in the expectation that we will do it all correctly
some day.

Cc: Lee Schermerhorn 
Cc: Rik van Riel 
Cc: Andrew Morton 
Signed-off-by: Linus Torvalds 
Signed-off-by: Greg Kroah-Hartman

System call wrappers part 14

2009-01-18T18:43:56+00:00

commit 3480b25743cb7404928d57efeaa3d085708b04c2 upstream.

Signed-off-by: Heiko Carstens 
Signed-off-by: Greg Kroah-Hartman

System call wrappers part 13

2009-01-18T18:43:56+00:00

commit 6a6160a7b5c27b3c38651baef92a14fa7072b3c1 upstream.

Signed-off-by: Heiko Carstens 
Signed-off-by: Greg Kroah-Hartman

unitialized return value in mm/mlock.c: __mlock_vma_pages_range()

2008-11-16T23:55:36+00:00

Fix an unitialized return value when compiling on parisc (with CONFIG_UNEVICTABLE_LRU=y):
	mm/mlock.c: In function `__mlock_vma_pages_range':
	mm/mlock.c:165: warning: `ret' might be used uninitialized in this function

Signed-off-by: Helge Deller 
[ It isn't ever really used uninitialized, since no caller should ever
  call this function with an empty range.  But the compiler is correct
  that from a local analysis standpoint that is impossible to see, and
  fixing the warning is appropriate.  ]
Signed-off-by: Linus Torvalds

mm: remove lru_add_drain_all() from the munlock path

2008-11-13T01:17:16+00:00

lockdep warns about following message at boot time on one of my test
machine.  Then, schedule_on_each_cpu() sholdn't be called when the task
have mmap_sem.

Actually, lru_add_drain_all() exist to prevent the unevictalble pages
stay on reclaimable lru list.  but currenct unevictable code can rescue
unevictable pages although it stay on reclaimable list.

So removing is better.

In addition, this patch add lru_add_drain_all() to sys_mlock() and
sys_mlockall().  it isn't must.  but it reduce the failure of moving to
unevictable list.  its failure can rescue in vmscan later.  but reducing
is better.

Note, if above rescuing happend, the Mlocked and the Unevictable field
mismatching happend in /proc/meminfo.  but it doesn't cause any real
trouble.

=======================================================
[ INFO: possible circular locking dependency detected ]
2.6.28-rc2-mm1 #2
-------------------------------------------------------
lvm/1103 is trying to acquire lock:
 (&cpu_hotplug.lock){--..}, at: [] get_online_cpus+0x29/0x50

but task is already holding lock:
 (&mm->mmap_sem){----}, at: [] sys_mlockall+0x4e/0xb0

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #3 (&mm->mmap_sem){----}:
       [] check_noncircular+0x82/0x110
       [] might_fault+0x4a/0xa0
       [] validate_chain+0xb11/0x1070
       [] might_fault+0x4a/0xa0
       [] __lock_acquire+0x263/0xa10
       [] lock_acquire+0x7c/0xb0			(*) grab mmap_sem
       [] might_fault+0x4a/0xa0
       [] might_fault+0x7b/0xa0
       [] might_fault+0x4a/0xa0
       [] copy_to_user+0x30/0x60
       [] filldir+0x7c/0xd0
       [] sysfs_readdir+0x11a/0x1f0			(*) grab sysfs_mutex
       [] filldir+0x0/0xd0
       [] filldir+0x0/0xd0
       [] vfs_readdir+0x86/0xa0			(*) grab i_mutex
       [] sys_getdents+0x6b/0xc0
       [] syscall_call+0x7/0xb
       [] 0xffffffff

-> #2 (sysfs_mutex){--..}:
       [] check_noncircular+0x82/0x110
       [] sysfs_addrm_start+0x2c/0xc0
       [] validate_chain+0xb11/0x1070
       [] sysfs_addrm_start+0x2c/0xc0
       [] __lock_acquire+0x263/0xa10
       [] lock_acquire+0x7c/0xb0			(*) grab sysfs_mutex
       [] sysfs_addrm_start+0x2c/0xc0
       [] mutex_lock_nested+0xa5/0x2f0
       [] sysfs_addrm_start+0x2c/0xc0
       [] sysfs_addrm_start+0x2c/0xc0
       [] sysfs_addrm_start+0x2c/0xc0
       [] create_dir+0x3f/0x90
       [] sysfs_create_dir+0x29/0x50
       [] _spin_unlock+0x25/0x40
       [] kobject_add_internal+0xcd/0x1a0
       [] kobject_set_name_vargs+0x3a/0x50
       [] kobject_init_and_add+0x2d/0x40
       [] sysfs_slab_add+0xd2/0x180
       [] sysfs_add_func+0x0/0x70
       [] sysfs_add_func+0x5c/0x70			(*) grab slub_lock
       [] run_workqueue+0x172/0x200
       [] run_workqueue+0x10f/0x200
       [] worker_thread+0x0/0xf0
       [] worker_thread+0x9c/0xf0
       [] autoremove_wake_function+0x0/0x50
       [] worker_thread+0x0/0xf0
       [] kthread+0x42/0x70
       [] kthread+0x0/0x70
       [] kernel_thread_helper+0x7/0x1c
       [] 0xffffffff

-> #1 (slub_lock){----}:
       [] check_noncircular+0xd/0x110
       [] slab_cpuup_callback+0x11f/0x1d0
       [] validate_chain+0xb11/0x1070
       [] slab_cpuup_callback+0x11f/0x1d0
       [] mark_lock+0x35d/0xd00
       [] __lock_acquire+0x263/0xa10
       [] lock_acquire+0x7c/0xb0
       [] slab_cpuup_callback+0x11f/0x1d0
       [] down_read+0x43/0x80
       [] slab_cpuup_callback+0x11f/0x1d0		(*) grab slub_lock
       [] slab_cpuup_callback+0x11f/0x1d0
       [] notifier_call_chain+0x3c/0x70
       [] _cpu_up+0x84/0x110
       [] cpu_up+0x4b/0x70				(*) grab cpu_hotplug.lock
       [] kernel_init+0x0/0x170
       [] kernel_init+0xb5/0x170
       [] kernel_init+0x0/0x170
       [] kernel_thread_helper+0x7/0x1c
       [] 0xffffffff

-> #0 (&cpu_hotplug.lock){--..}:
       [] validate_chain+0x5af/0x1070
       [] dev_status+0x0/0x50
       [] __lock_acquire+0x263/0xa10
       [] lock_acquire+0x7c/0xb0
       [] get_online_cpus+0x29/0x50
       [] mutex_lock_nested+0xa5/0x2f0
       [] get_online_cpus+0x29/0x50
       [] get_online_cpus+0x29/0x50
       [] lru_add_drain_per_cpu+0x0/0x10
       [] get_online_cpus+0x29/0x50			(*) grab cpu_hotplug.lock
       [] schedule_on_each_cpu+0x32/0xe0
       [] __mlock_vma_pages_range+0x85/0x2c0
       [] __lock_acquire+0x285/0xa10
       [] vma_merge+0xa9/0x1d0
       [] mlock_fixup+0x180/0x200
       [] do_mlockall+0x78/0x90			(*) grab mmap_sem
       [] sys_mlockall+0x81/0xb0
       [] syscall_call+0x7/0xb
       [] 0xffffffff

other info that might help us debug this:

1 lock held by lvm/1103:
 #0:  (&mm->mmap_sem){----}, at: [] sys_mlockall+0x4e/0xb0

stack backtrace:
Pid: 1103, comm: lvm Not tainted 2.6.28-rc2-mm1 #2
Call Trace:
 [] print_circular_bug_tail+0x7c/0xd0
 [] validate_chain+0x5af/0x1070
 [] dev_status+0x0/0x50
 [] __lock_acquire+0x263/0xa10
 [] lock_acquire+0x7c/0xb0
 [] get_online_cpus+0x29/0x50
 [] mutex_lock_nested+0xa5/0x2f0
 [] get_online_cpus+0x29/0x50
 [] get_online_cpus+0x29/0x50
 [] lru_add_drain_per_cpu+0x0/0x10
 [] get_online_cpus+0x29/0x50
 [] schedule_on_each_cpu+0x32/0xe0
 [] __mlock_vma_pages_range+0x85/0x2c0
 [] __lock_acquire+0x285/0xa10
 [] vma_merge+0xa9/0x1d0
 [] mlock_fixup+0x180/0x200
 [] do_mlockall+0x78/0x90
 [] sys_mlockall+0x81/0xb0
 [] syscall_call+0x7/0xb

Signed-off-by: KOSAKI Motohiro 
Tested-by: Kamalesh Babulal 
Cc: Lee Schermerhorn 
Cc: Christoph Lameter 
Cc: Heiko Carstens 
Cc: Nick Piggin 
Cc: Hugh Dickins 
Cc: Rik van Riel 
Cc: Peter Zijlstra 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mlock: make mlock error return Posixly Correct

2008-10-20T15:52:31+00:00

Rework Posix error return for mlock().

Posix requires error code for mlock*() system calls for some conditions
that differ from what kernel low level functions, such as
get_user_pages(), return for those conditions.  For more info, see:

http://marc.info/?l=linux-kernel&m=121750892930775&w=2

This patch provides the same translation of get_user_pages()
error codes to posix specified error codes in the context
of the mlock rework for unevictable lru.

[akpm@linux-foundation.org: fix build]
Signed-off-by: KOSAKI Motohiro 
Signed-off-by: Lee Schermerhorn 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

vmstat: mlocked pages statistics

2008-10-20T15:52:31+00:00

Add NR_MLOCK zone page state, which provides a (conservative) count of
mlocked pages (actually, the number of mlocked pages moved off the LRU).

Reworked by lts to fit in with the modified mlock page support in the
Reclaim Scalability series.

[kosaki.motohiro@jp.fujitsu.com: fix incorrect Mlocked field of /proc/meminfo]
[lee.schermerhorn@hp.com: mlocked-pages: add event counting with statistics]
Signed-off-by: Nick Piggin 
Signed-off-by: Lee Schermerhorn 
Signed-off-by: Rik van Riel 
Signed-off-by: KOSAKI Motohiro 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mmap: handle mlocked pages during map, remap, unmap

2008-10-20T15:52:31+00:00

Originally by Nick Piggin 

Remove mlocked pages from the LRU using "unevictable infrastructure"
during mmap(), munmap(), mremap() and truncate().  Try to move back to
normal LRU lists on munmap() when last mlocked mapping removed.  Remove
PageMlocked() status when page truncated from file.

[akpm@linux-foundation.org: cleanup]
[kamezawa.hiroyu@jp.fujitsu.com: fix double unlock_page()]
[kosaki.motohiro@jp.fujitsu.com: split LRU: munlock rework]
[lee.schermerhorn@hp.com: mlock: fix __mlock_vma_pages_range comment block]
[akpm@linux-foundation.org: remove bogus kerneldoc token]
Signed-off-by: Nick Piggin 
Signed-off-by: Lee Schermerhorn 
Signed-off-by: Rik van Riel 
Signed-off-by: KOSAKI Motohiro 
Signed-off-by: KAMEZAWA Hiroyuki 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mlock: downgrade mmap sem while populating mlocked regions

2008-10-20T15:52:31+00:00

We need to hold the mmap_sem for write to initiatate mlock()/munlock()
because we may need to merge/split vmas.  However, this can lead to very
long lock hold times attempting to fault in a large memory region to mlock
it into memory.  This can hold off other faults against the mm
[multithreaded tasks] and other scans of the mm, such as via /proc.  To
alleviate this, downgrade the mmap_sem to read mode during the population
of the region for locking.  This is especially the case if we need to
reclaim memory to lock down the region.  We [probably?] don't need to do
this for unlocking as all of the pages should be resident--they're already
mlocked.

Now, the caller's of the mlock functions [mlock_fixup() and
mlock_vma_pages_range()] expect the mmap_sem to be returned in write mode.
 Changing all callers appears to be way too much effort at this point.
So, restore write mode before returning.  Note that this opens a window
where the mmap list could change in a multithreaded process.  So, at least
for mlock_fixup(), where we could be called in a loop over multiple vmas,
we check that a vma still exists at the start address and that vma still
covers the page range [start,end).  If not, we return an error, -EAGAIN,
and let the caller deal with it.

Return -EAGAIN from mlock_vma_pages_range() function and mlock_fixup() if
the vma at 'start' disappears or changes so that the page range
[start,end) is no longer contained in the vma.  Again, let the caller deal
with it.  Looks like only sys_remap_file_pages() [via mmap_region()]
should actually care.

With this patch, I no longer see processes like ps(1) blocked for seconds
or minutes at a time waiting for a large [multiple gigabyte] region to be
locked down.  However, I occassionally see delays while unlocking or
unmapping a large mlocked region.  Should we also downgrade the mmap_sem
for the unlock path?

Signed-off-by: Lee Schermerhorn 
Signed-off-by: Rik van Riel 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds