linux-toradex.git/tools/testing/selftests/vm/Makefile, branch master

selftests/vm: rename selftests/vm to selftests/mm

2023-01-19T01:12:56+00:00

Rename selftets/vm to selftests/mm for being more consistent with the
code, documentation, and tools directories, and won't be confused with
virtual machines.

[sj@kernel.org: convert missing vm->mm changes]
  Link: https://lkml.kernel.org/r/20230107230643.252273-1-sj@kernel.org
Link: https://lkml.kernel.org/r/20230103180754.129637-5-sj@kernel.org
Signed-off-by: SeongJae Park 
Cc: Jonathan Corbet 
Cc: Shuah Khan 
Signed-off-by: Andrew Morton

selftests/vm: add KSM unmerge tests

2022-12-12T02:12:07+00:00

Patch series "mm/ksm: break_ksm() cleanups and fixes", v2.

This series cleans up and fixes break_ksm().  In summary, we no longer use
fake write faults to break COW but instead FAULT_FLAG_UNSHARE.  Further,
we move away from using follow_page() --- that we can hopefully remove
completely at one point --- and use new walk_page_range_vma() instead.

Fortunately, we can get rid of VM_FAULT_WRITE and FOLL_MIGRATION in common
code now.

Extend the existing ksm tests by an unmerge benchmark, and a some new
unmerge tests.

Also, add a selftest to measure MADV_UNMERGEABLE performance.  In my setup
(AMD Ryzen 9 3900X), running the KSM selftest to test unmerge performance
on 2 GiB (taskset 0x8 ./ksm_tests -D -s 2048), this results in a
performance degradation of ~6% -- 7% (old: ~5250 MiB/s, new: ~4900 MiB/s).
I don't think we particularly care for now, but it's good to be aware of
the implication.


This patch (of 9):

Let's add three unmerge tests (MADV_UNMERGEABLE unmerging all pages in the
range).

test_unmerge(): basic unmerge tests
test_unmerge_discarded(): have some pte_none() entries in the range
test_unmerge_uffd_wp(): protect the merged pages using uffd-wp

ksm_tests.c currently contains a mixture of benchmarks and tests, whereby
each test is carried out by executing the ksm_tests binary with specific
parameters.  Let's add new ksm_functional_tests.c that performs multiple,
smaller functional tests all at once.

Link: https://lkml.kernel.org/r/20221021101141.84170-1-david@redhat.com
Link: https://lkml.kernel.org/r/20221021101141.84170-5-david@redhat.com
Signed-off-by: David Hildenbrand 
Cc: Andrea Arcangeli 
Cc: Hugh Dickins 
Cc: Jason Gunthorpe 
Cc: John Hubbard 
Cc: Matthew Wilcox (Oracle) 
Cc: Peter Xu 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

selftests/vm: anon_cow: prepare for non-anonymous COW tests

2022-11-30T23:58:56+00:00

Patch series "mm/gup: remove FOLL_FORCE usage from drivers (reliable R/O
long-term pinning)".

For now, we did not support reliable R/O long-term pinning in COW
mappings.  That means, if we would trigger R/O long-term pinning in
MAP_PRIVATE mapping, we could end up pinning the (R/O-mapped) shared
zeropage or a pagecache page.

The next write access would trigger a write fault and replace the pinned
page by an exclusive anonymous page in the process page table; whatever
the process would write to that private page copy would not be visible by
the owner of the previous page pin: for example, RDMA could read stale
data.  The end result is essentially an unexpected and hard-to-debug
memory corruption.

Some drivers tried working around that limitation by using
"FOLL_FORCE|FOLL_WRITE|FOLL_LONGTERM" for R/O long-term pinning for now. 
FOLL_WRITE would trigger a write fault, if required, and break COW before
pinning the page.  FOLL_FORCE is required because the VMA might lack write
permissions, and drivers wanted to make that working as well, just like
one would expect (no write access, but still triggering a write access to
break COW).

However, that is not a practical solution, because
(1) Drivers that don't stick to that undocumented and debatable pattern
    would still run into that issue. For example, VFIO only uses
    FOLL_LONGTERM for R/O long-term pinning.
(2) Using FOLL_WRITE just to work around a COW mapping + page pinning
    limitation is unintuitive. FOLL_WRITE would, for example, mark the
    page softdirty or trigger uffd-wp, even though, there actually isn't
    going to be any write access.
(3) The purpose of FOLL_FORCE is debug access, not access without lack of
    VMA permissions by arbitrarty drivers.

So instead, make R/O long-term pinning work as expected, by breaking COW
in a COW mapping early, such that we can remove any FOLL_FORCE usage from
drivers and make FOLL_FORCE ptrace-specific (renaming it to FOLL_PTRACE).
More details in patch #8.


This patch (of 19):

Originally, the plan was to have a separate tests for testing COW of
non-anonymous (e.g., shared zeropage) pages.

Turns out, that we'd need a lot of similar functionality and that there
isn't a really good reason to separate it. So let's prepare for non-anon
tests by renaming to "cow".

Link: https://lkml.kernel.org/r/20221116102659.70287-1-david@redhat.com
Link: https://lkml.kernel.org/r/20221116102659.70287-2-david@redhat.com
Signed-off-by: David Hildenbrand 
Acked-by: Vlastimil Babka 
Cc: Alexander Shishkin 
Cc: Alexander Viro 
Cc: Alex Williamson 
Cc: Andrea Arcangeli 
Cc: Andy Walls 
Cc: Anton Ivanov 
Cc: Arnaldo Carvalho de Melo 
Cc: Arnd Bergmann 
Cc: Bernard Metzler 
Cc: Borislav Petkov 
Cc: Catalin Marinas 
Cc: Christian Benvenuti 
Cc: Christian Gmeiner 
Cc: Christophe Leroy 
Cc: Christoph Hellwig 
Cc: Daniel Vetter 
Cc: Daniel Vetter 
Cc: Dave Hansen 
Cc: David Airlie 
Cc: David S. Miller 
Cc: Dennis Dalessandro 
Cc: "Eric W . Biederman" 
Cc: Greg Kroah-Hartman 
Cc: Hans Verkuil 
Cc: "H. Peter Anvin" 
Cc: Hugh Dickins 
Cc: Ingo Molnar 
Cc: Inki Dae 
Cc: Ivan Kokshaysky 
Cc: James Morris 
Cc: Jason Gunthorpe 
Cc: Jiri Olsa 
Cc: Johannes Berg 
Cc: John Hubbard 
Cc: Kees Cook 
Cc: Kentaro Takeda 
Cc: Krzysztof Kozlowski 
Cc: Kyungmin Park 
Cc: Leon Romanovsky 
Cc: Leon Romanovsky 
Cc: Linus Torvalds 
Cc: Lucas Stach 
Cc: Marek Szyprowski 
Cc: Mark Rutland 
Cc: Matthew Wilcox 
Cc: Matt Turner 
Cc: Mauro Carvalho Chehab 
Cc: Michael Ellerman 
Cc: Mike Kravetz 
Cc: Muchun Song 
Cc: Nadav Amit 
Cc: Namhyung Kim 
Cc: Nelson Escobar 
Cc: Nicholas Piggin 
Cc: Oded Gabbay 
Cc: Oleg Nesterov 
Cc: Paul Moore 
Cc: Peter Xu 
Cc: Peter Zijlstra 
Cc: Richard Henderson 
Cc: Richard Weinberger 
Cc: Russell King 
Cc: Serge Hallyn 
Cc: Seung-Woo Kim 
Cc: Shuah Khan 
Cc: Tetsuo Handa 
Cc: Thomas Bogendoerfer 
Cc: Thomas Gleixner 
Cc: Tomasz Figa 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

selftests/vm: anon_cow: add liburing test cases

2022-11-09T01:37:15+00:00

io_uring provides a simple mechanism to test long-term, R/W GUP pins
-- via fixed buffers -- and can be used to verify that GUP pins stay
in sync with the pages in the page table even if a page would
temporarily get mapped R/O or concurrent fork() could accidentially
end up sharing pinned pages with the child.

Note that this essentially re-introduces local_config support that was
removed recently in commit 6f83d6c74ea5 ("Kselftests: remove support of
libhugetlbfs from kselftests").

[david@redhat.com: s/size_t/ssize_t/ on `cur', `total'.]
  Link: https://lkml.kernel.org/r/445fe1ae-9e22-0d1d-4d09-272231d2f84a@redhat.com
Link: https://lkml.kernel.org/r/20220927110120.106906-6-david@redhat.com
Signed-off-by: David Hildenbrand 
Cc: Andrea Arcangeli 
Cc: Christoph von Recklinghausen 
Cc: Don Dutile 
Cc: Jason Gunthorpe 
Cc: John Hubbard 
Cc: Mike Rapoport 
Cc: Nadav Amit 
Cc: Peter Xu 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

selftests/vm: anon_cow: test COW handling of anonymous memory

2022-11-09T01:37:14+00:00

Patch series "selftests/vm: test COW handling of anonymous memory".

This is my current set of tests for testing COW handling of anonymous
memory, especially when interacting with GUP.  I developed these tests
while working on PageAnonExclusive and managed to clean them up just now.

On current upstream Linux, all tests pass except the hugetlb tests that
rely on vmsplice -- these tests should pass as soon as vmsplice properly
uses FOLL_PIN instead of FOLL_GET.

I'm working on additional tests for COW handling in private mappings,
focusing on long-term R/O pinning e.g., of the shared zeropage, pagecache
pages and KSM pages.  These tests, however, will go into a different file.
So this is everything I have regarding tests for anonymous memory.


This patch (of 7):

Let's start adding tests for our COW handling of anonymous memory.  We'll
focus on basic tests that we can achieve without additional libraries or
gup_test extensions.

We'll add THP and hugetlb tests separately.

[david@redhat.com: s/size_t/ssize_t/ on `cur', `total', `transferred';]
  Link: https://lkml.kernel.org/r/51302b9e-dc69-d709-3214-f23868028555@redhat.com
Link: https://lkml.kernel.org/r/20220927110120.106906-1-david@redhat.com
Link: https://lkml.kernel.org/r/20220927110120.106906-2-david@redhat.com
Signed-off-by: David Hildenbrand 
Cc: Andrea Arcangeli 
Cc: Christoph von Recklinghausen 
Cc: Don Dutile 
Cc: Jason Gunthorpe 
Cc: John Hubbard 
Cc: Mike Rapoport 
Cc: Nadav Amit 
Cc: Peter Xu 
Cc: Shuah Khan 
Cc: Vlastimil Babka 
Signed-off-by: Andrew Morton

Merge tag 'mm-stable-2022-10-08' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2022-10-11T00:53:04+00:00

Pull MM updates from Andrew Morton:

- Yu Zhao's Multi-Gen LRU patches are here. They've been under test in
linux-next for a couple of months without, to my knowledge, any
negative reports (or any positive ones, come to that).

- Also the Maple Tree from Liam Howlett. An overlapping range-based
tree for vmas. It it apparently slightly more efficient in its own
right, but is mainly targeted at enabling work to reduce mmap_lock
contention.

Liam has identified a number of other tree users in the kernel which
could be beneficially onverted to mapletrees.

Yu Zhao has identified a hard-to-hit but "easy to fix" lockdep splat
at [1]. This has yet to be addressed due to Liam's unfortunately
timed vacation. He is now back and we'll get this fixed up.

- Dmitry Vyukov introduces KMSAN: the Kernel Memory Sanitizer. It uses
clang-generated instrumentation to detect used-unintialized bugs down
to the single bit level.

KMSAN keeps finding bugs. New ones, as well as the legacy ones.

- Yang Shi adds a userspace mechanism (madvise) to induce a collapse of
memory into THPs.

- Zach O'Keefe has expanded Yang Shi's madvise(MADV_COLLAPSE) to
support file/shmem-backed pages.

- userfaultfd updates from Axel Rasmussen

- zsmalloc cleanups from Alexey Romanov

- cleanups from Miaohe Lin: vmscan, hugetlb_cgroup, hugetlb and
memory-failure

- Huang Ying adds enhancements to NUMA balancing memory tiering mode's
page promotion, with a new way of detecting hot pages.

- memcg updates from Shakeel Butt: charging optimizations and reduced
memory consumption.

- memcg cleanups from Kairui Song.

- memcg fixes and cleanups from Johannes Weiner.

- Vishal Moola provides more folio conversions

- Zhang Yi removed ll_rw_block() :(

- migration enhancements from Peter Xu

- migration error-path bugfixes from Huang Ying

- Aneesh Kumar added ability for a device driver to alter the memory
tiering promotion paths. For optimizations by PMEM drivers, DRM
drivers, etc.

- vma merging improvements from Jakub Matěn.

- NUMA hinting cleanups from David Hildenbrand.

- xu xin added aditional userspace visibility into KSM merging
activity.

- THP & KSM code consolidation from Qi Zheng.

- more folio work from Matthew Wilcox.

- KASAN updates from Andrey Konovalov.

- DAMON cleanups from Kaixu Xia.

- DAMON work from SeongJae Park: fixes, cleanups.

- hugetlb sysfs cleanups from Muchun Song.

- Mike Kravetz fixes locking issues in hugetlbfs and in hugetlb core.

Link: https://lkml.kernel.org/r/CAOUHufZabH85CeUN-MEMgL8gJGzJEWUrkiM58JkTbBhh-jew0Q@mail.gmail.com [1]

* tag 'mm-stable-2022-10-08' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (555 commits)
hugetlb: allocate vma lock for all sharable vmas
hugetlb: take hugetlb vma_lock when clearing vma_lock->vma pointer
hugetlb: fix vma lock handling during split vma and range unmapping
mglru: mm/vmscan.c: fix imprecise comments
mm/mglru: don't sync disk for each aging cycle
mm: memcontrol: drop dead CONFIG_MEMCG_SWAP config symbol
mm: memcontrol: use do_memsw_account() in a few more places
mm: memcontrol: deprecate swapaccounting=0 mode
mm: memcontrol: don't allocate cgroup swap arrays when memcg is disabled
mm/secretmem: remove reduntant return value
mm/hugetlb: add available_huge_pages() func
mm: remove unused inline functions from include/linux/mm_inline.h
selftests/vm: add selftest for MADV_COLLAPSE of uffd-minor memory
selftests/vm: add file/shmem MADV_COLLAPSE selftest for cleared pmd
selftests/vm: add thp collapse shmem testing
selftests/vm: add thp collapse file and tmpfs testing
selftests/vm: modularize thp collapse memory operations
selftests/vm: dedup THP helpers
mm/khugepaged: add tracepoint to hpage_collapse_scan_file()
mm/madvise: add file and shmem support to MADV_COLLAPSE
...

selftests/vm: use top_srcdir instead of recomputing relative paths

2022-10-05T17:05:17+00:00

In various places both in t/t/s/v/Makefile as well as some of the test
sources, we were referring to headers or directories using some fairly
long relative paths.

Since we have a working top_srcdir variable though, which refers to the
root of the kernel tree, we can clean up all of these "up and over"
relative paths, just relying on the single variable instead.

Signed-off-by: Axel Rasmussen 
Signed-off-by: Shuah Khan

selftests/vm: add selftest for MADV_COLLAPSE of uffd-minor memory

2022-10-03T21:03:35+00:00

Add :collapse mod to userfaultfd selftest.  Currently this mod is only
valid for "shmem" test type, but could be used for other test types.

When provided, memory allocated by ->allocate_area() will be
hugepage-aligned enforced to be hugepage-sized.  userfaultf_minor_test,
after the UFFD-registered mapping has been populated by UUFD minor fault
handler, attempt to MADV_COLLAPSE the UFFD-registered mapping to collapse
the memory into a pmd-mapped THP.

This test is meant to be a functional test of what occurs during
UFFD-driven live migration of VMs backed by huge tmpfs where, after a
hugepage-sized region has been successfully migrated (in native page-sized
chunks, to avoid latency of fetched a hugepage over the network), we want
to reclaim previous VM performance by remapping it at the PMD level.

Link: https://lkml.kernel.org/r/20220907144521.3115321-11-zokeefe@google.com
Link: https://lkml.kernel.org/r/20220922224046.1143204-11-zokeefe@google.com
Signed-off-by: Zach O'Keefe 
Cc: Axel Rasmussen 
Cc: Chris Kennelly 
Cc: David Hildenbrand 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: James Houghton 
Cc: "Kirill A. Shutemov" 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Pasha Tatashin 
Cc: Peter Xu 
Cc: Rongwei Wang 
Cc: SeongJae Park 
Cc: Song Liu 
Cc: Vlastimil Babka 
Cc: Yang Shi 
Signed-off-by: Andrew Morton

selftests/vm: dedup THP helpers

2022-10-03T21:03:34+00:00

These files:

tools/testing/selftests/vm/vm_util.c
tools/testing/selftests/vm/khugepaged.c

Both contain logic to:

1) Determine hugepage size on current system
2) Read /proc/self/smaps to determine number of THPs at an address

Refactor selftests/vm/khugepaged.c to use the vm_util common helpers and
add it as a build dependency.

Since selftests/vm/khugepaged.c is the largest user of check_huge(),
change the signature of check_huge() to match selftests/vm/khugepaged.c's
useage: take an expected number of hugepages, and return a bool indicating
if the correct number of hugepages were found.  Add a wrapper,
check_huge_anon(), in anticipation of checking smaps for file and shmem
hugepages.

Update existing callsites to use the new pattern / function.

Likewise, check_for_pattern() was duplicated, and it's a general enough
helper to include in vm_util helpers as well.

Link: https://lkml.kernel.org/r/20220907144521.3115321-6-zokeefe@google.com
Link: https://lkml.kernel.org/r/20220922224046.1143204-6-zokeefe@google.com
Signed-off-by: Zach O'Keefe 
Reviewed-by: Zi Yan 
Cc: Axel Rasmussen 
Cc: Chris Kennelly 
Cc: David Hildenbrand 
Cc: David Rientjes 
Cc: Hugh Dickins 
Cc: James Houghton 
Cc: "Kirill A. Shutemov" 
Cc: Matthew Wilcox 
Cc: Miaohe Lin 
Cc: Minchan Kim 
Cc: Pasha Tatashin 
Cc: Peter Xu 
Cc: Rongwei Wang 
Cc: SeongJae Park 
Cc: Song Liu 
Cc: Vlastimil Babka 
Cc: Yang Shi 
Signed-off-by: Andrew Morton

Kselftests: remove support of libhugetlbfs from kselftests

2022-09-12T03:25:52+00:00

libhugetlbfs, the user side utitlity to work with hugepages, does not have
any active support.  There are only 2 selftests which are part of in
vm/hmm_test.c that depends on libhugetlbfs.

This patch modifies the tests so that they will not require libhugetlb
library.

[axelrasmussen@google.com: : remove orphaned references to local_config.{h,mk}]
  Link: https://lkml.kernel.org/r/20220831211526.2743216-1-axelrasmussen@google.com
Link: https://lkml.kernel.org/r/20220801070231.13831-1-tsahu@linux.ibm.com
Signed-off-by: Tarun Sahu 
Signed-off-by: Axel Rasmussen 
Tested-by: Zach O'Keefe 
Cc: "Aneesh Kumar K.V" 
Cc: Jerome Glisse 
Cc: Shivaprasad G Bhat 
Cc: Shuah Khan 
Cc: Vaibhav Jain 
Cc: Axel Rasmussen 
Signed-off-by: Andrew Morton