summaryrefslogtreecommitdiff
path: root/include/linux/mm.h
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2026-08-27 09:17:06 -0700
committerLinus Torvalds <torvalds@linux-foundation.org>2026-08-27 09:17:06 -0700
commit18fbf5151d2c0bfe433c7428eef03cabf5fdb2fa (patch)
tree6e151850bf46f900bbf9dcc682e0c75d91924865 /include/linux/mm.h
parent5e6ff28676dd92a608eb00eeb8d1319ad34024dc (diff)
parent0685630fdccb62dcb0e3f44525a40578da5f6dc8 (diff)
Merge tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton: - "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff" (Lorenzo Stoakes) Index MAP_PRIVATE file-backed folios by their anonymous page offset to resolve confusion around reverse mapping for zeroed and CoW'd file-backed memory. Use this new VMA anonymous page offset tracking to eliminate index conflicts and lay the foundation for scalable CoW performance improvements. - "promote mapped executable folios after first usage for MGLRU" (Baolin Wang) Make MGLRU's protection of mapped executable file folios more reliable. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage to give executable code a better chance to stay in memory and improve workload performance. - "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong Chen) Fix per-node proactive reclaim interface's ignoring the swappiness parameter when CONFIG_MEMCG is disabled by consolidating sc_swappiness() into a single function that checks proactive_swappiness regardless of kernel configuration. - "mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost" (Usama Arif) Reduce lru_lock contention in the reclaim path by deriving scan-balance costs from vmstat counters rather than lock-acquired producer updates. Read and decay these cost signals on the reclaim side under a dedicated per-lruvec lock, reducing total LRU lock wait time by over 60% without impacting scan throughput. - "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky) Fix two low-risk zram bugs which Sashiko spotted in drive-by review. - "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's memcg" (Zi Yan) Fix xas_split_alloc() by enabling target folio memcg charging during splits and adding the missing __GFP_ACCOUNT flag for proper XArray node memory accounting. - "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick) Replace hardcoded binary names in selftests/mm/.gitignore with a generic pattern-matching rule to automatically ignore generated test files and avoid manual updates when adding new tests. - "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon) Make the incompatibility between FLATMEM and NUMA explicit in mm/Kconfig and remove the unused pgdat_page_ext_init() function. - "zram: fix zstd error paths and add parameter validation" (Haoqin Huang) Clean up zram compression backends by removing redundant error cleanup, adding parameter and dictionary validation, auto-prefixing algorithm error logs, and resetting parameters prior to reinitialization. - "zram: fix stale scan bounds after reinitialization" (Longlong Xia) Prevent out-of-bounds slot accesses during concurrent zram resets by moving table scan bound calculations under dev_lock in writeback_store() and read_block_state(). - "add anon mTHP collapse test cases" (Baolin Wang) Extend selftests helper functions to support arbitrary page orders and add new test cases and options for mTHP collapse in khugepaged. - "selftests/mm: Handle unsupported and transient test conditions" (Muhammad Usama Anjum) Update MM selftests to report a SKIP status instead of a failure when required kernel or filesystem features are unsupported, while adding retry logic for transient page migration errors. - "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia) Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled and extend shrink_memcg() to support batch writeback for improved writeback efficiency. - "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren Baghdasaryan) Introduce an IOCTL-based binary interface for memory allocation profiling that enables kernel-side filtering before per-CPU counter aggregation. This eliminates the text-parsing overhead of /proc/allocinfo and provides up to a 20x speedup by transferring only filtered allocation data to userspace. - "better block swap batching and a different take on swap_ops v5" (Christoph Hellwig) Refactor block swap I/O to use swap_iocb for batching instead of single-bio requests and rebase the swap_ops interface, achieving faster swap throughput during kernel builds. - "mm: kmemleak: reduce transient false positives by confirming leaks" (Catalin Marinas) Reduce false-positive kmemleak reports by combining two kmemleak enhancements that add a second confirmation scan and a configurable minimum unreferenced scan count module parameter. - "mm: kmemleak: default min_unref_scans to 2 for verbose kernels" (Breno Leitao) Auto-scanning kernels can generate false-positive memory leak reports on single scans, so this patch defaults min_unref_scans to 2 when CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second confirming scan. - "swap_ops updates" (Christoph Hellwig) Batching I/O for synchronous swap devices causes performance regressions and filesystem-based swap suffers from double-indirection overhead. This series resolves both issues by reintroducing per-folio writes for synchronous swap and allowing filesystems to directly export their own swap_ops. - "mm/khugepaged: several cleanups" (Nico Pache) khugepaged accumulated redundant state-checking patterns and outdated comments following mTHP integration. Introduce dedicated helpers for PTE validation and event counting while refreshing the internal documentation. - "maple_tree: lock checking and clean ups" (Liam Howlett) Syzbot reports incorrectly blame memory management exit paths for locking bugs, maple tree erase operations risk allocation failures without gfp flags and internal documentation lacks clarity. Improve lock error detection, update docs, fix race and allocation edge cases and optimize erase allocations using a fallback to GFP_KERNEL | GFP_NOFAIL. * tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits) selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC memcg: move LRU size accounting on reparenting instead of copying it mm/vmscan: fix comment logic in balance_pgdat maple_tree: add helper mas_make_walkable() maple_tree: avoid extra gap calculation maple_tree: fix argument name in header maple_tree: change two GFP flags in tests maple_tree: document erase and allocations better maple_tree: avoid mas_erase() and mtree_erase() failures maple_tree: document that erase may use GFP_KERNEL for allocations maple_tree: catch race in mas_alloc_cyclic() maple_tree: add bulk parent set helper maple_tree: micro optimisation of mas_wr_store_type() maple_tree: optimise mas_wr_node_store() when not in rcu mode maple_tree: use prefetched value in mas_wr_store_type() maple_tree: clarify comments on mas_nomem() maple_tree: drop MAPLE_ALLOC_SLOTS maple_tree: drop dead code from mas_extend_spanning_null() maple_tree: documentation fix maple_tree: add write lock checking with lockdep sequence numbers ...
Diffstat (limited to 'include/linux/mm.h')
-rw-r--r--include/linux/mm.h132
1 files changed, 125 insertions, 7 deletions
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 87feaa5a2b78..dd09c438fa23 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1556,7 +1556,7 @@ static inline void vma_desc_set_anonymous(struct vm_area_desc *desc)
desc->vm_ops = NULL;
}
-static inline bool vma_is_anonymous(struct vm_area_struct *vma)
+static inline bool vma_is_anonymous(const struct vm_area_struct *vma)
{
return !vma->vm_ops;
}
@@ -2271,17 +2271,76 @@ void unpin_user_pages(struct page **pages, unsigned long npages);
void unpin_user_folio(struct folio *folio, unsigned long npages);
void unpin_folios(struct folio **folios, unsigned long nfolios);
-static inline bool is_cow_mapping(vm_flags_t flags)
+/**
+ * vma_flags_is_cow_mapping() - Do these VMA flags imply a CoW mapping?
+ * @flags: The VMA flags to check.
+ *
+ * Mappings which could be CoW'd (subject to Copy-On-Write faults) are
+ * described as CoW mappings.
+ *
+ * All mappings backed by anonymous folios (all anonymous mappings and most
+ * MAP_PRIVATE-file backed ranges) are CoW mappings.
+ *
+ * All other mappings (including all MAP_SHARED mappings) are non-CoW.
+ *
+ * The criteria are !VMA_SHARED_BIT, VMA_MAYWRITE_BIT.
+ *
+ * VMA_MAYWRITE_BIT is checked instead of VMA_WRITE_BIT to account for both
+ * future mprotect() calls which can render a read-only mapping writable, and
+ * GUP with FOLL_FORCE (e.g. ptrace) which can CoW a read-only mapping.
+ *
+ * - No anonymous mapping can ever clear VMA_MAYWRITE_BIT.
+ *
+ * - Writes to anonymous mappings do not immediately result in CoW faults but
+ * may do so after the process is forked or if a read is followed by a
+ * write.
+ *
+ * - Writes to MAP_PRIVATE file-backed mappings result in CoW faults and may
+ * do so again after fork.
+ *
+ * - MAP_SHARED mappings of a file opened read-only are transformed into
+ * VMA_MAYSHARE_BIT, !VMA_SHARED_BIT, !VMA_MAYWRITE_BIT mappings, so remain
+ * non-CoW.
+ *
+ * - Drivers may clear VMA_MAYWRITE_BIT but do so at mmap() time and cannot
+ * mark themselves anonymous. Having cleared this flag it is not valid for
+ * them to leave the VMA_WRITE_BIT flag set.
+ *
+ * As a consequence, the anonymous reverse mapping only tracks CoW mappings.
+ *
+ * Returns: true if the flags indicate a CoW mapping, otherwise false.
+ */
+static inline bool vma_flags_is_cow_mapping(const vma_flags_t *flags)
{
- return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE;
+ return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
+ !vma_flags_test(flags, VMA_SHARED_BIT);
}
-static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
+/**
+ * vma_is_cow_mapping() - Is this VMA a CoW mapping?
+ * @vma: The VMA to check.
+ *
+ * See vma_flags_is_cow_mapping() for details.
+ *
+ * Returns: true if the VMA is a CoW mapping, otherwise false.
+ */
+static inline bool vma_is_cow_mapping(const struct vm_area_struct *vma)
{
- const vma_flags_t *flags = &desc->vma_flags;
+ return vma_flags_is_cow_mapping(&vma->flags);
+}
- return vma_flags_test(flags, VMA_MAYWRITE_BIT) &&
- !vma_flags_test(flags, VMA_SHARED_BIT);
+/**
+ * vma_desc_is_cow_mapping() - Is this VMA descriptor a CoW mapping?
+ * @desc: The VMA descriptor to check.
+ *
+ * See vma_flags_is_cow_mapping() for details.
+ *
+ * Returns: true if the VMA descriptor describes a CoW mapping, otherwise
+ * false.
+ */
+static inline bool vma_desc_is_cow_mapping(struct vm_area_desc *desc)
+{
+ return vma_flags_is_cow_mapping(&desc->vma_flags);
}
#ifndef CONFIG_MMU
@@ -4393,6 +4452,65 @@ static inline pgoff_t vma_last_pgoff(const struct vm_area_struct *vma)
return vma_end_pgoff(vma) - 1;
}
+/**
+ * vma_start_anon_pgoff() - Get the anonymous page offset of the start of @vma
+ * @vma: The VMA whose anonymous page offset is required.
+ *
+ * If unfaulted, then this is vma->vm_start >> PAGE_SHIFT, if faulted then the
+ * anonymous page offset at the time of first fault.
+ *
+ * If the VMA is anonymous, this returns the same value as vma_start_pgoff().
+ *
+ * This value is used for tracking MAP_PRIVATE file-backed mappings by their
+ * anonymous page offset.
+ *
+ * Returns: The anonymous page offset of the start of @vma.
+ */
+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
+{
+ pgoff_t pgoff = 0;
+
+#ifdef CONFIG_64BIT
+ pgoff += vma->__vm_anon_pgoff_hi;
+ pgoff <<= 32;
+#endif
+ pgoff += vma->__vm_anon_pgoff_lo;
+ return pgoff;
+}
+
+/**
+ * vma_end_anon_pgoff() - Get the anonymous page offset of the exclusive end of
+ * @vma.
+ * @vma: The VMA whose end anonymous page offset is required.
+ *
+ * This returns the anonymous exclusive end page offset of @vma, which is useful
+ * for expressing page offset ranges.
+ *
+ * See the description of vma_start_anon_pgoff() for a description of VMA
+ * anonymous page offsets.
+ *
+ * Returns: The exclusive end anonymous page offset of @vma.
+ */
+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_start_anon_pgoff(vma) + vma_pages(vma);
+}
+
+/**
+ * vma_last_anon_pgoff() - Get the anonymous page offset of the last page in
+ * @vma.
+ * @vma: The VMA whose last anonymous page offset is required.
+ *
+ * See the description of vma_start_anon_pgoff() for a description of VMA
+ * anonymous page offsets.
+ *
+ * Returns: The last anonymous page offset of @vma.
+ */
+static inline pgoff_t vma_last_anon_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_end_anon_pgoff(vma) - 1;
+}
+
static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)
{
return desc->end - desc->start;