| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock
Pull memblock fixes from Mike Rapoport:
"Fix for regions display in debugfs and MAINTAINERS update:
- Make sure that multiple flags on a memblock region are all
displayed in debugfs
- Update memblock tree tags in MAINTAINERS"
* tag 'fixes-2026-09-14' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock:
MAINTAINERS: update memblock tree URLs
mm: memblock: show all region flags in debugfs
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull misc fixes from Andrew Morton:
"14 hotfixes. 10 are cc:stable. 11 are for MM.
All are singletons - please see the changelogs for details"
* tag 'mm-hotfixes-stable-2026-09-13-21-50' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm:
mm/folio: EXPORT_SYMBOL_FOR_KVM(lru_cache_drain_for_folio)
mm/shrinker: fix bogus set_shrinker_bit() with cgroup.memory=nokmem
mm/vma: correctly unaccount on mmap_prepare() failure
mm/mlock: use the IRQ-safe accessor for NR_MLOCK in __munlock_folio()
remove old lib/alloc_tag.c
fs/dax: check zero or empty entry before converting xarray entry
fs: fix missed removal of super_fs_objects_eligible()
mm: filemap: retain mapped dropbehind folios
mailmap: update entry for Christopher Obbard
memcg: avoid charging the root memcg from obj_cgroup_charge_pages()
mm, swap: fix SWAP_USAGE_OFFLIST_BIT collision with real usage count
mailmap: map Coiby Xu's address
mm/mremap: account mm->locked_vm correctly for MREMAP_DONTUNMAP
mm/huge_memory: bypass THP tuneables for huge pfnmap mappings
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mm/slab
Pull slab fixes from Vlastimil Babka:
- Stable fix for an ABA issue causing slab list corruption introduced
in 7.2 (Harry Yoo, with big thanks to Hyunwoo Kim for the thorough
report and initial version of the fix)
- Fix for 7.3 regression of kvfree_rcu() on PREEMPT_RT which can cause
a deadlock from the set_cpus_allowed_force() caller (Vlastimil Babka)
* tag 'slab-for-7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/slab:
mm/slab: take n->list_lock in __slab_try_return_freelist() to avoid race
mm/slab: disallow kfree_rcu_sheaf() on PREEMPT_RT again
|
|
Commit 493f349e38d0 ("memblock: Add flags and nid info in memblock
debugfs") made memblock_debug_show() stop after finding the first set
flag. A memblock region can carry multiple flags, so the remaining flags
are hidden from debugfs.
Walk all bits in the region flags and print every set flag separated by
"|". Keep walking beyond flagname[] so that a set flag without a known
name is reported as UNKNOWN rather than silently ignored.
Fixes: 493f349e38d0 ("memblock: Add flags and nid info in memblock debugfs")
Signed-off-by: Meijing Zhao <zhaomeijing@lixiang.com>
Link: https://patch.msgid.link/20260902075944.3742866-1-zhaomeijing100@gmail.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
To simplify independent development in the KVM and MM subsystems, now
export to KVM the lru_cache_drain_for_folio() which MM added in 7.3-rc1.
Link: https://lore.kernel.org/lkml/bd6c9c74-e374-a9d3-ba1f-8b6f430894fc@google.com/T/#u
Link: https://lore.kernel.org/02876cea-5727-2ca4-bead-73659ea6fec4@google.com
Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Suggested-by: David Hildenbrand <david@kernel.org>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Sean Christopherson <seanjc@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
With cgroup.memory=nokmem, shrinker_memcg_alloc() bails out early and
never allocates an id, so shrinker->id keeps the 0 it got from the
kzalloc() in shrinker_alloc(). __list_lru_init() then copies that 0 into
lru->shrinker_id, where it looks like a valid bit index.
Nothing calls expand_shrinker_info() on nokmem either, so shrinker_nr_max
stays 0 and every memcg ends up with an empty map (map_nr_max == 0).
deferred_split_folio() hands a real memcg to __list_lru_add() regardless
of whether the lru is memcg aware, so the first THP queued in a cgroup
does set_shrinker_bit(memcg, nid, 0) and trips the bounds check:
WARNING: mm/shrinker.c:212 at set_shrinker_bit+0x7d/0x90, CPU#126
Call Trace:
<TASK>
deferred_split_folio+0x18c/0x220
map_anon_folio_pmd_nopf+0xdd/0x130
map_anon_folio_pmd_pf+0x14/0xb0
do_huge_pmd_anonymous_page+0x1a1/0x620
__handle_mm_fault+0xea9/0x10d0
handle_mm_fault+0xe5/0x320
do_user_addr_fault+0x1cc/0x870
exc_page_fault+0x81/0x1b0
asm_exc_page_fault+0x27/0x30
</TASK>
Harmless, the WARN_ON_ONCE() is what keeps the out of bounds unit[] read
from happening, but the id should not look valid in the first place.
Clear it before returning.
Two other spots could paper over this: drop the id in __list_lru_init()
when nokmem turns memcg_aware off, or make deferred_split_folio() pass
NULL like list_lru_add_obj() does. Both leave shrinker->id lying around
for the next caller, so fix it where the id is handed out.
Link: https://lore.kernel.org/20260902073800.305481-1-jiayuan.chen@linux.dev
Fixes: fafaeceb89a5 ("mm: switch deferred split shrinker to list_lru")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
__mmap_setup() accounts memory for relevant mappings via:
security_vm_enough_memory_mm()
-> __vm_enough_memory()
-> vm_acct_memory()
If __mmap_setup() fails, this indicates that this accounting did not take
place, and thus it's appropriate for __mmap_region() to jump to
abort_munmap.
However if call_mmap_prepare() fails, it also jumps there and any accounted
memory is not correctly unaccounted.
Fix this by handling each error separately.
Link: https://lore.kernel.org/20260902-fix-unaccount-mmap_prepare-v1-1-ea070189fdfb@kernel.org
Fixes: c84bf6dd2b83 ("mm: introduce new .mmap_prepare() file callback")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
NR_MLOCK is updated from interrupt context. __free_pages_prepare() clears
a stray PG_mlocked and adjusts NR_MLOCK, and a folio can reach it with the
flag still set from a bio completion handler:
__free_pages_ok+0x6af/0x7a0
<IRQ>
__bio_release_pages+0xde/0x260
__iomap_dio_bio_end_io+0x16e/0x1a0
blk_update_request+0x14b/0x3d0
blk_mq_end_request+0x18/0x30
blk_done_softirq+0x49/0x60
The folio gets there like this. A MAP_SHARED file mapping is mlocked, so
its page cache folios carry PG_mlocked, and an O_DIRECT write sourced from
that mapping GUP-pins those same folios. munlock() then runs
mlock_vma_pages_range(), which clears VM_LOCKED before walking the page
tables to munlock each folio. A concurrent hole punch reaches the folio
through the rmap (i_mmap_rwsem, not mmap_lock) and can land inside that
window: __folio_remove_rmap() -> munlock_vma_folio() sees VM_LOCKED
already clear, so it neither queues the folio on the mlock batch nor takes
a reference, and the pte it clears makes the pending mlock_pte_range()
walk skip the folio at its !pte_present() check. filemap_remove_folio()
then drops the page cache reference, leaving the bio's pin as the last
one, released from the completion handler above.
So __zone_stat_mod_folio() here needs interrupts disabled, not merely
preemption, and __munlock_folio() has a path where they are not: when the
folio has already been taken off the LRU by somebody else the function
jumps straight to the counter update without taking the lruvec lock. The
read-modify-write of the per-CPU NR_MLOCK diff can then be interrupted by
the softirq above, and one of the two decrements is lost, leaving Mlocked
in /proc/meminfo permanently overstated.
Use zone_stat_mod_folio(). mod_zone_state()'s this_cpu_try_cmpxchg() is
atomic against a same-CPU interrupt and retries, and on the path where the
lruvec lock is held its cost is negligible next to the lock itself.
The UNEVICTABLE_PG* events are deliberately left on the __ accessors: they
occupy different vm_event_states slots from the UNEVICTABLE_PGCLEARED that
__free_pages_prepare() bumps, and nothing updates those two from interrupt
context.
Link: https://lore.kernel.org/20260901180109.3797944-1-shakeel.butt@linux.dev
Fixes: 2fbb0c10d1e8 ("mm/munlock: mlock_page() munlock_page() batch by pagevec")
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Reported-by: syzbot+cd2073ee6d958a8d0fcd@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/linux-mm/6a931c5a.08e933ee.dbf97.0093.GAE@google.com/
Acked-by: Hugh Dickins <hughd@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Fault-around can map ready dropbehind folios without going through the
normal page-cache lookup that clears dropbehind. A mapping represents a
competing cached user, so retain the folio instead of forcibly unmapping
it when writeback completes.
For a mapped folio, folio_unmap_invalidate() can call
unmap_mapping_folio(), which takes i_mmap_rwsem and may sleep. Retaining
mapped folios avoids this path when folio_end_dropbehind() runs in
non-preemptible task context.
Tal was able to trigger a sleeping-in-atomic warning due to this [1].
Unmapped dropbehind folios continue through the existing invalidation path.
Link: https://lore.kernel.org/4aba05e1a2c3b61cb337d373eb9b7a8db4ddd822.1788024049.git.qiwenjie@xiaomi.com
Link: https://lore.kernel.org/076bb01b-6fcf-4691-be8c-0e8507c9fe64@columbia.edu [1]
Fixes: fb7d3bc41493 ("mm/filemap: drop streaming/uncached pages when writeback completes")
Signed-off-by: Wenjie Qi <qiwenjie@xiaomi.com>
Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Tal Zussman <tz2294@columbia.edu>
Tested-by: Tal Zussman <tz2294@columbia.edu>
Cc: Barry Song <baohua@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Trond Myklebust <trond.myklebust@hammerspace.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
obj_cgroup_charge_pages() resolves the objcg to its memcg and calls
try_charge_memcg(), which does not short circuit the root memcg. That
memcg can be the root memcg: obj_cgroup_is_root() reflects the memcg the
objcg was created for and is never updated, while memcg_reparent_objcgs()
does redirect objcg->memcg to the parent on rmdir. An objcg of a dying
child of root therefore passes every obj_cgroup_is_root() filter but
resolves to the root memcg.
Folios keep the objcg they were charged with, so this is easy to reach
through zswap: allocate anon memory in a cgroup, move the task out, remove
the cgroup, then write to the root cgroup's memory.reclaim. The reclaimed
folios are charged through the reparented objcg and end up in
refill_stock() with the root memcg:
WARNING: mm/memcontrol.c:2198 at refill_stock+0x644/0x940
refill_stock+0x644/0x940
try_charge_memcg+0x12d6/0x1570
__obj_cgroup_charge+0x35/0xf0
obj_cgroup_charge+0x1de/0x210
obj_cgroup_charge_zswap+0x83/0x270
zswap_store+0x1620/0x2000
swap_writeout+0x94c/0x14c0
shrink_folio_list+0x3388/0x52b0
[...]
try_to_free_mem_cgroup_pages+0x30d/0x830
user_proactive_reclaim+0x504/0x840
memory_reclaim+0x1f/0x30
Beyond the warning, the charge is asymmetric: obj_cgroup_uncharge_pages()
skips refill_stock() for the root memcg, so the root's page counter grows
and is never uncharged. It is not user visible, since memory.current is
not exposed on the root, but it is a leak.
Use try_charge(), which returns early for the root memcg, restoring the
symmetry with obj_cgroup_uncharge_pages().
The above sequence was scripted into a standalone reproducer (zswap on,
swap on a virtio disk, 512MB of anon memory faulted in inside a child of
the root cgroup, the task then migrated to the root cgroup, the child
removed, followed by "echo 600M swappiness=max > memory.reclaim" on the
root) and run in a CONFIG_DEBUG_VM=y VM. It reproduces the splat on the
first zswap store of a reparented folio, with the same call chain as the
report. With this patch applied the splat is gone while the zswap store
count over the run is unchanged, so the same path is still exercised.
cgroup selftests test_zswap, test_kmem and test_memcontrol show no new
failures.
Link: https://lore.kernel.org/20260829023251.474083-1-shakeel.butt@linux.dev
Fixes: 20d6c1725228 ("memcg: avoid refill_stock for root memcg")
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Closes: https://lore.kernel.org/all/CA+0ovCgWzUMK+nNbbtH7eV65Ca=fDN4Ozu7iASgryjvv8Tk8zQ@mail.gmail.com/
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
SWAP_USAGE_OFFLIST_BIT is embedded in the si->inuse_pages usage counter,
and is meant to sit above any value that counter can reach. However, it
is defined from BITS_PER_TYPE(atomic_t), so it is bit 30. On a system
with 4 KiB pages the flag collides with the usage count once that count
reaches 4 TiB.
swap_usage_in_pages() masks bit 30 out, so whenever the real count has
that bit set, every caller of it reads 4 TiB low:
* /proc/swaps understates Used by 4 TiB.
* A raw count of exactly 2^30 masks to zero, so try_to_unuse() takes its
"if (!swap_usage_in_pages(si)) goto success;" early exit and swapoff
tears the device down while pages are still swapped out. Nothing in
the rest of swapoff aborts the teardown, so those pages are lost.
Independently of swapoff, the collision also corrupts the counter and the
plist. On a device in normal use, a free that leaves bit 30 set in the
count makes swap_usage_sub() see the flag where there is only count, and
call add_to_avail_list(). It clears the bit with
fetch_and(~SWAP_USAGE_OFFLIST_BIT), leaving the stored count 4 TiB below
the real one, and calls plist_add() on a device that is already listed,
tripping the WARN_ON(!plist_node_empty(node)) in plist_add() and linking
the node a second time.
Change the definition of SWAP_USAGE_OFFLIST_BIT to be based on
atomic_long_t instead. Note that the usage counter field itself is of
this same type, so it is still a valid bit.
Link: https://lore.kernel.org/20260828191433.3304458-1-nphamcs@gmail.com
Fixes: b228386cf237 ("mm, swap: clean up plist removal and adding")
Signed-off-by: Nhat Pham <nphamcs@gmail.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260825153238.2695446-1-nphamcs%40gmail.com
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Kairui Song <kasong@tencent.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Gregory Price <gourry@gourry.net>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Youngjun Park <youngjun.park@lge.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
When a VMA is mremap()'d with MREMAP_DONTUNMAP set, that results in the
VMA being copied, but the source VMA not being unmapped.
If the VMA is mlock()'d this is a legal operation, though the source VMA
has its VMA_LOCKED_BIT cleared.
However this is done in dontunmap_complete(), after mm->locked_vm was
incremented via vrm_stat_account(), resulting in double-counting.
Worse, this is not even corrected when source VMA is unmapped, due to the
VMA_LOCKED_BIT flag having been cleared.
This all works fine in the usual mremap() case (without MREMAP_DONTUNMAP),
as the source VMA is unmapped with VMA_LOCKED_BIT intact, at which time
mm->locked_vm is decremented accordingly.
Resolve the issue by invoking vrm_stat_account() only after
dontunmap_complete() has run.
Note that MREMAP_DONTUNMAP requires old_len == new_len, so no need to
account for a delta in size in this case.
The bug was introduced by commit b714ccb02a76 ("mm/mremap: complete
refactor of move_vma()") which incorrectly reordered the accounting and
the clearing of the VMA_LOCKED_BIT flag.
Link: https://lore.kernel.org/20260828-mremap-fix-locked-vm-v1-1-c80be7505d1e@kernel.org
Fixes: b714ccb02a76 ("mm/mremap: complete refactor of move_vma()")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260825-fix-mremap-dontunmap-pgoff-v1-1-39a40b2c98b3@kernel.org
Reported-by: Kunwu Chan <kunwu.chan@gmail.com>
Closes: https://lore.kernel.org/all/20260828094823.594279-1-kunwu.chan@linux.dev/
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Tested-by: Kunwu Chan <kunwu.chan@gmail.com>
Reviewed-by: Kunwu Chan <kunwu.chan@gmail.com>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The sysfs THP tuneables at /sys/kernel/mm/transparent_huge_pages/ rather
confusingly only control the behaviour of THP in some instances.
They are not applicable to MADV_COLLAPSE operations, nor to DAX mappings.
Long-term, THP is predicated upon compaction being able to obtain large
folios to populate THP ranges.
However, vm_normal_folio() returns NULL for PFN map mappings, thus their
reference count is maintained by the driver, not core mm.
As a consequence, the folios are not subject to reclaim nor compaction, so
are not truly part of the THP mechanism at all.
However, since commit 5dd40721f147 ("mm: allow THP orders for PFNMAPs")
introduced the ability to establish huge PFN maps, they have been subject
to THP tuneables.
This is incorrect - if a huge PFN map is available (defined by
vma->vm_ops->huge_fault being non-NULL for a VMA_PFNMAP_BIT VMA), then it
should be mapped huge upon fault-in.
Correct this by explicitly checking for this while ensuring that smaps
continues to accurately report THPeligible statistics.
While here, abstract the entire file-backed THP check in
vma_can_map_huge_file(), with sensible separation of logic into helper
functions.
Note that drm_gem_shmem_mmap() and panthor_gem_mmap() establish huge PFN
maps of shmem folios, however they are marked unevictable in
drm_gem_get_pages(), and in any case would fail the reference check in
__remove_mapping() even if they weren't.
Failing to map huge PFN maps has resulted in significant real-world
performance degradation, see links for details.
[ziy@nvidia.com: rename some functions]
Link: https://lore.kernel.org/DL1HIHWYJ7TB.1CY76SJS0V03L@nvidia.com
Link: https://lore.kernel.org/20260827-hugepfn-allowable-orders-v1-1-94819c8807c8@kernel.org
Fixes: 5dd40721f147 ("mm: allow THP orders for PFNMAPs")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reported-by: Cedric Le Goater <clg@redhat.com>
Closes: https://lore.kernel.org/linux-mm/20260805055544.1568534-1-clg@redhat.com/
Reported-by: Saravanan D <saravanand@crusoe.ai>
Closes: https://lore.kernel.org/linux-mm/20260821070520.25759-1-saravanand@crusoe.ai/
Reviewed-by: Zi Yan <ziy@nvidia.com>
Tested-by: Saravanan D <saravanand@crusoe.ai>
Tested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: SJ Park <sj@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
|
|
Commit ba7425312607 ("mm, slab: add an optimistic
__slab_try_return_freelist()") incorrectly assumed that nobody has freed
an object to the slab as long as slab->freelist is NULL and cmpxchg
succeeds.
However, as reported by Hyunwoo Kim [1], other CPUs might have freed
an object to the slab, insert the slab to the partial list, then
allocated an object from the slab, and be in the middle of removing
the slab from the list under n->list_lock.
Since __refill_objects_node() puts the slab back on pc.slabs
outside n->list_lock, it might insert the slab into that list while
the slab is concurrently being removed from n->partial.
This led to a list corruption [1]:
list_add corruption. next->prev should be prev
(ffff888100000248), but was dead000000000122.
(next=ffffea000416e410).
kernel BUG at lib/list_debug.c:29!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 1 UID: 65534 PID: 144 Comm: poc Not tainted
7.2.0-16172-gcf72cbb39da8-dirty #1 PREEMPT(lazy)
RIP: 0010:__list_add_valid_or_report+0x80/0xd0
...
Call Trace:
alloc_from_new_slab+0x183/0x300
___slab_alloc+0x31c/0x890
__kmalloc_noprof+0x3d4/0x800
lsm_blob_alloc+0x2d/0x50
security_msg_msg_alloc+0x26/0x90
load_msg+0x1aa/0x210
do_msgsnd+0x91/0x800
do_syscall_64+0x109/0x5d0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
...
Kernel panic - not syncing: Fatal exception
This is a classic ABA problem where cmpxchg succeeds but the state has
changed since __refill_objects_node() took the freelist from the slab.
As Vlastimil Babka mentioned [2], it should be rare to return more than
one slab (due to the racy read of slab->counters in
get_partial_node_bulk()). Therefore, instead of introducing additional
complexity, acquire and release n->list_lock twice in the worst case.
Return the slab directly to the partial list and hold n->list_lock
across the cmpxchg and add_partial(). This is similar to the initial
version of commit ba7425312607 [3]. This is enough to avoid the race as
the list manipulation is serialized by n->list_lock. While at it,
bring back unlikely() hint now that the condition is unlikely.
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Closes: https://lore.kernel.org/linux-mm/apPa-cGLcyt90l-E@v4bel [1]
Link: https://lore.kernel.org/linux-mm/ae25c193-b95f-40c1-83b6-1c2546467e41@kernel.org [2]
Link: https://lore.kernel.org/all/20260421-b4-refill-optimistic-return-v1-1-24f0bfc1acff@kernel.org [3]
Fixes: ba7425312607 ("mm, slab: add an optimistic __slab_try_return_freelist()")
Cc: stable@vger.kernel.org
Signed-off-by: Harry Yoo (Meta) <harry@kernel.org>
Link: https://patch.msgid.link/20260903-slab-fix-aba-v3-1-b44cb6badd54@kernel.org
Reviewed-by: Hao Li <hao.li@linux.dev>
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
|
|
This partially reverts commit 2a8bb29ec9b2 ("mm/slab: allow
kfree_rcu_sheaf() on PREEMPT_RT"). It was based on the assumption that
local_trylock() is safe on PREEMPT_RT from any context.
However kvfree_rcu() is also called by set_cpus_allowed_force() with
task_struct::pi_lock acquired and there it's not safe, as syzbot
has reported.
For the immediate fix, skip kfree_rcu_sheaf() on PREEMPT_RT again from
kvfree_call_rcu(). In theory, kfree_rcu_nolock() would have the same
problem when called from under pi_lock on PREEMPT_RT but that can
be addressed if such a caller is proposed.
Add an explanation comment, courtesy of Sebastian.
Reported-by: syzbot+acf142088e0182172e58@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=acf142088e0182172e58
Reported-by: ThangNN99 <ngocthang2710.1999@gmail.com>
Fixes: 2a8bb29ec9b2 ("mm/slab: allow kfree_rcu_sheaf() on PREEMPT_RT")
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260831-b4-kfree_rcu_hotfix-v1-1-4f0fb882638b@kernel.org
Reviewed-by: Harry Yoo (Meta) <harry@kernel.org>
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
|
|
secretmem accounts folios by treating memory as if it were mlock()'d and
thus limited by the RLIMIT_MEMLOCK limit.
However the folios are unevictable and remain so until the inode is
evicted, eliminating usual mlock() semantics - mapping folios then
unmapping them does not clear their unevictable state, since it depends on
AS_UNEVICTABLE, not PG_mlocked.
A user can therefore easily work around the RLIMIT_MEMLOCK limit - simply
map then unmap and VmLck no longer counts the secretmem range. Worse,
folios are not accounted in the process's RSS, meaning the OOM killer
won't know to kill the process.
Repeatedly mapping/unmapping (or forking) can then result in the
consumption of all available system memory with unevictable folios and
cause system instability.
A secretmem fd can be passed between processes and over fork so a
per-process limit simply does not make sense, so follow the precedent set
by io_uring, perf, skbuff, iommufd and xdp by tracking the number of
locked pages in user_struct->locked_vm.
Since the scope tracked is actually inode lifetime, the RLIMIT_MEMLOCK
applies per-user not per-process, so it doesn't make sense to bypass for
users with CAP_IPC_LOCK, therefore remove this bypass.
There is simply no reason to carry on marking the mapping as mlock()'d
since it's misleading and the lifecycle is now correctly handled, so
remove this too.
Note that secretmem does not support any form of truncation (including
hole punching) and the folios are unreclaimable, so the folios need only
be accounted on fault and unaccounted on inode destruction.
__secretmem_account_pages() is more or less a duplicate of the code that
io_uring etc. use, but since this is a bug fix that needs backporting,
defer any de-duplication efforts to a follow-up.
test_mlock_limit() asserts mlock_future_ok() on mmap(), however this has
been removed, so remove the test altogether for the fix. A new test will
be sent separately for upstream.
Link: https://lore.kernel.org/20260826-secretmem-accounting-v3-1-94cb04399510@kernel.org
Fixes: 1507f51255c9 ("mm: introduce memfd_secret system call to create "secret" memory areas")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reported-by: Daehyeon Ko <4ncienth@gmail.com>
Closes: https://lore.kernel.org/linux-mm/20260813225328.2010303-1-4ncienth@gmail.com/
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Daehyeon Ko <4ncienth@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: David S. Miller <davem@davemloft.net>
Cc: Hagen Paul Pfeifer <hagen@jauu.net>
Cc: Jakub Kacinski <kuba@kernel.org>
Cc: James Bottomley <james.bottomley@HansenPartnership.com>
Cc: Jesper Dangaard Brouer <hawk@kernel.org>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Stanislav Fomichev <sdf@fomichev.me>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Uniquely an mremap() invocation using the MREMAP_DONTUNMAP flag can reset
a faulted VMA into an unfaulted one.
It does so after the page tables have been moved to the copied VMA with
MREMAP_DONTUNMAP leaving the old VMA in place which is naturally unfaulted
as the page tables it had are no longer present.
However, in doing so, it violates the invariant that the anonymous page
offset of an unfaulted VMA is vma->vm_start >> PAGE_SHIFT.
This is because a VMA may have been faulted in, mremap()'d (causing a
delta between its page offset and vma->vm_start >> PAGE_SHIFT), and then
mremap()'d again with MREMAP_DONTUNMAP resulting in the unfaulting.
This condition is a violation of a fundamental assumption in mm, but now
also triggers an assert in assert_sane_pgoff() which explicitly checks for
this condition.
Correct it by resetting the VMA's page offset at the point of completing
the MREMAP_DONTUNMAP operation.
Link: https://lore.kernel.org/20260825-fix-mremap-dontunmap-pgoff-v1-1-39a40b2c98b3@kernel.org
Fixes: 1583aa278f5f ("mm: mremap: unlink anon_vmas when mremap with MREMAP_DONTUNMAP success")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reported-by: syzbot+f12658786a4153df5113@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a87853b.ae6ddae5.3da009.0023.GAE@google.com/
Tested-by: syzbot+f12658786a4153df5113@syzkaller.appspotmail.com
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Kunwu Chan <kunwu.chan@gmail.com>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Li Xinhai <lixinhai.lxh@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Commit ba23f58de896 ("mm/migrate: don't call
folio_putback_active_hugetlb() on dst hugetlb folio") moved setting of the
migratable flag and active-list placement from
folio_putback_active_hugetlb(dst) into move_hugetlb_state(), so that the
freshly allocated destination folio is handled where allocation is known
to have succeeded.
Unfortunately, the new code was appended after the existing
temporary-folio block in move_hugetlb_state(), which contains an early
return added earlier by commit 5af1ab1d24e08 ("mm/hugetlb: optimize the
surplus state transfer code in move_hugetlb_state()"):
if (folio_test_hugetlb_temporary(new_folio)) {
...
if (new_nid == old_nid)
return; <-- skips the new code
...
}
/* added by ba23f58 */
folio_set_hugetlb_migratable(new_folio);
list_move_tail(&new_folio->lru, ...&h->hugepage_activelist);
When the destination folio is temporary (i.e. the hugetlb pool was
exhausted and the migration callback fell back to
alloc_migrate_hugetlb_folio()) and the migration does not cross a node --
the common case, and always true on a single-NUMA system --
move_hugetlb_state() returns before setting the migratable flag or adding
the new folio to the active list. The destination folio is then installed
in the page table but cannot be isolated afterwards, since
folio_isolate_hugetlb() rejects folios without the migratable flag; a
subsequent soft-offline, hard-offline or memory-hotplug offline of that
folio fails with -EBUSY.
This was reproduced on a single-NUMA arm64 VM: a second MADV_SOFT_OFFLINE
on an already-migrated hugetlb page returned EBUSY and logged "hugepage
isolation failed".
Keep the surplus adjustment, which is the only part that depends on the
node crossing, guarded by `if (new_nid != old_nid)', while making the
migratable flag and active-list placement unconditional. This preserves
the cleanup intent of ba23f58 and closes the early-return hole.
Link: https://lore.kernel.org/20260707110254.3147686-1-mawupeng1@huawei.com
Fixes: ba23f58de896 ("mm/migrate: don't call folio_putback_active_hugetlb() on dst hugetlb folio")
Signed-off-by: Wupeng Ma <mawupeng1@huawei.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
syzbot reported a sleeping function called from invalid context splat in
bucket_table_alloc().
When rhashtable_insert_slow() rehashes the table under rcu_read_lock(), it
calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN). If the bucket
table allocation uses vmalloc, __vmalloc_node_range_noprof() invokes
vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with the
passed GFP_ATOMIC flags.
If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
GFP_KERNEL when allocating the temporary weights array, triggering a
might_alloc() splat in atomic/RCU contexts.
Pass the gfp flags (masked with GFP_RECLAIM_MASK to strip page-allocator
zone modifiers like __GFP_HIGHMEM) received by
alloc_pages_bulk_weighted_interleave() to kmalloc() instead of hardcoding
GFP_KERNEL. Since the weights buffer is immediately initialized in full,
kmalloc() is sufficient.
Link: https://lore.kernel.org/20260821170407.3721004-1-edumazet@google.com
Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: syzbot+0dbf6d295b3350944f0b@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/lkml/6a88837e.ae6ddae5.3da009.0040.GAE@google.com/T/#u
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Zi Yan <ziy@nvidia.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
zap_huge_pmd_folio() propagates the pmd young bit to the folio for the
file case, but not the dirty bit. The pte path does propagate it, in
zap_present_folio_ptes() and so does the pmd split path, in
__split_huge_pmd_locked().
For most file mappings the omission is harmless, because writing to a
shared file mapping goes through page_mkwrite(), which dirties the folio.
tmpfs is different: it has no page_mkwrite(), and vma_wants_writenotify()
is false for it, so a *read* fault on a MAP_SHARED tmpfs mapping installs
a writable pmd via do_read_fault(). do_read_fault() does not call
fault_dirty_shared_page(), so subsequent stores through that mapping set
only the hardware dirty bit in the pmd and never call folio_mark_dirty().
A shmem folio allocated by a fault is marked uptodate but not dirty (see
the clear: block in shmem_get_folio_gfp()), so PG_dirty is never set at
all.
Unmapping such a folio - munmap(), or exit_mmap() when the process dies -
then loses the only record that it was written, because zap_huge_pmd()
drops the pmd without transferring the dirty bit. Reclaim afterwards sees
a clean shmem folio: the whole swap-out block in shrink_folio_list() is
inside "if (folio_test_dirty(folio))", so pageout() is skipped and the
folio falls into __remove_mapping(). There, folio_is_file_lru() is false
for a swapbacked folio, so no shadow entry is created and
__filemap_remove_folio(folio, NULL) simply empties the i_pages slot. The
data is freed without ever being written to swap, and the next fault on
that index returns a freshly zeroed folio.
This is silent data loss for any process that keeps state in a MAP_SHARED
tmpfs segment across an unmap - for example a cache handed from one
process generation to the next through /dev/shm. It requires the folio to
be PMD-mapped, so it only shows up once shmem THP is enabled (which is
what we did in Meta fleet and started noticing crashes); with THP off the
pte path transfers the dirty bit correctly. It also only becomes visible
when swap is enabled, because with no swap device shmem folios (which are
on the anon LRU) are not scanned by reclaim at all, so the clean folio is
never dropped.
Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a
2MB-backed region, write a known pattern through the resulting mapping,
munmap, force reclaim of the cgroup, then re-map and read back. Without
this patch the region reads back as zeros and vmstat shows zswpout 0 - the
data was discarded rather than swapped. With this patch the region reads
back correctly and the pages are swapped out as expected. With
huge=never, or when the first touch is a write, the test passes either
way.
Link: https://lore.kernel.org/20260819101222.3732660-1-usama.arif@linux.dev
Fixes: b5072380eb61 ("thp: support file pages in zap_huge_pmd()")
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Kiryl Shutsemau <kas@kernel.org>
Acked-by: Hugh Dickins <hughd@google.com>
Tested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
During move_pages() operation, when move_pages_ptes() returns EAGAIN, the
error code is not cleared even after we processed it. This leads to a
successful retry but then the same pages are retried again due to the
stale error code. This time move fails because pages are already moved,
loop is terminated and move_pages() reports a failure. Clear the error
code once we processes EAGAIN.
Link: https://lore.kernel.org/e1e0b5f8-c3c6-0537-670b-4397f822f980@gmail.com
Fixes: 50944692052b ("userfaultfd: opportunistic TLB-flush batching for present pages in MOVE")
Assisted-by: ChatGPT:GPT-5.6-Luna
Signed-off-by: Bryan Lim <foxieflakey@gmail.com>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Lance reported that when nothing else causes the mm to be considered for
khugepaged collapse, an MADV_HUGEPAGE-advised tmpfs VMA alone does not
trigger scanning.
After commit 6beeab870e70 ("mm: shmem: move shmem_huge_global_enabled()
into shmem_allowable_huge_orders()"), the shmem/tmpfs allowable order
check reads vma->flags directly. However, when MADV_HUGEPAGE is handled,
khugepaged_enter_vma() is called before the VMA's flags have been updated,
so the check uses stale flags and incorrectly rejects the VMA for
collapse. As a result, khugepaged does not collapse the tmpfs file into
PMD order in time.
Fix this by calling khugepaged_enter_vma() with the new VMA flags in
madvise_update_vma(). Meanwhile we can remove the khugepaged_enter_vma()
in hugepage_madvise().
Link: https://lore.kernel.org/7d5b5eb27be798f89d563b06254c947ff53db0b2.1787020910.git.baolin.wang@linux.alibaba.com
Fixes: 6beeab870e70 ("mm: shmem: move shmem_huge_global_enabled() into shmem_allowable_huge_orders()")
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reported-by: Lance Yang <lance.yang@linux.dev>
Closes: https://lore.kernel.org/all/20260815181632.21453-1-lance.yang@linux.dev/
Suggested-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
dissolve_free_hugetlb_folio() can remove a free folio as surplus when its
node has surplus pages. In that case remove_hugetlb_folio() decrements
both nr_huge_pages and surplus_huge_pages, leaving the persistent pool
size unchanged.
Updating max_huge_pages as if a persistent folio had been removed can
therefore corrupt the persistent pool target and underflow it when
max_huge_pages is zero. Keep max_huge_pages unchanged for surplus folios,
including the vmemmap restoration rollback path.
Link: https://lore.kernel.org/20260814083027.1419487-1-xialonglong2025@163.com
Fixes: cb402bbdabca ("mm/hugetlb: fix surplus pages in dissolve_free_huge_page()")
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Longlong Xia <xialonglong@kylinos.cn>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jinjiang Tu <tujinjiang@huawei.com>
Cc: Longlong Xia <xialonglong@kylinos.cn>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
migrate_device_range() and migrate_device_pfns() clear the entries
following a compound folio so that the PFN arrays retain their
page-granular representation.
If a compound folio extends beyond the end of the caller-provided range,
the loops clear all following folio entries without limiting them to the
number of slots remaining in the npages-sized array, causing an
out-of-bounds write.
Do not proceed with a compound folio if its page-granular representation
does not fit entirely in the remaining PFN array. If this happens, drop
any reference and lock acquired for the folio, clear the remaining
entries, and stop collecting.
Observed with a KASAN x86 QEMU kernel using the HMM migrate_anon_huge_zero
selftest. Closing /dev/hmm_dmirror0 after migrating an anonymous huge
page to device memory exercises:
dmirror_fops_release()
-> dmirror_device_evict_chunk()
-> migrate_device_range()
Link: https://lore.kernel.org/20260817120758.669807-3-sh_def@163.com
Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
Signed-off-by: Hui Su <sh_def@163.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Balbir Singh <balbirs@nvidia.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Zi Yan <ziy@nvidia.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
hugetlb_cgroup_css_alloc() rounds the counter limit down to a multiple of
the huge page size and then applies it inside an assertion:
VM_BUG_ON(page_counter_set_max(fault, limit));
VM_BUG_ON(page_counter_set_max(rsvd, limit));
With CONFIG_DEBUG_VM=n, VM_BUG_ON(cond) is BUILD_BUG_ON_INVALID(cond),
i.e. ((void)(sizeof((__force long)(cond)))), whose operand is never
evaluated. page_counter_set_max() is not a predicate - it performs
xchg(&counter->max, nr_pages) - so on every non-debug kernel the limit is
never applied and the counters keep page_counter_init()'s
PAGE_COUNTER_MAX.
That is user-visible, because hugetlb_cgroup_read_u64_max() recomputes the
same rounded value and uses equality as its "unlimited" sentinel.
PAGE_COUNTER_MAX is LONG_MAX / PAGE_SIZE = 2251799813685247, which is odd,
so round_down() really does change it and the two sides disagree. With
CONFIG_DEBUG_VM=n:
$ cat /sys/fs/cgroup/t/hugetlb.2MB.max
9223372036854771712
and with this patch:
$ cat /sys/fs/cgroup/t/hugetlb.2MB.max
max
A debug option should not change cgroup output.
Call the function, then assert the result, as v6.12 did. Use
VM_WARN_ON_ONCE() rather than restoring VM_BUG_ON(): the two are identical
under CONFIG_DEBUG_VM=n, and checkpatch asks that new code not use BUG()
variants.
Link: https://lore.kernel.org/20260817103433.191266-1-njilav@gmail.com
Fixes: 0e2759afcaf9 ("page_counter: track failcnt only for legacy cgroups")
Signed-off-by: Narek Jilavyan <njilav@gmail.com>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The v1 soft limit has been deprecated since v6.12 and nobody has reported
depending on it. Start the removal by decoupling the interface from the
implementation: keep memory.soft_limit_in_bytes, but ignore writes to it
and always report the maximum value on read similar to what
memory.kmem.limit_in_bytes already does.
Writes are still parsed, so malformed input keeps returning -EINVAL. The
knob now also behaves the same everywhere: it used to return -EOPNOTSUPP
on PREEMPT_RT, where soft limit reclaim has always been disabled.
This also fixes the syzbot report linked below. Soft limit reclaim is the
only caller that runs shrink_lruvec() from kswapd against a specific
memcg, so it is the only way to reach lru_gen_shrink_lruvec() and in turn
set_mm_walk(), which warns when called from kswapd.
Link: https://lore.kernel.org/20260811203203.3456029-2-shakeel.butt@linux.dev
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Reported-by: syzbot+12ee2725d5fde63a9c96@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a7a6929.b50370da.49fe0.005e.GAE@google.com/
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
alloc_buddy_hugetlb_folio_with_mpol() can pass a NULL nodemask to
alloc_fresh_hugetlb_folio() as a fallback to allocate from all nodes. If
order is gigantic, alloc_fresh_hugetlb_folio() propagates the NULL
nodemask down to hugetlb_cma_alloc_frozen_folio() via
alloc_gigantic_frozen_folio().
Additionally, hugetlb_cma_alloc_frozen_folio() previously attempted
allocation on hugetlb_cma[nid] without verifying if nid is included in the
caller's nodemask. Adding a node_isset(nid, *nodemask) check ensures the
initial preferred node allocation honors the memory policy / nodemask.
However, hugetlb_cma_alloc_frozen_folio() dereferences the nodemask in
node_isset(nid, *nodemask) and for_each_node_mask(node, *nodemask),
leading to a null pointer dereference kernel panic when nodemask is NULL.
Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio() and defaulting it to
cpuset_current_mems_allowed. Enclose the allocation attempts within the
cpuset seqcount retry loop so that if the cpuset changes concurrently
during allocation, the attempts are retried using the updated nodemask.
This ensures that the initial node check and fallback loop safely honor
the task's cpuset without violating cpuset constraints or causing NULL
pointer dereferences or unexpected allocation failures.
From a userspace perspective, this bug allows an unprivileged user to
crash the kernel (trigger a panic) by requesting a gigantic hugepage
allocation with MPOL_PREFERRED_MANY on a system where CMA is only
configured on a subset of NUMA nodes.
This can be reproduced by booting a VM with two NUMA nodes, restricting
CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G
hugepages=0), and running a program that allocates a 1GB hugepage area
without reserving, restricts allocation to Node 0 using mbind() with
MPOL_PREFERRED_MANY, and triggers a page fault:
void *ptr = mmap(NULL, 1UL << 30, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB |
MAP_HUGE_1GB | MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */
mbind(ptr, 1UL << 30, MPOL_PREFERRED_MANY, &nodemask,
sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */
This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
Oops: Oops: 0000 [#1] SMP NOPTI
RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120
Call Trace:
<TASK>
only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160
alloc_surplus_hugetlb_folio+0x6d/0x100
alloc_hugetlb_folio+0x3c5/0x660
hugetlb_no_page+0x3d9/0x650
Link: https://lore.kernel.org/20260811052909.475635-1-souravpanda@google.com
Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Signed-off-by: Sourav Panda <souravpanda@google.com>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Rik van Riel <riel@surriel.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton:
- "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff"
(Lorenzo Stoakes)
Index MAP_PRIVATE file-backed folios by their anonymous page offset
to resolve confusion around reverse mapping for zeroed and CoW'd
file-backed memory.
Use this new VMA anonymous page offset tracking to eliminate index
conflicts and lay the foundation for scalable CoW performance
improvements.
- "promote mapped executable folios after first usage for MGLRU"
(Baolin Wang)
Make MGLRU's protection of mapped executable file folios more
reliable. Follow the classical LRU's logic, promoting mapped
executable file folios after their first usage to give executable
code a better chance to stay in memory and improve workload
performance.
- "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong
Chen)
Fix per-node proactive reclaim interface's ignoring the swappiness
parameter when CONFIG_MEMCG is disabled by consolidating
sc_swappiness() into a single function that checks
proactive_swappiness regardless of kernel configuration.
- "mm/vmscan: reduce lru_lock contention via vmstat-derived
scan-balance cost" (Usama Arif)
Reduce lru_lock contention in the reclaim path by deriving
scan-balance costs from vmstat counters rather than lock-acquired
producer updates.
Read and decay these cost signals on the reclaim side under a
dedicated per-lruvec lock, reducing total LRU lock wait time by over
60% without impacting scan throughput.
- "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky)
Fix two low-risk zram bugs which Sashiko spotted in drive-by review.
- "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's
memcg" (Zi Yan)
Fix xas_split_alloc() by enabling target folio memcg charging during
splits and adding the missing __GFP_ACCOUNT flag for proper XArray
node memory accounting.
- "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick)
Replace hardcoded binary names in selftests/mm/.gitignore with a
generic pattern-matching rule to automatically ignore generated test
files and avoid manual updates when adding new tests.
- "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon)
Make the incompatibility between FLATMEM and NUMA explicit in
mm/Kconfig and remove the unused pgdat_page_ext_init() function.
- "zram: fix zstd error paths and add parameter validation" (Haoqin
Huang)
Clean up zram compression backends by removing redundant error
cleanup, adding parameter and dictionary validation, auto-prefixing
algorithm error logs, and resetting parameters prior to
reinitialization.
- "zram: fix stale scan bounds after reinitialization" (Longlong Xia)
Prevent out-of-bounds slot accesses during concurrent zram resets by
moving table scan bound calculations under dev_lock in
writeback_store() and read_block_state().
- "add anon mTHP collapse test cases" (Baolin Wang)
Extend selftests helper functions to support arbitrary page orders
and add new test cases and options for mTHP collapse in khugepaged.
- "selftests/mm: Handle unsupported and transient test conditions"
(Muhammad Usama Anjum)
Update MM selftests to report a SKIP status instead of a failure when
required kernel or filesystem features are unsupported, while adding
retry logic for transient page migration errors.
- "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia)
Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled
and extend shrink_memcg() to support batch writeback for improved
writeback efficiency.
- "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren
Baghdasaryan)
Introduce an IOCTL-based binary interface for memory allocation
profiling that enables kernel-side filtering before per-CPU counter
aggregation.
This eliminates the text-parsing overhead of /proc/allocinfo and
provides up to a 20x speedup by transferring only filtered allocation
data to userspace.
- "better block swap batching and a different take on swap_ops v5"
(Christoph Hellwig)
Refactor block swap I/O to use swap_iocb for batching instead of
single-bio requests and rebase the swap_ops interface, achieving
faster swap throughput during kernel builds.
- "mm: kmemleak: reduce transient false positives by confirming leaks"
(Catalin Marinas)
Reduce false-positive kmemleak reports by combining two kmemleak
enhancements that add a second confirmation scan and a configurable
minimum unreferenced scan count module parameter.
- "mm: kmemleak: default min_unref_scans to 2 for verbose kernels"
(Breno Leitao)
Auto-scanning kernels can generate false-positive memory leak reports
on single scans, so this patch defaults min_unref_scans to 2 when
CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second
confirming scan.
- "swap_ops updates" (Christoph Hellwig)
Batching I/O for synchronous swap devices causes performance
regressions and filesystem-based swap suffers from double-indirection
overhead. This series resolves both issues by reintroducing per-folio
writes for synchronous swap and allowing filesystems to directly
export their own swap_ops.
- "mm/khugepaged: several cleanups" (Nico Pache)
khugepaged accumulated redundant state-checking patterns and outdated
comments following mTHP integration. Introduce dedicated helpers for
PTE validation and event counting while refreshing the internal
documentation.
- "maple_tree: lock checking and clean ups" (Liam Howlett)
Syzbot reports incorrectly blame memory management exit paths for
locking bugs, maple tree erase operations risk allocation failures
without gfp flags and internal documentation lacks clarity.
Improve lock error detection, update docs, fix race and allocation
edge cases and optimize erase allocations using a fallback to
GFP_KERNEL | GFP_NOFAIL.
* tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits)
selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC
memcg: move LRU size accounting on reparenting instead of copying it
mm/vmscan: fix comment logic in balance_pgdat
maple_tree: add helper mas_make_walkable()
maple_tree: avoid extra gap calculation
maple_tree: fix argument name in header
maple_tree: change two GFP flags in tests
maple_tree: document erase and allocations better
maple_tree: avoid mas_erase() and mtree_erase() failures
maple_tree: document that erase may use GFP_KERNEL for allocations
maple_tree: catch race in mas_alloc_cyclic()
maple_tree: add bulk parent set helper
maple_tree: micro optimisation of mas_wr_store_type()
maple_tree: optimise mas_wr_node_store() when not in rcu mode
maple_tree: use prefetched value in mas_wr_store_type()
maple_tree: clarify comments on mas_nomem()
maple_tree: drop MAPLE_ALLOC_SLOTS
maple_tree: drop dead code from mas_extend_spanning_null()
maple_tree: documentation fix
maple_tree: add write lock checking with lockdep sequence numbers
...
|
|
When a memory cgroup is offlined its LRU folios are reparented to the
parent. lruvec_reparent_lru() splices the child's lists into the
parent's and credits the parent with the child's per-zone
lru_zone_size[], but never clears the child's copy, so the size is
copied rather than moved. lru_gen_reparent_memcg() does the same for
MGLRU.
The parent is left correct, credited with exactly the folios it took
over. The stale value sits on the child and nothing will correct it:
folio->memcg_data now resolves to the parent, so every later
update_lru_size() for those folios goes there.
Dying cgroups are not freed immediately and mem_cgroup_iter() still
walks them, so shrink_lruvec() keeps being called on them.
get_scan_count() reads the phantom counter through lruvec_lru_size() and
the scan loop then grinds through nr[] in SWAP_CLUSTER_MAX steps against
an empty list, for as long as the dead cgroup lives. Under MGLRU the
MGLRU scanner runs instead, but count_shadow_nodes() sums all of
NR_LRU_LISTS through lruvec_lru_size() and over-budgets the shadow node
limit just the same.
On one 251 GiB host a sweep of every mz->lru_zone_size[] found 380
counters describing folios on no list at all: 124777314 pages, 476 GiB,
1.89x the machine's RAM, across 57 cgroups. All were on memcgs with
CSS_DYING set and CSS_ONLINE clear, and parent/child pairs reported
byte-identical sizes.
LRU_UNEVICTABLE needs its size moved too. Its list is deliberately not
spliced because lruvec_init() poisons the head - the unevictable LRU is
imaginary and folios are never threaded on it - but the size is kept by
lruvec_add_folio()/lruvec_del_folio() and those folios account to the
parent from here on.
This depends on commit bf4ade7dbd76 ("memcg: keep folio's objcg same as
its node") and must not be backported ahead of it. Without that
invariant a folio's objcg can belong to another node, so a folio already
spliced onto the parent's list can still resolve to the child's lruvec
until the objcg's node is reparented in a later iteration of
memcg_reparent_objcgs(); clearing the child's counter early then lets
lruvec_del_folio() underflow it and trip the WARN_ONCE()/VM_BUG_ON() in
mem_cgroup_update_lru_size().
Link: https://lore.kernel.org/20260822024707.77192-1-shakeel.butt@linux.dev
Fixes: 07a6e9a2c199 ("mm: vmscan: prepare for reparenting traditional LRU folios")
Fixes: f304652609ea ("mm: vmscan: prepare for reparenting MGLRU folios")
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: <stable@vger.kernel.org> # After: bf4ade7dbd76: memcg: keep folio's objcg same as its node
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
In balance_pgdat(), when the low watermark is met, processes sleeping on
pfmemalloc_wait are woken up because they are able to safely make forward
progress.
However, the comment incorrectly states "they should not be able",
which contradicts the actual code behavior. Fix this typo to accurately
reflect the logic.
Link: https://lore.kernel.org/20260821064057.4081-1-enlin.mu@linux.dev
Signed-off-by: Enlin Mu <enlin.mu@unisoc.com>
Signed-off-by: Enlin Mu <enlin.mu@linux.dev>
Reviewed-by: Barry Song <baohua@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
A corrupt page table hands the same bogus entry to get_swap_device() on
every access to the mapping, and every rejection is logged. One machine
logged 6185620 copies of the same line in a few hours.
swap_dup_entry_direct() prints the same message from the fork path, once
per call: the WARN_ON_ONCE() guarding it warns once, the pr_err() inside
does not.
Rate limit all three prints.
Link: https://lore.kernel.org/20260818-swap_part_one-v1-1-a4fc58119fc0@debian.org
Fixes: 23b230ba8ac3 ("mm/swap: print bad swap offset entry in get_swap_device")
Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Barry Song <baohua@kernel.org>
Reviewed-by: Nhat Pham <nphamcs@gmail.com>
Acked-by: Kairui Song <kasong@tencent.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Chris Li <chrisl@kernel.org>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
consume_stock() can drive a stock slot's nr_pages to zero while its
cached[] pointer stays set, so the slot keeps pinning the css reference
that refill_stock() took. The offlining drain only flushes slots with
cached pages, so the reference is never released unless the slot happens
to be displaced by an unrelated charge or by CPU hotplug, and the memcg
lingers in the dying state - up to NR_MEMCG_STOCK (7) of them per CPU
under container churn.
Keeping the slot populated past the last page only saves a
css_get()/css_put() pair on the next charge of the same memcg, and costs
more than that: the offlining drain has to know about empty slots, and
refill_stock() cannot reuse them either, so a charge under a different
memcg evicts a live batch through the drain_idx rotation instead.
Drop the reference in consume_stock() when the slot empties. Empty slots
stop existing, so is_memcg_drain_needed() and the drain path stay as they
are, and refill_stock() reuses emptied slots directly. The cost is one
refcount pair per emptied slot, at most once per MEMCG_CHARGE_BATCH pages.
Link: https://lore.kernel.org/20260818130135.154315-1-husong@kylinos.cn
Fixes: d1a05b6973c7 ("memcg: do not try to drain per-cpu caches without pages")
Signed-off-by: Song Hu <husong@kylinos.cn>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Audra Mitchell <audra@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Nico Pache <npache@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
migrate_vma_insert_huge_pmd_page() calls flush_cache_page() before
replacing an existing huge zero PMD. However, the third argument to
flush_cache_page() is a PFN, while addr + HPAGE_PMD_SIZE is an end virtual
address.
More importantly, the mapping being invalidated is PMD-sized rather than
PAGE_SIZE-sized. Flush the whole PMD range with flush_cache_range(),
matching other huge PMD invalidation paths.
There is no userspace-visible effect today. The architectures that
currently enable ARCH_ENABLE_THP_MIGRATION use no-op implementations of
flush_cache_page()/flush_cache_range(). 32-bit ARM has non-trivial
implementations, but does not enable ARCH_ENABLE_THP_MIGRATION.
So this appears to be a latent API misuse rather than a currently
observable bug, and I don't think a stable backport is necessary.
Link: https://lore.kernel.org/20260817060845.377800-2-sh_def@163.com
Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
Signed-off-by: Hui Su <sh_def@163.com>
Reviewed-by: Balbir Singh <balbirs@nvidia.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
When building ARCH=riscv using clang with CONFIG_FORTIFY_SOURCE and
CONFIG_UBSAN_BOUNDS enabled, CONFIG_NR_CPUS > 64, and the default value of
2 for CONFIG_NODES_SHIFT, there is a compiletime warning from the fortify
routines.
In file included from mm/arch_numa.c:11:
In file included from include/linux/acpi.h:14:
In file included from include/linux/resource_ext.h:11:
In file included from include/linux/slab.h:17:
In file included from include/linux/gfp.h:7:
In file included from include/linux/mmzone.h:8:
In file included from include/linux/spinlock.h:60:
In file included from include/linux/interrupt_rc.h:17:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:11:
In file included from include/linux/bitmap.h:13:
In file included from include/linux/string.h:383:
include/linux/fortify-string.h:430:4: warning: call to '__write_overflow_field' declared with 'warning' attribute: detected write beyond size of field (1st parameter); maybe use struct_group()? [-Wattribue-warning]
430 | __write_overflow_field(p_size_field, size);
| ^
include/linux/fortify-string.h:430:4: note: called by function 'fortify_memset_chk(unsigned long, unsigned long, unsigned long)'
include/linux/bitmap.h:248:3: note: inlined by function 'setup_node_to_cpumask_map'
248 | memset(dst, 0, len);
| ^
include/linux/fortify-string.h:462:25: note: expanded from macro 'memset'
462 | #define memset(p, c, s) __fortify_memset_chk(p, c, s, \
| ^
include/linux/fortify-string.h:453:2: note: expanded from macro '__fortify_memset_chk'
453 | fortify_memset_chk(__fortify_size, p_size, p_size_field), \
| ^
include/linux/fortify-string.h:430:4: note: use '-gline-directives-only' (implied by '-g1') or higher for more accurate inlining chain locations
430 | __write_overflow_field(p_size_field, size);
| ^
1 warning generated.
In this configuration, MAX_NUMNODES is 4. clang unrolls the for loop in
setup_node_to_cpumask_map() past this, which triggers the fortify check
when accessing node_to_cpumask_map on the theoretical fifth loop iteration
because it would be an out of bounds write.
Make it clear to clang that nr_node_ids is bounded by MAX_NUMNODES due to
the logic in setup_nr_node_ids() by early returning in
setup_node_to_cpumask_map() should that condition be violated.
Link: https://lore.kernel.org/20260813-arch_numa-avoid-fortify-warning-v2-1-093ad97a78df@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Closes: https://github.com/ClangBuiltLinux/linux/issues/2174
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Bill Wendling <morbo@google.com>
Cc: Justin Stitt <justinstitt@google.com>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Currently, in anon_vma_clone(), src vma's anon_vma is assigned to
active_anon_vma and is used when unlocking anon_vma after linking new
AVCs. However, the anon_vma is locked using src->anon_vma, instead of
active_anon_vma, making the lock and unlock target inconsistent.
Use active_anon_vma for both locking and unlocking.
Link: https://lore.kernel.org/OS7PR01MB139142FE16EC63B892559D40496DA2@OS7PR01MB13914.jpnprd01.prod.outlook.com
Signed-off-by: Eric Kim <seohyun.kim@outlook.kr>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Harry Yoo <harry@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
mm/hmm.c:673 hmm_do_fault() error: we previously assumed 'hmm_vma_walk->locked' could be null (see line 654)
Stanislav says this can't happen. Waste a few cycles to make the warning
go away.
[akpm@linux-foundation.org: WARN_ON_ONCE() if the handler didn't set ->locked, per Stanislav]
Link: https://lore.kernel.org/anu1N-DOnQwxO1kF@skinsburskii
Fixes: 121170831228 ("mm/hmm: add hmm_range_fault_unlocked_timeout() for mmap lock-drop support")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/202608101053.PhnVUM4u-lkp@intel.com
Cc: Stanislav Kinsburskii <skinsburskii@gmail.com>
Cc: David Hildenbrand <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
For embedded devices, lacking support for NUMA, memory hotplug/hotremove,
CMA and huge pages is a quite common scenario. In this scenario, the
demand for contiguous physical memory allocation is very low. To reduce
the kernel image size, some devices disable the compaction. However,
their SoCs do support DDR ECC, meaning that memory-failure may be needed.
Migration is very useful for soft_offline_page() in memory-failure, which
may be triggered by correctable memory errors. Most anonymous and
file-mapped faulty pages can be migrated to other healthy pages.
Currently, MEMORY_FAILURE does not explicitly select MIGRATION. When
COMPACTION, MEMORY_HOTREMOVE, NUMA_MIGRATION and CMA are all disabled,
MEMORY_FAILURE can be enabled, but MIGRATION cannot be selected.
Make MEMORY_FAILURE select MIGRATION to handle this situation.
Link: https://lore.kernel.org/20260813134916.292733-1-xieyuanbin1@huawei.com
Signed-off-by: Xie Yuanbin <xieyuanbin1@huawei.com>
Suggested-by: Mike Rapoport <rppt@kernel.org>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Miaohe Lin <linmiaohe@huawei.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Byungchul Park <byungchul@sk.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Gregory Price <gourry@gourry.net>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: liaohua <liaohua4@huawei.com>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Yuanbin Xie <xieyuanbin1@huawei.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
alpha requires percpu variables in modules to be defined as weak so that the
compiler generates GOT based external references for them. This puts two
extra restrictions on percpu variable definitions. The symbol must be
globally unique even when static and a static percpu variable can't be
defined inside a function. DEBUG_FORCE_WEAK_PER_CPU exists to give generic
code build coverage for these restrictions without building for alpha.
MEM_ALLOC_PROFILING defines a static percpu counter at each allocation call
site and thus can't be built with weak percpu definitions, so it depends on
!DEBUG_FORCE_WEAK_PER_CPU. As allmodconfig enables DEBUG_FORCE_WEAK_PER_CPU,
this knocks MEM_ALLOC_PROFILING out of allmodconfig build coverage.
allmodconfig coverage for MEM_ALLOC_PROFILING is worth more than build
coverage for restrictions which only matter to alpha module builds. Drop
DEBUG_FORCE_WEAK_PER_CPU. Restriction violations will now show up only on
alpha builds.
Link: https://lore.kernel.org/178656406317.2437052.7257990869957704195@slm.duckdns.org
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Gabriele Monaco <gmonaco@redhat.com> [include/rv/da_monitor.h]
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
sort_folio() has a shortcut for moving folios that are no longer evictable
but are still sitting on a generation list. However, this shortcut is
buggy. It does not follow the PG_lru usage convention, and it has a more
serious issue.
Unevictable folios are not threaded on lists[LRU_UNEVICTABLE], so that
folio->lru can be reused to hold folio->mlock_count (see the comment in
lruvec_init()). Hence lruvec_add_folio() skips the list_add() for them,
and every other place that turns a folio unevictable initialises
mlock_count explicitly: lru_add() sets it to 0, __mlock_folio() and
__mlock_new_folio() set it to !!folio_test_mlocked(folio). sort_folio()
sets nothing, and the lru_gen_del_folio() right above it may have already
poisoned folio->lru via list_del(), so mlock_count ends up aliasing
LIST_POISON2, which reads as 0x122, i.e. 290. The result is user
visible. On munlock, __munlock_folio() decrements that bogus count, finds
it still non-zero and bails out before clearing PG_mlocked, so the folio
remains unevictable and the Mlocked accounting stays inflated until the
folio is freed.
The shortcut also touches the LRU flags in the wrong order. It calls
lru_gen_del_folio() while PG_lru is still set, so a concurrent
folio_test_clear_lru() (e.g. compaction, folio_isolate_lru()) can succeed
on a folio that has already been taken off the generation list, which may
lead to unexpected behavior.
So fix it by isolating them as common folios and letting the generic
shrink path cull them. This matches the classical LRU behavior, and there
should be no visible effect on the generic eviction or isolation behavior.
There is no performance concern either, such a folio goes through this
once, and then it is off the generation lists for good.
Link: https://lore.kernel.org/20260812-mglru-mlock-fix-v2-1-a3fec5853c08@tencent.com
Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation")
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Barry Song <baohua@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Brian Geffon <bgeffon@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Oleksandr Natalenko <oleksandr@natalenko.name>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Steven Barrett <steven@liquorix.net>
Cc: Suleiman Souhlal <suleiman@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Yu Zhao <yuzhao@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
We are currently dropping the anon_vma write lock before unmapping the
PTE. Although this is safe, due to us still holding the mmap_write_lock,
its safer and less confusing to switch the order of these two operations.
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-6-ddac39d61c4a@linux.dev
Signed-off-by: Nico Pache (Red Hat) <nico.pache@linux.dev>
Suggested-by: David Hildenbrand <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Fix comment in collapse_scan_pmd() that still described the old
folio_mapcount() > folio_ref_count() check and a "512" false-positive
scenario. The code now uses folio_expected_ref_count() !=
folio_ref_count() which doesn't suffer from the same limitation.
Fix comment in collapse_huge_page() that referenced ptep_clear_flush, when
the code actually uses pmdp_collapse_flush.
Fix comment in __collapse_huge_page_swapin() that referenced the old
function name khugepaged_scan_pmd, now collapse_scan_pmd.
Also clean up some simple typos and stale terminology (mmap_sem ->
mmap_lock, PG_lock -> folio lock, page -> folio, grammar).
We also clarify a comment regarding where the max_ptes_none check is
deferred to in mthp_collapse() from the original collapse_scan_pmd check.
Update all comments that references a function to include parentheses.
[nico.pache@linux.dev: fix outdated comments]
Link: https://lore.kernel.org/1c96e2f3-802f-472b-81e6-4af17a721a3c@linux.dev
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-4-ddac39d61c4a@linux.dev
Signed-off-by: Nico Pache (Red Hat) <nico.pache@linux.dev>
Acked-by: Usama Arif <usama.arif@linux.dev>
Assisted-by: Cursor(claude-sonnet-4):4.6
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Provide a simple helper function to help reduce a often used, and
duplicate pattern across the khugepaged code.
When collapsing to a PMD we need to record a vm_event and the mTHP_stat
event. When doing mTHP collapse we only update the mTHP stat.
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-3-ddac39d61c4a@linux.dev
Signed-off-by: Nico Pache (Red Hat) <nico.pache@linux.dev>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
This change deduplicates the "is this PTE/folio referenced enough to be
considered for a collapse" condition that was repeated in both
__collapse_huge_page_isolate() and collapse_scan_pmd(), extracting it into
a single inline helper function.
Also move the comment and use it as the function header. While we are at
it, updated the comment to clarify that a young pte is a recently accessed
one.
[nico.pache@linux.dev: drop the trivial helper kerneldoc and inline marker per review]
Link: https://lore.kernel.org/9038f552-926b-4c4c-b023-69271f45e3d5@linux.dev
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-2-ddac39d61c4a@linux.dev
Signed-off-by: Nico Pache (Red Hat) <nico.pache@linux.dev>
Acked-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
collapse_control_init_scan()
Patch series "mm/khugepaged: several cleanups", v4.
The following changes stem from a number of reviews during my khugepaged
mTHP support series [1]. Some of these are minor code cleanups, issues or
reviews that we decided to deferred to a followup series, or in the case
of the more major patch of the series, changes [2] Lance Yang attempted
while my series was in-flight and we decided to wait till later to try.
The first 3 patches introduce helper functions to increase code reuse and
readability. This includes a per-scan state clearing function, extracting
the young page check into a helper, and a count_collapse_event() function
to reduce a repetative pattern used across mTHP collapse.
The 4th patch was the byproduct of me throwing Claude at all the comments
in khugepaged verifying and looking for any outdated info.
The 5th patch is based on Lance Yang's commit series [2] trying to extract
the PTE state checking into a helper function. This required a bit of
rewriting due to differences after mTHP collapse was introduced. I also
took into account the changes requested during his patches review cycle.
The remaining 2 patches were review points during my mTHP series that we
agreed can be deferred to a later series.
Thank you to those whos reviews and work I leveraged to achieve these
cleanups.
This patch (of 6):
Extract the repeated clearing of node_load, alloc_nmask, and
mthp_present_ptes into a helper to reduce duplication in
collapse_scan_pmd() and collapse_scan_file(). Althought file scans do not
current use the bitmap, they will in the future, and clearing it now is
harmless.
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-0-ddac39d61c4a@linux.dev
Link: https://lore.kernel.org/20260811-khugepaged_pte_refactor-v4-1-ddac39d61c4a@linux.dev
Link: https://lore.kernel.org/all/20260605161422.213817-1-npache@redhat.com/ [1]
Link: https://lore.kernel.org/all/20251008043748.45554-1-lance.yang@linux.dev/ [2]
Signed-off-by: Nico Pache (Red Hat) <nico.pache@linux.dev>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
lru_gen_look_around() feeds its local 'young' counter into
suitable_to_scan(), which decides whether the current PMD is added to
the bloom filter and checked again on the next aging round.
The folio triggering the look-around is processed at function entry:
test_and_clear_young_ptes_notify() clears the accessed bits of the nr PTEs
it maps, and the function bails out if none of them is young. The loop
that follows therefore never recounts this folio, since its accessed bits
are already cleared. Every other young folio the loop finds is accounted
as a batch (young += nr), where nr is the number of consecutive PTEs it
maps. The triggering folio, however, still contributes a fixed young = 1
regardless of its size -- a leftover from before PTE batching. A large
triggering folio is thus accounted inconsistently with the rest of the
window.
Initialize young to nr so the triggering folio is accounted the same way
as any other young folio batch in the loop.
Note this is a deliberate overestimate, not a measured value. The
test-and-clear helper only reports whether any of the nr PTEs is young,
not how many were accessed, so the true number of accessed PTEs in a large
folio is unknown and can be smaller than nr. Counting the full batch is
intentional: the mm core tracks accessed/dirty state per folio, not per
page, so a per-page count is neither obtainable nor meaningful. The only
consumer is suitable_to_scan(), and the bloom filter it feeds tolerates
error. Overestimating is also the safe direction: at worst a PMD that saw
little access is rescanned, whereas underestimating could skip rescanning
a PMD whose folios are still hot and reclaim them incorrectly. (nr here
is the PTE batch size, not necessarily folio_nr_pages().)
Link: https://lore.kernel.org/20260813061019.49806-1-hui.zhu@linux.dev
Link: https://lore.kernel.org/20260812065933.103627-1-hui.zhu@linux.dev
Fixes: 56e5b60b2114 ("mm: support batched checking of the young flag for MGLRU")
Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
quarantine_size tracks the total number of bytes stored in
global_quarantine[]. It is incremented when per-CPU quarantine objects
are moved into the global quarantine and decremented when a global batch
is evicted by kasan_quarantine_reduce().
kasan_quarantine_remove_cache() also removes objects from the global
quarantine. qlist_move_cache() rebuilds the source batch and updates its
.bytes field, but quarantine_size is not adjusted accordingly.
As a result, quarantine_size remains over-counted by the size of the
removed objects. The stale accounting accumulates across cache removals.
Once the inflated value exceeds quarantine_max_size,
kasan_quarantine_reduce() can evict a batch even though the actual number
of bytes in global_quarantine[] is still below quarantine_max_size,
shortening the quarantine window.
Fix the accounting by recording each batch's size before
qlist_move_cache() and subtracting the number of bytes actually removed
from quarantine_size while holding quarantine_lock.
A KUnit reproducer used during testing observed the over-count grow by
4698864 bytes after one kasan_quarantine_remove_cache() call with the fix
reverted. With this change applied, the over-count did not grow.
Link: https://lore.kernel.org/20260811073332.1351893-1-sh_def@163.com
Fixes: 64abdcb24351 ("kasan: eliminate long stalls during quarantine reduction")
Signed-off-by: Hui Su <sh_def@163.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260808031459.3032812-1-sh_def%40163.com
Reviewed-by: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Currently swap to and from file systems goes through two indirect calls
between the swap ops and the swap_rw method. Reduce this by directly
providing the swap_ops from the file system.
For this refactor swap_fs_submit into a swap_fs_prepare_rw helper that
initializes the iov_iter on the callers stack so that file systems can
call it directly, and use that to initialize file system specific ops in
the NFS and SMB clients, which then get passed to swap_fs_activate.
Link: https://lore.kernel.org/20260723054622.3460249-4-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Chris Li <chrisl@kernel.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Steve French <sfrench@samba.org>
Cc: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Add a new header to declare the swap_iocb, swap_ops and swap_ctx to allow
for swap_ops implementations outside of mm/page_io.c. This will be used
to remove the double indirection for file system-based swap. There is no
functional change, just a move of the declarations.
Note that there already is a swapops.h header, which is totally unrelated
to struct swap_ops. The close naming is a bit unfortunate, but I could
not think of a better name for this header.
Link: https://lore.kernel.org/20260723054622.3460249-3-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Chris Li <chrisl@kernel.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Steve French <sfrench@samba.org>
Cc: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "swap_ops updates", v2.
This series is a follow on to the swap ops series now in mm-unstable.
The first patch reintroduces direct folio writes for synchronous
swap files, the other two remove the double indirect for file
system based swap.
This patch (of 3):
Kairui Song reported that zram benefits from submitting each folio
directly instead of batching up I/O because the classic LRU scanning
benefits from clearing the folio writeback bit in the scan loop.
Accommodate that by kicking off writes for synchronous devices for each
iteration.
Link: https://lore.kernel.org/20260723054622.3460249-1-hch@lst.de
Link: https://lore.kernel.org/20260723054622.3460249-2-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Usama Arif <usama.arif@linux.dev>
Acked-by: Chris Li <chrisl@kernel.org>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Steve French <sfrench@samba.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
