summaryrefslogtreecommitdiff
path: root/kernel/debug/debug_core.c
diff options
context:
space:
mode:
authorZi Yan <ziy@nvidia.com>2026-11-10 22:19:25 -0500
committerDavid Hildenbrand (Arm) <david@kernel.org>2026-11-13 11:14:48 +0100
commit7183af7f01c839ecef87840a90c0d8c5843f37d7 (patch)
treeca3a8d530a525c768570b1efd56ae886cd7b427d /kernel/debug/debug_core.c
parent2d9b9352ef885837f246225e9b814a05e9ed2a4a (diff)
mm/huge_memory: prevent losing PG_waiters in __split_folio_to_order()
__split_folio_to_order() calls prep_compound_page() on each new head page to create a compound page after clear_compound_head(). prep_compound_page() uses non atomic __SetPageHead(). It is mostly OK, since the folio is locked and frozen and no one else is supposed to modify the folio's flags. However, code calling migration_entry_wait() can reach the folio by PFN, then access its flags. It is because softleaf_entry_wait_on_locked() used by migration_entry_wait() sets PG_waiters concurrently without taking a reference and the frozen refcount during split cannot block it. Meanwhile, __SetPageHead() does a plain read and write, so PG_waiters atomically set by softleaf_entry_wait_on_locked() will be lost if the set happens after __SetPageHead()'s read and before its write. One example code interleaving: CPU0: __split_folio_to_order(new_order>0) CPU1: do_swap_page() 1. clear_compound_head(new_head) new_head->compound_info = 0 (page_folio(new_head) == new_head now) 2. migration_entry_wait() softleaf_to_folio(entry) page_folio() -> new_head (not the old head any more) 3. prep_compound_page(new_head, new_order) __SetPageHead(new_head) tmp = new_head->flags.f <-- load; PG_waiters == 0 in tmp 4. folio_set_waiters(new_head) set_bit(PG_waiters, &flags.f) enqueue on waitqueue, sleep 5. new_head->flags.f = tmp | (1UL << PG_head) <-- store; overwrites step 4, PG_waiters == 0 again Later, folio_unlock(new_folio) sees no PG_waiters, skips folio_wake_bit(), and CPU1 can remain blocked indefinitely. Fix it by setting PG_head along with other flags in new_folio->flags before smp_wmb() and clear_compound_head(). The remaining compound page metadata is set after clear_compound_head(). In that way, after clear_compound_head() makes new_folio visible, nothing overwrites the PG_waiters from softleaf_entry_wait_on_locked() by modifying new_folio->flags non-atomically. While at it, use data_race() to suppress the KCSAN issue reported by Ye Liu's patch (the second Closes) on reading folio->flags in __split_folio_to_order(). Suggested-by: Kiryl Shutsemau <kas@kernel.org> Reported-by: David Hildenbrand <david@kernel.org> Closes: https://lore.kernel.org/all/DMHE8BEEER1H.2BIY52NXATPIA@nvidia.com/ Reported-by: Ye Liu <ye.liu@linux.dev> Closes: https://lore.kernel.org/all/20261029-data_race_huge_mem-v1-1-112e331291c1@linux.dev/ Fixes: c010d47f107f6 ("mm: thp: split huge page to any lower order pages") Cc: stable@vger.kernel.org Signed-off-by: Zi Yan <ziy@nvidia.com> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Ye Liu <liuye@kylinos.cn> Link: https://patch.msgid.link/20261110-fix-split_folio_to_order-race-v4-1-cc5042f33408@nvidia.com Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Diffstat (limited to 'kernel/debug/debug_core.c')
0 files changed, 0 insertions, 0 deletions