diff options
| author | Gregory Price (Meta) <gourry@gourry.net> | 2026-10-31 22:13:36 -0400 |
|---|---|---|
| committer | Vlastimil Babka (SUSE) <vbabka@kernel.org> | 2026-11-05 10:58:52 +0100 |
| commit | bcfc8286f0cd5a74c827d502f9f1ad408d7eff30 (patch) | |
| tree | b86d779e3307bac2d9c80b4dcee15078b636c7b9 /include/linux/debugobjects.h | |
| parent | e55119ab611a0e47595db7e1d2ac1e69f7a73ff0 (diff) | |
mm/page_alloc: retry allocations that raced with a zonelist rebuild
Onlining the first block of a zone rebuilds every zonelist in place,
shifting the slots after the new zone. The iterator approves a slot,
then the allocator loads the zone from it, so a GFP_KERNEL request can
land in ZONE_MOVABLE and a nodemask request on a node outside its mask:
CPU 0 CPU 1
prepare_alloc_pages()
first_zones_zonelist(nodemask {1})
slot 0 (NORMAL1) approved
online_pages()
build_all_zonelists()
[NORMAL1, NORMAL2, ...]
-> [MOVABLE1, NORMAL1, ...]
get_page_from_freelist()
zone = zonelist_zone(slot 0) /* MOVABLE1 */
allocates from MOVABLE1
zonelist_update_seq detects the rebuild, but only the slowpath's failure
exits check it.
Check it whenever an allocation succeeds too, and if it changed, free
the page and retry, as the memcg charge failure path does. Eligible
pages are dropped as well: zonelists only change when memory comes or
goes, and a retry is simpler than rechecking the page.
Freeing the page does not undo a block the allocation claimed. A
GFP_KERNEL request that lands in a fresh ZONE_MOVABLE converts a free
block to MIGRATE_UNMOVABLE, which stays until the zone's movable free
list runs out and a movable allocation takes it back.
The fast and bulk paths read the seqlock's raw seqcount without waiting:
an ALLOC_NOLOCK caller can be an NMI that interrupted the writer, and on
PREEMPT_RT the seqlock accessor takes the spinlock. A sample taken
mid-write costs a retry, or NULL under ALLOC_NOLOCK.
check_retry_zonelist() returns 0 when compiled out, so a backport
without the hot-add change stays harmless.
Fixes: 11cd8638c37f ("mm, page_alloc: remove stop_machine from build_all_zonelists")
Cc: stable@vger.kernel.org
Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Link: https://patch.msgid.link/20261101021340.2251928-9-gourry@gourry.net
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Diffstat (limited to 'include/linux/debugobjects.h')
0 files changed, 0 insertions, 0 deletions
