linux-toradex.git/mm/page_alloc.c, branch v5.12

mm/memcg: set memcg when splitting page

2021-03-13T19:27:31+00:00

As described in the split_page() comment, for the non-compound high order
page, the sub-pages must be freed individually.  If the memcg of the first
page is valid, the tail pages cannot be uncharged when be freed.

For example, when alloc_pages_exact is used to allocate 1MB continuous
physical memory, 2MB is charged(kmemcg is enabled and __GFP_ACCOUNT is
set).  When make_alloc_exact free the unused 1MB and free_pages_exact free
the applied 1MB, actually, only 4KB(one page) is uncharged.

Therefore, the memcg of the tail page needs to be set when splitting a
page.

Michel:

There are at least two explicit users of __GFP_ACCOUNT with
alloc_exact_pages added recently.  See 7efe8ef274024 ("KVM: arm64:
Allocate stage-2 pgd pages with GFP_KERNEL_ACCOUNT") and c419621873713
("KVM: s390: Add memcg accounting to KVM allocations"), so this is not
just a theoretical issue.

Link: https://lkml.kernel.org/r/20210304074053.65527-3-zhouguanghui1@huawei.com
Signed-off-by: Zhou Guanghui 
Acked-by: Johannes Weiner 
Reviewed-by: Zi Yan 
Reviewed-by: Shakeel Butt 
Acked-by: Michal Hocko 
Cc: Hanjun Guo 
Cc: Hugh Dickins 
Cc: Kefeng Wang 
Cc: "Kirill A. Shutemov" 
Cc: Nicholas Piggin 
Cc: Rui Xiang 
Cc: Tianhong Ding 
Cc: Weilong Chen 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kasan, mm: fix crash with HW_TAGS and DEBUG_PAGEALLOC

2021-03-13T19:27:31+00:00

Currently, kasan_free_nondeferred_pages()->kasan_free_pages() is called
after debug_pagealloc_unmap_pages(). This causes a crash when
debug_pagealloc is enabled, as HW_TAGS KASAN can't set tags on an
unmapped page.

This patch puts kasan_free_nondeferred_pages() before
debug_pagealloc_unmap_pages() and arch_free_page(), which can also make
the page unavailable.

Link: https://lkml.kernel.org/r/24cd7db274090f0e5bc3adcdc7399243668e3171.1614987311.git.andreyknvl@google.com
Fixes: 94ab5b61ee16 ("kasan, arm64: enable CONFIG_KASAN_HW_TAGS")
Signed-off-by: Andrey Konovalov 
Cc: Catalin Marinas 
Cc: Will Deacon 
Cc: Vincenzo Frascino 
Cc: Dmitry Vyukov 
Cc: Andrey Ryabinin 
Cc: Alexander Potapenko 
Cc: Marco Elver 
Cc: Peter Collingbourne 
Cc: Evgenii Stepanov 
Cc: Branislav Rankov 
Cc: Kevin Brodsky 
Cc: Christoph Hellwig 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/page_alloc.c: refactor initialization of struct page for holes in memory layout

2021-03-13T19:27:30+00:00

There could be struct pages that are not backed by actual physical memory.
This can happen when the actual memory bank is not a multiple of
SECTION_SIZE or when an architecture does not register memory holes
reserved by the firmware as memblock.memory.

Such pages are currently initialized using init_unavailable_mem() function
that iterates through PFNs in holes in memblock.memory and if there is a
struct page corresponding to a PFN, the fields of this page are set to
default values and it is marked as Reserved.

init_unavailable_mem() does not take into account zone and node the page
belongs to and sets both zone and node links in struct page to zero.

Before commit 73a6e474cb37 ("mm: memmap_init: iterate over memblock
regions rather that check each PFN") the holes inside a zone were
re-initialized during memmap_init() and got their zone/node links right.
However, after that commit nothing updates the struct pages representing
such holes.

On a system that has firmware reserved holes in a zone above ZONE_DMA, for
instance in a configuration below:

	# grep -A1 E820 /proc/iomem
	7a17b000-7a216fff : Unknown E820 type
	7a217000-7bffffff : System RAM

unset zone link in struct page will trigger

	VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);

in set_pfnblock_flags_mask() when called with a struct page from a range
other than E820_TYPE_RAM because there are pages in the range of
ZONE_DMA32 but the unset zone link in struct page makes them appear as a
part of ZONE_DMA.

Interleave initialization of the unavailable pages with the normal
initialization of memory map, so that zone and node information will be
properly set on struct pages that are not backed by the actual memory.

With this change the pages for holes inside a zone will get proper
zone/node links and the pages that are not spanned by any node will get
links to the adjacent zone/node.  The holes between nodes will be
prepended to the zone/node above the hole and the trailing pages in the
last section that will be appended to the zone/node below.

[akpm@linux-foundation.org: don't initialize static to zero, use %llu for u64]

Link: https://lkml.kernel.org/r/20210225224351.7356-2-rppt@kernel.org
Fixes: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rather that check each PFN")
Signed-off-by: Mike Rapoport 
Reported-by: Qian Cai 
Reported-by: Andrea Arcangeli 
Reviewed-by: Baoquan He 
Acked-by: Vlastimil Babka 
Reviewed-by: David Hildenbrand 
Cc: Borislav Petkov 
Cc: Chris Wilson 
Cc: "H. Peter Anvin" 
Cc: Łukasz Majczak 
Cc: Ingo Molnar 
Cc: Mel Gorman 
Cc: Michal Hocko 
Cc: "Sarvela, Tomi P" 
Cc: Thomas Gleixner 
Cc: 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm/page_alloc: count CMA pages per zone and print them in /proc/zoneinfo

2021-02-26T17:41:00+00:00

Let's count the number of CMA pages per zone and print them in
/proc/zoneinfo.

Having access to the total number of CMA pages per zone is helpful for
debugging purposes to know where exactly the CMA pages ended up, and to
figure out how many pages of a zone might behave differently, even after
some of these pages might already have been allocated.

As one example, CMA pages part of a kernel zone cannot be used for
ordinary kernel allocations but instead behave more like ZONE_MOVABLE.

For now, we are only able to get the global nr+free cma pages from
/proc/meminfo and the free cma pages per zone from /proc/zoneinfo.

Example after this patch when booting a 6 GiB QEMU VM with
"hugetlb_cma=2G":
  # cat /proc/zoneinfo | grep cma
          cma      0
        nr_free_cma  0
          cma      0
        nr_free_cma  0
          cma      524288
        nr_free_cma  493016
          cma      0
          cma      0
  # cat /proc/meminfo | grep Cma
  CmaTotal:        2097152 kB
  CmaFree:         1972064 kB

Note: We print even without CONFIG_CMA, just like "nr_free_cma"; this way,
      one can be sure when spotting "cma 0", that there are definetly no
      CMA pages located in a zone.

[david@redhat.com: v2]
  Link: https://lkml.kernel.org/r/20210128164533.18566-1-david@redhat.com
[david@redhat.com: v3]
  Link: https://lkml.kernel.org/r/20210129113451.22085-1-david@redhat.com

Link: https://lkml.kernel.org/r/20210127101813.6370-3-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Oscar Salvador 
Acked-by: David Rientjes 
Cc: Thomas Gleixner 
Cc: "Peter Zijlstra (Intel)" 
Cc: Mike Rapoport 
Cc: Michal Hocko 
Cc: Wei Yang 
Cc: Zi Yan 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: simplify free_highmem_page() and free_reserved_page()

2021-02-24T21:38:32+00:00

adjust_managed_page_count() as called by free_reserved_page() properly
handles pages in a highmem zone, so we can reuse it for
free_highmem_page().

We can now get rid of totalhigh_pages_inc() and simplify
free_reserved_page().

Link: https://lkml.kernel.org/r/20210126182113.19892-3-david@redhat.com
Signed-off-by: David Hildenbrand 
Reviewed-by: Oscar Salvador 
Reviewed-by: Anshuman Khandual 
Cc: Thomas Gleixner 
Cc: "Peter Zijlstra (Intel)" 
Cc: Mike Rapoport 
Cc: Michal Hocko 
Cc: Wei Yang 
Cc: "Gustavo A. R. Silva" 
Cc: Sam Ravnborg 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: remove unneeded local variable in free_area_init_core

2021-02-24T21:38:31+00:00

Local variable 'zone_start_pfn' is not needed since there's only one call
site in free_area_init_core().  Let's remove it and pass
zone->zone_start_pfn directly to init_currently_empty_zone().

Link: https://lkml.kernel.org/r/20210122135956.5946-6-bhe@redhat.com
Signed-off-by: Baoquan He 
Reviewed-by: Mike Rapoport 
Reviewed-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: simplify parameter of setup_usemap()

2021-02-24T21:38:31+00:00

Parameter 'zone' has got needed information, let's remove other
unnecessary parameters.

Link: https://lkml.kernel.org/r/20210122135956.5946-5-bhe@redhat.com
Signed-off-by: Baoquan He 
Reviewed-by: Mike Rapoport 
Reviewed-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: simplify parater of function memmap_init_zone()

2021-02-24T21:38:31+00:00

As David suggested, simply passing 'struct zone *zone' is enough.  We can
get all needed information from 'struct zone*' easily.

Link: https://lkml.kernel.org/r/20210122135956.5946-4-bhe@redhat.com
Signed-off-by: Baoquan He 
Suggested-by: David Hildenbrand 
Reviewed-by: Mike Rapoport 
Reviewed-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: rename memmap_init() and memmap_init_zone()

2021-02-24T21:38:31+00:00

The current memmap_init_zone() only handles memory region inside one zone,
actually memmap_init() does the memmap init of one zone.  So rename both
of them accordingly.

Link: https://lkml.kernel.org/r/20210122135956.5946-3-bhe@redhat.com
Signed-off-by: Baoquan He 
Reviewed-by: Mike Rapoport 
Reviewed-by: David Hildenbrand 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages

2021-02-24T21:38:29+00:00

Currently we use struct per_cpu_nodestat to cache the vmstat counters,
which leads to inaccurate statistics especially THP vmstat counters.  In
the systems with hundreds of processors it can be GBs of memory.  For
example, for a 96 CPUs system, the threshold is the maximum number of 125.
And the per cpu counters can cache 23.4375 GB in total.

The THP page is already a form of batched addition (it will add 512 worth
of memory in one go) so skipping the batching seems like sensible.
Although every THP stats update overflows the per-cpu counter, resorting
to atomic global updates.  But it can make the statistics more accuracy
for the THP vmstat counters.

So we convert the NR_SHMEM_PMDMAPPED account to pages.  This patch is
consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page
arrival").  Doing this also can make the unit of vmstat counters more
unified.  Finally, the unit of the vmstat counters are pages, kB and
bytes.  The B/KB suffix can tell us that the unit is bytes or kB.  The
rest which is without suffix are pages.

Link: https://lkml.kernel.org/r/20201228164110.2838-6-songmuchun@bytedance.com
Signed-off-by: Muchun Song 
Cc: Alexey Dobriyan 
Cc: Feng Tang 
Cc: Greg Kroah-Hartman 
Cc: Hugh Dickins 
Cc: Johannes Weiner 
Cc: Joonsoo Kim 
Cc: Michal Hocko 
Cc: NeilBrown 
Cc: Pankaj Gupta 
Cc: Rafael. J. Wysocki 
Cc: Randy Dunlap 
Cc: Roman Gushchin 
Cc: Sami Tolvanen 
Cc: Shakeel Butt 
Cc: Vladimir Davydov 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds