diff options
| author | Vlastimil Babka <vbabka@suse.cz> | 2026-01-23 07:52:44 +0100 |
|---|---|---|
| committer | Vlastimil Babka <vbabka@suse.cz> | 2026-01-29 09:17:28 +0100 |
| commit | e47c897a29491ade20b27612fdd3107c39a07357 (patch) | |
| tree | 31dc1eb2f6dd9db71266722010feaf0fe666381a /include | |
| parent | 4b038a9670154e8bb4832d80f0f2b68b1b812171 (diff) | |
slab: add sheaves to most caches
In the first step to replace cpu (partial) slabs with sheaves, enable
sheaves for almost all caches. Treat args->sheaf_capacity as a minimum,
and calculate sheaf capacity with a formula that roughly follows the
formula for number of objects in cpu partial slabs in set_cpu_partial().
This should achieve roughly similar contention on the barn spin lock as
there's currently for node list_lock without sheaves, to make
benchmarking results comparable. It can be further tuned later.
Don't enable sheaves for bootstrap caches as that wouldn't work. In
order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists
even for !CONFIG_SLAB_OBJ_EXT.
This limitation will be lifted for kmalloc caches after the necessary
bootstrapping changes.
Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid
recursion with kmemleak tracking (thanks to Breno Leitao).
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Reviewed-by: Hao Li <hao.li@linux.dev>
Tested-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Diffstat (limited to 'include')
| -rw-r--r-- | include/linux/slab.h | 6 |
1 files changed, 0 insertions, 6 deletions
diff --git a/include/linux/slab.h b/include/linux/slab.h index 2482992248dc..2682ee57ec90 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -57,9 +57,7 @@ enum _slab_flag_bits { #endif _SLAB_OBJECT_POISON, _SLAB_CMPXCHG_DOUBLE, -#ifdef CONFIG_SLAB_OBJ_EXT _SLAB_NO_OBJ_EXT, -#endif _SLAB_FLAGS_LAST_BIT }; @@ -238,11 +236,7 @@ enum _slab_flag_bits { #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ /* Slab created using create_boot_cache */ -#ifdef CONFIG_SLAB_OBJ_EXT #define SLAB_NO_OBJ_EXT __SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT) -#else -#define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED -#endif /* * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests. |