summaryrefslogtreecommitdiff
path: root/kernel/cgroup
AgeCommit message (Collapse)Author
40 hoursMerge tag 'cgroup-for-7.4-rc2-fixes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup Pull cgroup fixes from Tejun Heo: - A file notification racing the file's removal could re-arm the notification timer after it was shut down, corrupting the timer wheel or firing on freed memory. - The v1 freezer could leave an exiting task frozen forever after its cgroup was thawed or removed. - With memcg built without the block cgroup controller, memcg reclaim waited on writeback and could deadlock against a filesystem freeze. memcg now selects the controller. - cpuset partition state could go wrong when CPUs move between nested partitions or go offline: isolated CPUs leaked or went missing, invalid partitions kept exclusive CPUs or came back with no usable CPU, and expanding a remote partition could fail. Separately, a forked child could inherit a corrupted memory node mask. * tag 'cgroup-for-7.4-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: cgroup/cpuset: Don't revive invalid partition without active CPU on hotplug cgroup/cpuset: Remove redundant WARN_ON_ONCE() in remote_cpus_update() memcg: make MEMCG select BLK_CGROUP cpuset: generalize the validate_partition() interface cpuset: settle isolation state when CPUs are given back to parent cgroup/cpuset: Correctly set isolated_cpus when switching partition state cgroup/cpuset: Protect current->mems_allowed read in cpuset_fork() selftests/cgroup: Fix "test_cpuset_prs.sh -v" failures cpuset: add early empty cpumask check in partition_xcpus_add/del cgroup/cpuset: Reset partition data when an invalid partition stays invalid cgroup/freezer: Fix exiting tasks left frozen forever after rmdir cgroup/freezer: walk exiting tasks in the v1 freezer cgroup: arm the notify timer only while the file exists cgroup/cpuset: Do housekeeping check when isolated child with root parent may get more xcpus cgroup/cpuset: Recompute effective cpumask in update_cpumasks_hier() after calling update_parent_effective_cpumask() cgroup/cpuset: Fix incorrect CPU accounting with parent's effective_xcpus expansion
42 hourscgroup/cpuset: Don't revive invalid partition without active CPU on hotplugRidong Chen
When a CPU hotplug leaves no CPU outside of the partitions for the top cpuset, cpuset_handle_hotplug() clears subpartitions_cpus and gives all the active CPUs back to the top cpuset. cpuset_hotplug_update_tasks() then walks the hierarchy and invalidates the partitions, as subpartitions_cpus is empty. However, an invalid partition visited in the same walk may be revived if its effective_xcpus is not empty. tasks_nocpu_error() doesn't catch it when its CPUs are all offline but it has no task. Its CPUs are put back into subpartitions_cpus, and the valid partitions visited after it are no longer invalidated, as subpartitions_cpus is not empty any more. The top cpuset is left with CPUs overlapping with these partitions. It can be reproduced on a system with 8 CPUs: # cd /sys/fs/cgroup # echo +cpuset > cgroup.subtree_control # mkdir A1 B1 # echo 1 > A1/cpuset.cpus # echo 1 > A1/cpuset.cpus.exclusive # echo 0,2-7 > B1/cpuset.cpus # echo root > B1/cpuset.cpus.partition # echo root > A1/cpuset.cpus.partition # cat A1/cpuset.cpus.partition root invalid (Parent unable to distribute cpu downstream) # echo 0 > /sys/devices/system/cpu/cpu1/online # cat A1/cpuset.cpus.partition B1/cpuset.cpus.partition root root # cat cpuset.cpus.effective 0,2-7 A1 can't take CPU 1, the last CPU of the root cgroup which has tasks. When CPU 1 is offlined, A1 is revived without any active CPU, and B1 stays valid while its CPUs are also in the effective CPUs of the root cgroup, which triggers WARNING: kernel/cgroup/cpuset.c:919 at rebuild_sched_domains_locked Only revive an invalid partition in hotplug if its effective_xcpus has an active CPU, the same as what is required to create a partition with tasks. With that, A1 and B1 are both invalidated in the above case. Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2") Cc: stable@vger.kernel.org # v6.7+ Reported-by: Sashiko <sashiko-bot@kernel.org> Link: https://sashiko.dev/#/patchset/20261114083124.1340140-1-ridong.chen%40linux.dev?part=11 Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
42 hourscgroup/cpuset: Remove redundant WARN_ON_ONCE() in remote_cpus_update()Ridong Chen
remote_cpus_update() warns if the CPUs to be added to a remote partition intersect subpartitions_cpus, and then checks the very same condition right below to set PERR_NOCPUS and invalidate the partition: WARN_ON_ONCE(cpumask_intersects(tmp->addmask, subpartitions_cpus)); if (!capable(CAP_SYS_ADMIN)) WRITE_ONCE(cs->prs_err, PERR_ACCESS); else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) || cpumask_subset(top_cpuset.effective_cpus, tmp->addmask)) WRITE_ONCE(cs->prs_err, PERR_NOCPUS); Sashiko reported that the warning could be triggered by a write to a cpuset control file, before the CAP_SYS_ADMIN check, which would panic a system with panic_on_warn set. The warning was added by commit 86888c7bd117 ("cgroup/cpuset: Add warnings to catch inconsistency in exclusive CPUs") as an assertion that such CPUs have been filtered out before. No way to trigger it with the current interfaces has been found, as these CPUs are rejected when cpuset.cpus.exclusive is written. But the warning is redundant, as the conflict is handled right below anyway. Remove the warning and rely on the PERR_NOCPUS check, the same as remote_partition_enable() already does for this conflict. Reported-by: Sashiko <sashiko-bot@kernel.org> Link: https://sashiko.dev/#/patchset/20261114083124.1340140-1-ridong.chen%40linux.dev?part=6 Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
4 dayscpuset: generalize the validate_partition() interfaceRidong Chen
Refactor validate_partition() to take the parent partition, the new partition root state, the new effective exclusive CPUs and the CPUs to be deleted, instead of a trial cpuset. A NULL parent stands for a remote partition, whose CPUs come from top_cpuset. This makes the interface flexible enough to handle both local and remote partition validation. This also fixes a false PERR_NOCPUS error when expanding a remote partition. The no CPU left for tasks check in partition_cpus_change() is done against the cgroup parent, but the CPUs of a remote partition come from top_cpuset, not from its parent. For example, on a 4-CPU system: echo 2-3 > P/cpuset.cpus echo 2-3 > P/cpuset.cpus.exclusive echo 2-3 > P/C/cpuset.cpus echo 2 > P/C/cpuset.cpus.exclusive echo root > P/C/cpuset.cpus.partition (a task in P) echo 2-3 > P/C/cpuset.cpus.exclusive cat P/C/cpuset.cpus.partition root invalid (Parent is not a partition root) The effective_cpus of P is 3, a subset of the new exclusive CPUs 2-3 of C, and P has a task, so the expansion is rejected. However, P is not a partition root and does not give any CPU to C, and top_cpuset still has CPUs 0-1 for the task. Enabling C directly with exclusive CPUs 2-3 succeeds, as remote_partition_enable() checks top_cpuset instead. The no CPU left for tasks check now uses the given parent instead of parent_cs(cs), and top_cpuset for a remote partition, as that is where its CPUs are taken from. The housekeeping conflict check is folded into validate_partition(). It is skipped when the parent is not a valid partition root, as @cs can't be a valid local partition in that case. Use the new interface for local partition enablement in update_parent_effective_cpumask() and in partition_cpus_change(), replacing the open-coded checks there. The order of the checks is now the same for both paths: empty CPUs, housekeeping conflict, then no CPU left for tasks. Add a test case to test_cpuset_prs.sh for the remote partition expansion above. Fixes: 181c8e091aae ("cgroup/cpuset: Introduce remote partition") Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
4 dayscpuset: settle isolation state when CPUs are given back to parentRidong Chen
partition_xcpus_del() only updates isolated_cpus when the partition that gives back the CPUs has a different state than its parent. CPUs that were further distributed to descendant isolated partitions are not taken into account, so isolated_cpus_release() has to be called afterwards in several places to walk up the ancestors and fix isolated_cpus. This misses some cases and leaks isolated CPUs, as reported by Guopeng. An isolated partition can be invalidated after its parent has already become invalid, and its CPUs are left in isolated_cpus. For example, on a 32-CPU system with all CPUs online and without isolcpus= or nohz_full= boot parameters: cd /sys/fs/cgroup echo +cpuset > cgroup.subtree_control mkdir A echo 0,2-31 > A/cpuset.cpus echo root > A/cpuset.cpus.partition echo +cpuset > A/cgroup.subtree_control mkdir A/B echo 2-3 > A/B/cpuset.cpus echo isolated > A/B/cpuset.cpus.partition cat cpuset.cpus.isolated 2-3 echo 0 > /sys/devices/system/cpu/cpu1/online cat A/B/cpuset.cpus.partition isolated invalid cat cpuset.cpus.isolated 2-3 When A is invalidated, partition_xcpus_del() gives CPUs 0,2-31 back to top cpuset. Both A and top cpuset are root partitions, so isolated_cpus is left untouched although CPUs 2-3 were distributed to the isolated B. B is then invalidated with an invalid parent, and nothing releases CPUs 2-3 either. Make partition_xcpus_del() always set the isolation state of the given back CPUs to the state of the parent that receives them. As the effective_xcpus of a partition covers the CPUs distributed to its descendants, the isolation state of those CPUs is settled at the same time. Boot-time isolated CPUs are still preserved by isolated_cpu_update(). compute_partition_effective_cpumask() invalidates a child partition without going through partition_xcpus_del(), so do the same there: the CPUs of the invalidated child take the isolation state of its parent. As the parent may be shrunk and its effective_xcpus not updated yet, only do so for the CPUs the parent still owns. This is to keep the isolation state correct for now, the in-place invalidation here will be removed later in the partition rework. isolated_cpus_update() assumes the old and new states are different, which no longer holds now that partition_xcpus_del() proceeds regardless of the states. Drop its @old_prs argument and the WARN_ON_ONCE() so that it simply sets the isolation state of the given CPUs, and use it everywhere. The state check in partition_xcpus_add() is dropped as well, as setting the state is a no-op when the states are the same. isolated_cpus_release() is no longer needed and is removed. Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions") Reported-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Closes: https://lore.kernel.org/all/20261105130344.664336-4-guopeng.zhang@linux.dev/ Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
7 dayscgroup/cpuset: Correctly set isolated_cpus when switching partition stateWaiman Long
When update_prstate() is called to switch the partition state from root to isolated or vice versa, the whole effective_xcpus is passed to isolated_cpus_update(). However, if there are child partitions present, some of the effective_xcpus are dedicated to those child partitions and their partition states will remain the same or get invalidated in the case of child root partition with isolated parent in the latter update_cpumasks_hier() call. In the latter case, their exclusive CPUs will be returned to their parent. Therefore isolated_cpus may be set incorrectly with child partitions underneath. Fix that by excluding their exclusive CPUs in computing the correct cpumask of exclusive CPUs used by that partition alone. Add new test cases into test_cpuset_prs.sh to test for such cases. Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions") Cc: stable@vger.kernel.org # v6.8+ Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ridong Chen <ridong.chen@linux.dev> Tested-by: Ridong Chen <ridong.chen@linux.dev> Signed-off-by: Tejun Heo <tj@kernel.org>
7 dayscgroup/cpuset: Protect current->mems_allowed read in cpuset_fork()Waiman Long
Reading mems_allowed of the current task outside of cpuset_mutex critical section should be protected by using mems_allowed_seq seqlock as the nodemask can be updated concurrently leading to a torn read. Such protection is currently not implemented in cpuset_fork() for the !CLONE_INTO_CGROUP case. As a result, the child task may inherit an empty or corrupted mems_allowed nodemask. This violates NUMA memory policy constraints and causes subsequent page allocations to unconditionally fail, potentially leading to crashes or spurious OOM killer invocations for the child task. Fix that by using a read_mems_allowed_begin/read_mems_allowed_retry() loop to protect the mems_allowed read. Fixes: 06f4e9489891 ("cpuset: make sure new tasks conform to the current config of the cpuset") Reported-by: Sashiko <sashiko-bot@kernel.org> Link: https://sashiko.dev/#/bug/linux-601100b4-fa77-45fd-99f9-bfbd4385201f Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ridong Chen <ridong.chen@linux.dev> Signed-off-by: Tejun Heo <tj@kernel.org>
7 daysselftests/cgroup: Fix "test_cpuset_prs.sh -v" failuresWaiman Long
When the -v option is used to run test_cpuset_prs.sh, it activates the use of sched-debug to determine the set of isolated CPUs which will also consider 1-CPU partition as being isolated though they will not be reflected in "cpuset.cpus.isolated". As a result, some of the expected isolated CPUs values have 2 different set of values to correspond to these two ways of determining isolated CPUs. Commit 362212853cd0 ("cgroup/cpuset: Fix incorrect CPU accounting with parent's effective_xcpus expansion") introduces some new test cases which fail when the -v option is used. Update the test cases to make them pass with the -v option as well. Also remove an obsolete comment in update_parent_effective_cpumask() of cpuset.c. Reported-by: Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/lkml/886a562b52b1c5094142d547a8336457@kernel.org Fixes: 362212853cd0 ("cgroup/cpuset: Fix incorrect CPU accounting with parent's effective_xcpus expansion") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ridong Chen <ridong.chen@linux.dev> Tested-by: Ridong Chen <ridong.chen@linux.dev> Signed-off-by: Tejun Heo <tj@kernel.org>
8 dayscpuset: add early empty cpumask check in partition_xcpus_add/delChen Ridong
Add a check for an empty cpumask at the start of partition_xcpus_add() and partition_xcpus_del(). This allows the functions to return early, avoiding unnecessary computation when there is no work to be done. With these changes, partition_xcpus_add() and partition_xcpus_del() can be called even if xcpus is empty. The caller no longer needs to check whether xcpus is empty, eliminating many conditional statements. Signed-off-by: Chen Ridong <chenridong@huawei.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Signed-off-by: Tejun Heo <tj@kernel.org>
8 dayscgroup/cpuset: Reset partition data when an invalid partition stays invalidRidong Chen
When cpuset.cpus or cpuset.cpus.exclusive of a partition root is changed, update_cpumask() and update_exclusive_cpumask() copy the new effective_xcpus computed for the trial cpuset into the cpuset, and call reset_partition_data() only if the partition was valid before and is invalid now: if ((old_prs > 0) && !is_partition_valid(cs)) reset_partition_data(cs); An invalid partition root (old_prs < 0) that stays invalid is not covered. Its effective_xcpus is then left with the trial value, which is computed from user_xcpus() and so falls back to cpuset.cpus when cpuset.cpus.exclusive is cleared. As an invalid partition root does not own any exclusive CPUs, and rm_siblings_excl_cpus() treats a non-empty effective_xcpus of a sibling without cpuset.cpus.exclusive as owned, those CPUs can then no longer be claimed by a sibling: # A1: cpuset.cpus=2-3, cpuset.cpus.exclusive=2-3, root, with tasks # offline CPUs 2-3 -> A1 becomes invalid echo > A1/cpuset.cpus.exclusive cat A1/cpuset.cpus.exclusive.effective # 2-3, expected empty echo 3 > B1/cpuset.cpus.exclusive # -EINVAL Reset the partition data whenever the cpuset is a partition root, valid or not, and ends up invalid. As effective_xcpus may be cleared by that, also force update_cpumasks_hier() to update the subtree, so that the effective_xcpus of the descendants doesn't keep the released CPUs. Add test cases for it. Fixes: de9f15e21c55 ("cpuset: use parse_cpulist for setting cpus.exclusive") Fixes: c5866c9a007d ("cpuset: refactor CPU mask buffer parsing logic") Reported-by: Sashiko <sashiko-bot@kernel.org> Link: https://sashiko.dev/#/patchset/20261101114206.2434398-1-ridong.chen%40linux.dev?part=15 Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
11 dayscgroup/freezer: Fix exiting tasks left frozen forever after rmdirTejun Heo
Since 93618edf7538 ("cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated"), rmdir succeeds once cgroup.procs is empty, but exiting tasks hidden from it can stay in the cgroup until their final context switch. If such a task is frozen in a v1 freezer cgroup, nothing is left to thaw it after rmdir: freezer.state is gone with the cgroup, and a THAWED write to an ancestor clears only CGROUP_FREEZING_PARENT. 8ab6980c7007 ("cgroup/freezer: walk exiting tasks in the v1 freezer") made the freezer walks include exiting tasks. A FROZEN write now also freezes a hidden task in a freezable sleep on its exit path. The fix depends on that commit, as the thaw reaches hidden tasks through the same walk. Since 81933038f5e2 ("pid_namespace: make zap_pid_ns_processes() freezable"), an exiting pid namespace init sleeps that way until a parent outside the namespace reaps a zombie in it. Thaw the cgroup from ->css_killed(), as a THAWED write to it would. A descendant removed earlier can still hold hidden tasks frozen through CGROUP_FREEZING_PARENT. The thaw reaches it too, as its css stays online until its tasks are gone. rmdir removes freezer.state only after ->css_killed() returns, and a write to it takes no cgroup_mutex. A FROZEN write after the thaw would freeze the removed cgroup again, so refuse to freeze a dead cgroup. Verified with a reproducer, including under a frozen parent, from an already removed child and with a racing FROZEN write. Fixes: 93618edf7538 ("cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated") Cc: stable@vger.kernel.org # v7.1+ Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Tao Cui <cuitao@kylinos.cn> Tested-by: Tao Cui <cuitao@kylinos.cn>
12 dayscgroup/freezer: walk exiting tasks in the v1 freezerMani Tofigh
An exiting task can stay frozen after its v1 freezer cgroup is thawed. Since commit a72f73c4dd9b ("cgroup: Don't expose dead tasks in cgroup") the css task walk hides an exiting task once its process has no live threads left. That count drops when another thread of the process exits, which can happen between the freeze and the thaw walk. A pid namespace init hits this easily since it waits freezably in zap_pid_ns_processes() after commit 81933038f5e2 ("pid_namespace: make zap_pid_ns_processes() freezable"): Task A (pid ns init) Task B (A's thread) Task C do_exit() exit_signals() do_exit() zap_pid_ns_processes() write FROZEN /* A frozen */ /* live count 0 */ write THAWED /* A skipped */ A then stays frozen even after SIGKILL: [ 10.775155] task:fz-init state:D stack:28152 pid:65 tgid:65 ppid:1 task_flags:0xc0004c flags:0x00080000 [ 10.775176] Call Trace: [ 10.775180] <TASK> [ 10.775188] __schedule+0x10f1/0x4760 [ 10.775252] schedule+0xd7/0x2c0 [ 10.775265] zap_pid_ns_processes+0x2c7/0x5f0 [ 10.775346] do_exit+0x1bf4/0x2560 [ 10.775403] __x64_sys_exit+0x3d/0x50 [ 10.775414] x64_sys_call+0x156c/0x1570 [ 10.775425] do_syscall_64+0xfb/0x590 [ 10.775450] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 10.775529] </TASK> Use CSS_TASK_ITER_WITH_DEAD in the three v1 freezer walks so they all see the same tasks, as commit 60f21a264930 ("cgroup, sched_ext: Include exiting tasks in cgroup iter") did for sched_ext. With the fix, A is thawed and exits. This bug was identified and confirmed with the help of a deterministic hypervisor currently under development. The C reproducer below however hits it within about 10s. Fixes: a72f73c4dd9b ("cgroup: Don't expose dead tasks in cgroup") Cc: stable@vger.kernel.org # v7.1+ Signed-off-by: Mani Tofigh <mt4022@columbia.edu> Reviewed-by: Tao Cui <cuitao@kylinos.cn> Tested-by: Tao Cui <cuitao@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-11-02cgroup: arm the notify timer only while the file existsMani Tofigh
cgroup_rm_file() clears cfile->kn under cfile->lock and then deletes the notify timer. Since commit 4616120fca7f ("cgroup: add lockless fast-path checks to cgroup_file_notify()"), cgroup_file_notify() checks kn and arms the timer without that lock and can arm it right after it was deleted: Task A (memcg charge) Task B (subtree_control write) cgroup_file_notify() READ_ONCE(cfile->kn) != NULL cgroup_rm_file() cfile->kn = NULL timer_delete_sync() timer_reduce(&cfile->notify_timer) /* timer queued again */ If the css is only hidden, showing it again runs timer_setup() on the still queued timer and corrupts the timer wheel: [ 14.324093] ODEBUG: init active (active state 0) object: 0000000009e526fd object type: timer_list hint: cgroup_file_notify_timer+0x0/0x10 [ 14.324123] WARNING: lib/debugobjects.c:629 at debug_print_object+0x1a4/0x3b0, CPU#1: cgnotify/64 [ 14.329806] CPU: 1 UID: 0 PID: 64 Comm: cgnotify Tainted: G W 7.3.0-17087-gd865f2677c44 #1 PREEMPT [ 14.335109] RIP: 0010:debug_print_object+0x1ae/0x3b0 [ 14.354351] Call Trace: [ 14.354927] <TASK> [ 14.356650] __debug_object_init+0x203/0x3c0 [ 14.361799] timer_init_key+0x2c/0x240 [ 14.362682] cgroup_addrm_files+0x754/0xc40 [ 14.366744] css_populate_dir+0x12e/0x440 [ 14.368636] cgroup_apply_control_enable+0x2f3/0xb60 [ 14.371769] cgroup_subtree_control_write+0x740/0xc10 [ 14.376924] cgroup_file_write+0x19e/0x700 [ 14.381809] kernfs_fop_write_iter+0x337/0x510 [ 14.382818] vfs_write+0x545/0x1250 [ 14.386491] ksys_write+0xfa/0x200 [ 14.390144] do_syscall_64+0xfb/0x590 [ 14.390994] entry_SYSCALL_64_after_hwframe+0x77/0x7f If the css is freed instead, the timer can fire on freed memory. Arm the timer under cfile->lock after checking kn again and keep only the kn and timer_pending() checks outside the lock. With the fix, the timer stays idle once the file is removed. This bug was identified and confirmed with the help of a deterministic hypervisor currently under development. The C reproducer below however hits it within a minute. Fixes: 4616120fca7f ("cgroup: add lockless fast-path checks to cgroup_file_notify()") Cc: stable@vger.kernel.org Signed-off-by: Mani Tofigh <mt4022@columbia.edu> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-11-01cgroup/cpuset: Do housekeeping check when isolated child with root parent ↵Waiman Long
may get more xcpus It is reported by sashiko and Guopeng that when an isolated child with root parent gets more isolated CPUs from the expansion of the parent's effective_xcpus, it is possible that all the active housekeeping CPUs may be gone. So a prstate_housekeeping_conflict() should be performed when that happens. Add the check to make sure that we can't exhaust all the active housekeeping CPUs. Test cases are added to the NOHZ_FULL_TEST_MATRIX of test_cpuset_prs.sh to test that. Reported-by: Guopeng Zhang <guopeng.zhang@linux.dev> Link: https://lore.kernel.org/lkml/ed6fecb4-38e9-4a73-8c1a-7bdd80175a69@linux.dev Fixes: e2ffe502ba45 ("cgroup/cpuset: Add cpuset.cpus.exclusive for v2") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-11-01cgroup/cpuset: Recompute effective cpumask in update_cpumasks_hier() after ↵Waiman Long
calling update_parent_effective_cpumask() It is reported by sashiko that calling update_parent_effective_cpumask() may clobber tmp->new_cpus when update_sibling_cpumasks() needs to be called due to possible change in the parent's effective_cpus value. Therefore the new effective cpumask has to be recomputed after calling update_parent_effective_cpumask() to get a correct value back into tmp->new_cpus. Fixes: f0af1bfc27b5 ("cgroup/cpuset: Relax constraints to partition & cpus changes") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-11-01cgroup/cpuset: Fix incorrect CPU accounting with parent's effective_xcpus ↵Waiman Long
expansion Commit ec0254b9eb49 ("cgroup/cpuset: Release invalid isolated CPUs with a shrunk root partition parent") fixes incorrect isolated_cpus accounting when contraction of a parent's effective_xcpus can cause a child partition to lose isolated CPUs. Expansion of parent's effective_cpus can also allow a child partition to gain more effective_xcpus. However, the current code doesn't handle this case correctly leading to incorrect effective_cpus and isolated_cpus accounting. Fix that by looking for this possible expansion of effective_xcpus and perform the proper accounting action. New test cases are added to test this effective_xcpus expansion scenarios. Reported-by: Guopeng Zhang <guopeng.zhang@linux.dev> Link: https://lore.kernel.org/lkml/20261027123837.100909-1-guopeng.zhang@linux.dev Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-29Merge tag 'cgroup-for-7.4-fixes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup Pull cgroup fixes from Tejun Heo: - cgroup.kill did not kill a process whose thread group leader had exited while other threads still ran, nor those threads once moved to another cgroup, letting a process survive its container being killed - A cgroup.procs write under favordynmods could migrate a newly forked process before fork linked it to its cgroup, corrupting cgroup task lists - cpuset partition fixes for nested hierarchies: invalidation, revival, shrinking and switches between root and isolated could leak isolated CPUs, skip or miscompute the housekeeping check, or leave wrong effective CPUs: - A root partition can no longer be created under an isolated parent or stay valid under one, as disabling it could leave no housekeeping CPUs - Enabling a partition failed when a grandchild held tasks, although the parent had none of its own - Separately, an allocation racing with a cpuset.mems update could drop its mempolicy nodemask and land on the wrong node, and a memoryless node in cpuset.mems triggered a spurious "movable nodes only" warning - Two no-functional-change cleanups of the effective CPU computation * tag 'cgroup-for-7.4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (27 commits) cgroup/cpuset: Integrate compute_partition_effective_cpumask() into compute_effective_cpumask() cgroup/cpuset: Relocate compute_*effective_cpumask() and compute_effective_nodemask() cgroup/cpuset: Release invalid isolated CPUs with a shrunk root partition parent cgroup/cpuset: Release invalid isolated CPUs in compute_partition_effective_cpumask() cgroup/cpuset: Make sure compute_partition_effective_cpumask() is only called for valid partition cgroup/cpuset: Prevent recursive call of remote_cpus_update() via update_cpumasks_hier() selftests/cgroup: Check for a single range nohz_full CPUs before running test matrix cgroup/cpuset: Minor fixes to prstate_housekeeping_conflict() cgroup/cpuset: Fix incorrect effective_cpus bug in update_cpumasks_hier() cgroup/cpuset: Properly compute effective xcpus in update_parent_effective_cpumask() cgroup/cpuset: Find furthest isolated ancestor for invalidated isolated partition cgroup: don't kill migrated processes through their exiting threads cgroup/cpuset: check effective mems for a movable-only configuration cgroup/cpuset: Fix task checking when enabling a partition cgroup: serialize migrations of newly forked processes selftests/cgroup: test cgroup.kill with an exited thread group leader cgroup: fix cgroup.kill for exited thread group leader selftests/cgroup: Skip test_cpuset_prs.sh test if some required CPUs are offline selftests/cgroup: Add tests for nohz_full housekeeping check selftests/cgroup: Add tests for isolcpus housekeeping check ...
2026-10-26cgroup/cpuset: Integrate compute_partition_effective_cpumask() into ↵Waiman Long
compute_effective_cpumask() Update compute_effective_cpumask() to call compute_partition_effective_cpumask() directly if the given cpuset is a valid remote or local partition root. This simplifies the call sites where they are being involved. No functional change is expected. Suggested-by: Ridong Chen <ridong.chen@linux.dev> Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Relocate compute_*effective_cpumask() and ↵Waiman Long
compute_effective_nodemask() Relocate compute_partition_effective_cpumask(), compute_effective_cpumask(), and compute_effective_nodemask() together so that compute_effective_cpumask() and compute_partition_effective_cpumask() can be further integrated together. There is no functional change. Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Release invalid isolated CPUs with a shrunk root partition parentWaiman Long
It is also reported by sashiko that there may be leakages of isolated CPUs when a root partition parent shrinks its effective_xcpus with isolated partition children. Leakage here means that isolated_cpus may contain extra CPUs that are not in any isolated partitions and not boot time isolated. Those CPUs cannot be used in any isolated partitions anymore. There are 2 different cases here. The effective_xcpus becomes empty and the isolated partition should be invalidated in this case. Alternatively some of its effective_xcpus are not in parent's effective xcpus but still have some valid effective_xcpus left. The isolated partition can remain valid. In both cases, the invalid isolated CPUs will have to be released from isolated_cpus if appropriate. Fix that by calling isolated_cpus_release() to release those isolated CPUs from isolated_cpus. New test cases are added to test_cpuset_prs.sh to test this. Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Release invalid isolated CPUs in ↵Waiman Long
compute_partition_effective_cpumask() It is found by sashiko that when a child isolated partition with parent root partition is invalidated in compute_partition_effective_cpumask(), the switching of the isolated exclusive CPUs from child to parent will not be reflected in isolated_cpus leading to CPU leakage. Fix that by calling isolated_cpus_update() when that happens. Also simplify compute_partition_effective_cpumask() by doing child partition invalidation only for the special case of no more effective_cpus left in a populated parent. The other case doesn't actually warrant invalidation at this stage and subsequent iteration in update_cpumasks_hier() and update_parent_effective_cpumask() for that child should handle that. Also drop the make_partition_invalid() helper by inlining it into compute_partition_effective_cpumask() as it is the only caller. A new test case is add to test_cpuset_prs.sh to verify this. Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Make sure compute_partition_effective_cpumask() is only ↵Waiman Long
called for valid partition With visual inspection to make sure that compute_partition_effective_cpumask() is only called for valid partition root, it was found that the function can be called for an invalidated partition near the end of cpuset_hotplug_update_tasks(). When partcmd is set, update_parent_effective_cpumask() is called to either invalidate a valid partition or with partcmd_update to check if an invalid partition can be switched back to valid. In both cases, cpuset_force_rebuild() should be invoked. However, compute_partition_effective_cpumask() should only be called when an invalid partition is switched back to valid. When a partition is invalidated, compute_effective_cpumask() should be called instead. Also add a WARN_ON_ONCE() in compute_partition_effective_cpumask() to ensure that a valid partition root is passed in. Fixes: 72c6303acfa1 ("cgroup/cpuset: Take isolated CPUs out of workqueue unbound cpumask") Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Prevent recursive call of remote_cpus_update() via ↵Waiman Long
update_cpumasks_hier() Guopeng reports that changing the cpumask of a remote partition root may set the effective_cpus of the top_cpuset incorrectly. It is due to the following call sequence: update_cpumask() -> partition_cpus_change() -> remote_cpus_update() -> update_sibling_cpumasks() -> update_cpumasks_hier() -> remote_cpus_update() The second remote_cpus_update() call passes in the cs->effective_xcpus instead of the trialcs->effective_xcpus of the first remote_cpus_update() call. As a result, the second call can revert what has been done in the first call leading to incorrect effective_cpus of the top_cpuset as well as subpartitions_cpus. Fix that by adding a new remote_partition_updating flag into the cpuset structure to indicate that a remote partition is being updated and update_cpumasks_hier() should skip the whole subtree as update_cpumask() or update_exclusive_cpumask() which calls partition_cpus_change() will eventually calls update_cpumasks_hier() for that subtree anyway. This new flag is currently set only in remote_cpus_update() to prevent recursive call of remote_cpus_update() on the same cpuset. A test case is added to test_cpuset_prs.sh to verify this change. Reported-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Link: https://lore.kernel.org/lkml/20261010075947.333549-2-guopeng.zhang@linux.dev Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Fixes: f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make update_cpumasks_hier() handle remote partition") Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Minor fixes to prstate_housekeeping_conflict()Waiman Long
Commit 9527ff78a413 ("cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping check") changes the way prstate_housekeeping_conflict() is being used by considering all the exclusive xcpus to be owned by the partition when doing housekeeping check. However there are two minor issues reported by sashiko with that patch: 1) In partition_cpus_change(), prstate_housekeeping_conflict() can be called with an invalid partition root. In that case, there are no isolated CPUs that are going to be returned back to the system to become housekeeping CPUs, so the calculation of the del_cpus cpumask is meaningless. 2) The checking of del_cpus in prstate_housekeeping_conflict() to see if it can contain an active housekeeping CPU that can be returned to the system ignores the fact that an active CPU not in HK_TYPE_DOMAIN_BOOT HK cpumask will not become an active housekeeping CPU when it is released by an isolated partition. Update the existing code to handle these two issues properly to make prstate_housekeeping_conflict() produce more accurate result. Fixes: 9527ff78a413 ("cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping check") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Fix incorrect effective_cpus bug in update_cpumasks_hier()Waiman Long
With the introduction of cpuset.cpus.exclusive, the computation of effective_cpus differs depending on if a cpuset is a valid partition root or not. In update_cpumasks_hier(), effective_cpus is computed early to check if the current subtree can be skipped. Later on update_parent_effective_cpumask() may be called which can change the partition state from valid to invalid or vice versa. If the partition state changes, the previously computed effective_cpus may no longer be valid. Fix that by recomputing effective_cpus again when that happens. Also add a new test case into test_cpuset_prs.sh to verify this change. Fixes: e2ffe502ba45 ("cgroup/cpuset: Add cpuset.cpus.exclusive for v2") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Properly compute effective xcpus in ↵Waiman Long
update_parent_effective_cpumask() Since the introduction of cpuset.cpus.exclusive in commit e2ffe502ba45 ("cgroup/cpuset: Add cpuset.cpus.exclusive for v2"), a compute_effective_exclusive_cpumask() helper which is later renamed to compute_excpus() is added to compute the effective xcpus of a partition root. However update_parent_effective_cpumask() is not updated in all proper places to use the new helper to check if a given cpuset should remain a valid partition root. This can cause a cpuset to incorrectly remain a valid partition root even if no effective xcpus is left. Fix that problem by using compute_excpus() to compute the effective xcpus into delmask in all cases except partcmd_disable, partcmd_invalidate or partcmd_update with newmask set and use it for tasks_nocpu_error() check as well as checking its emptiness. This enables us to catch invalid partition root more accurately in update_parent_effective_cpumask(). In the case of partcmd_update with newmask set, the passed in new effective xcpus value will directly be used in the tasks_nocpu_error() check without relying on old data in the cpuset. For the partcmd_update with NULL newmask case, the cpu exclusivity test there is also relaxed to be performed only if exclusive_cpus is empty. Some test case are added to test_cpuset_prs.sh for invalidating a partition root due to its effective xcpus becoming empty as well as making sure that an invalid partition can only be revived when exclusive CPUs are available agin. Fixes: e2ffe502ba45 ("cgroup/cpuset: Add cpuset.cpus.exclusive for v2") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Tested-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-26cgroup/cpuset: Find furthest isolated ancestor for invalidated isolated ↵Waiman Long
partition Commit 46b885416c27 ("cgroup/cpuset: Fix a missing isolated_cpus update in update_cpumasks_hier()") searches for the closet partition root ancestor to determine if the isolated CPUs in a to be invalidated isolation partition should be removed from isolated_cpus or not. As reported by sashiko, this method may not work when there is a deeply nested isolated partition hierarchy. For instance, Consider a nested hierarchy A0 -> A1 -> A2 -> A3 where all four are PRS_ISOLATED: A0: effective_xcpus = 0-4 A1: effective_xcpus = 1-4 A2: effective_xcpus = 2-4 A3: effective_xcpus = 3-4 When A1 shrinks its exclusive_cpus from 1-4 to 1-2, A2 will update it effective_xcpus to 2. A3 will find its closet partition root ancestor A2 and decides that its effective_xcpus 3-4 should be removed from isolated_cpus which is incorrect. Address this corner case by adding a new isolated_cpus_release() helper to replace the existing ancestor_partition() helper. This new helper will search for the furthest isolated partition root ancestor to decide if some or all of its isolated CPUs should be removed from isolated_cpus and also perform the removal when appropriate. The is_remote_partition() and is_local_partition() helpers are also moved up so they can be used in earlier functions. There are other pre-existing problems that will still prevent the above test case to work correctly. These problems will be addressed in subsequent patches. Fixes: 46b885416c27 ("cgroup/cpuset: Fix a missing isolated_cpus update in update_cpumasks_hier()") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-23Merge tag 'drm-next-2026-10-21' of https://gitlab.freedesktop.org/drm/kernelLinus Torvalds
Pull drm updates from Dave Airlie: "Highlights: - core: a lot of atomic refactoring in progress, panic got split into a core/helpers model - amdgpu: enables HDMI 2.1 FRL and other good things finally, UAlink over Ethernet support - xe: has a bunch of CRI support, ras error for bad vram, memory offlining support. - msm: enables a bunch more GPUs. Otherwise it's a the usual mountain of stuff all over the place. Details: rust: - I/O tree shared with driver-core - DMA tree shared with driver-core - allow aux registration data closures to return covariant sub-fields - Implement Send and Sync for MapleTree - add pci resource flags and IORESOURCE_MEM_64 drm/rust: - add dma_fence abstraction - init DRM registrations and their data in place - drop unused flags argument to registration vgaarb: - rework API to pass a private ptr instead of pci device sound: - adapt to omapdrm changes gem: - add callback for handle count going to 0 gpusvm: - share one HMM fault and keep single mappings inline pagemap: - Prevent CPU stalls during unbounded pagemap teardown dma-fence: - fix docs vga_switcheroo: - disconnect from fbcon - update fbcon from within fbdev emulation atomic: - move DRM from state reset callbacks to atomic_create_state - remove drm_simple_encoder_init panic: - split into core and helpers buddy: - improve dirty tracking clients: - improve vmap handling - fbdev: Move helpers from drm_kms_helper.ko to drm_client_lib.ko display: - export HDMI SCDC status data via debugfs - rework SCDC for better HDMI 2.0 support - dp/mst: improve DPCD compliance edid: - parse AMD VSDB - parse ALLM/VRR capabilities - support detecting panel size log: - add drm_warn_ratelimited sched: - cleanup interface - add missing locking - fix run-queue cleanup - fix locking sysfb: - remove obsolete quirk for Lenovo D330 - fix panel orientation for non-native display modes - drop drm_class_device_register colorop: - provide DRM_COLOR_OP_FIXED_MATRIX bridge: - analogix: Support eDP lane mapping - ti-sn65dsi83: Support LVDS Channel B on SN65DSI84 - Select DRM_PANEL_BRIDGE for various drivers that require it - add chrontel ch7218a support panel: - panel-edp: Support MNE007QS3-F, TM140VDXP15, and KD116N36-30NB-A001 - samsung-s6d16d0: Use mipi_dsi_*_multi() functions - support Ilitek ILI7836A OLED plus DT bindings - support Novatek NT36532 plus DT bindings - convert several drivers to managed cleanup - fix Kconfig selections - edp: - Add support for BOE NV140FHM-T0A - Support CSOT MNE007QB1-1 - NV140WU2-M70 - ebbg-ft8719: Fix MODULE_LICENSE() string; Improve initialization - samsung: Share test-key helpers - ilitek-ili9882t: Support Ilitek IL79600A plus DT bindings - simple: Support Truly TFT320240-229-E plus DT bindings; Support BOE DV215FHM-R01 plus DT bindings - Refactor struct drm_panel to contain struct drm_bridge plus fallout in various panel drivers - Add support for BOE Novatek NT51021, Retroid Pocket 6, Raydium RM69220 DDIC panels, Sharp LQ138P1JX61 eDP, BOE NV140FHD-N43, AUO B140HAN04.0, AUO 0x0db3, panels. amdgpu: - use AMD VSDB for freesync - implement YUV-to-RGB with fixed-matrix colorop - enable FRL for HDMI - HDMI 2.1 freesync support - HDMI 2.1 ALLM support - HDMI 2.1 VRR support - UALoE (UALink over Ethernet) support - add syncobj_points to userq signal ioctl - drop unused kunit tests - NBIO/GC/MMHUB/SMU updates/fixes - GPU spatial partitioning updates - display kunit fixes - userq fixes - RGB quantization fixes - DC RMCM refactor - DP MST fixes - USB4 fixes - add new trace events - Rework display instance handling - stack frame size fixes for various compilers/configs - VCN 4/5 reset fixes - freesync updates - cursor refactor - Fix direct scanout alpha on some DRM_FORMATs - Expand Mac reserved memory workaround - Make runtime pm more fine grained amdkfd: - updates and fixes - GC 10 CU occupancy support radeon: - eDP fix for DCE3.2 iMacs xe: - add drm_ras netlink error event support - add memory page offlining support - introduce error threshold to drm_ras - fix stale GTT mappings on unmap - reject page faults for non-page fault mode scratch VM - enable CRI package and VRAM temperatures - introduce cold reset recovery - handle CRI+ power interval encoding - VF migration for multi-queue - handle exec queue teardown after hot unplug - alerts and controller enabling modifications - use IBCv3 on PTL - VRAM health check - report GT TDR and VM rebind faults using SIGID - add multi_queue_active_lrca debugfs - shrinker: Runtime PM reference for non-system memory - guc: Add simple KLV decoder helpers - debugfs: Expose per-engine idle status - Keep walking on SVM eviction failure - Nuke PTs only after unlinking contested VMAs - limit sg segment size to PAGE_SIZE on Xen PV - keep VF LMEM bar size low if no VFs - CPU binds and ULLS on migration queue - New PTL PCI ID - Enable PXP for NVL-S - Do not dma_fence_put zero fences in err_out path - Refresh TTM LRU on SVM prefetch - export device UID in sysfs i915: - rename GEN6_PCODE_DATA to match spec - fix incorrect RCU teardown order - fix potential UAF in command parsing - avoid WARN splat during RPS boosting race - use simpler min functions - Fixes to make i915 work well with PREEMPT_RT i915/display: - Enable CMRR (Content Match Refresh Rate) in fixed-RR VRR path - Switch DG2 MPLL to DPLL framework - DP-HDMI PCON DSC fixes - limit invalid joiner combinations - CDCLK related fixes and refactor around PCODE integration - Gate UHBR SST SDP splitting on sink caps - add HDMI TMDS audio bandwidth check - vswing/preemph override - enable AS SDP Skip frames - Handle negative configuration index on DP link caps for robustness - Preparation for enabling SDR plane color pipeline - An eDP quirk for HP Pavilion Plus - Introduce PME (Power Management Event) support on display hotplug - Handle VSC SDP revision 7 in unpack msm: - CI: - Dropped sc7180 trogdor lazor from the CI - Core: - Fixed msm_ioremap for resources without a name - Changed msm.separate_gpu_kms to default to auto selection - GPU: - Enable CLX (Current Limit eXtension) for gen8 to avoid exceeding limits on CX/GX rails - New GPUs: a830 (Pakala), a845 (Maili), a850 (Hawi), x2-85 (Mahua) - Fix VBIF halt mask on a306, which was preventing runpm suspend - Enable THP support for GEM buffers - Add modparam to disable waiting on fences in shrinker - Drop use of drm_prime_sg_to_page_array() - a5xx: don't read the GPMU registers on GPUs without a GPMU - a2xx/a3xx/a5xx: hangcheck progress detection - a6xx: gmuwrapper runpm fix - Calculate reported GMEM size based on active slices - Split a6xx/a7xx/a8xx catalog - DP: - Picked up some preparation patches for the MST support - Fixed reported status of the link training - DPU: - Fixed CRTC bandwidth computation - Fixed DSC programming - Fixed encoder disablement path - DSI: - Fixed VM leak in error path - Indention fix - MDP5: - Added MSM8952 configuration - Dropped incorrect cursor pipe on MSM8976/56 - HDMI: - Small fixes nouveau: - improve runtime PM on R570 GSP firmware - various fixes throughout the driver - dispnv50: support 2.147 GHz pixel clock in GB20x nova: - add initial uapi concept - add initial PRAMIN access support - add GpuMm centralized memory manager - add GPU VM and BAR1 support - add GPU interrupts and GSP events support - GSP messaging fixes tyr: - use bitfield! - zero unsupported and newly added GPU info fields - limit doorbell array to 17 entries exynos: - FIMD fixes - Fix clock leak on resume error path - Fix Sparse warnings by adding missing __iomem annotation - VIDI cleanup - Replace kmalloc() and copy_from_user() with memdup_user() amdxdna: - fix BO handling - Support AIE2 rev 9 - Fix UAF access in scheduler fences - Update workqueue interface - expose busy time via fdinfo ast: - support 256-byte EDID hyperv_drm: - remove support for pre win10 hosts komeda: - fix GLB_CORE_ID usage omap: - report HDMI hotplug events to ASoC HDMI codec panthor: - provide gpu_cache_flush tracepoint - improve dma_fence signalling latency - improve locking - export scheduling information via debugfs - Validate user-space input against firmware limits qaic: - reject BOs that exceed maximum page count - add missing include statements verisilicon: - fix hardware cursor offsets vkms: - implement YUV-to-RGB with fixed-matrix colorop gma500: - avoid unsupported ARGB formats in primary plane hibmc: - handle vblank interrupts correctly ivpu: - add DRM_IVPU_CAP_CMDQ_SET_PRIORITY - support command-queue priority ioctl - Simplify NULL-pointer handling in cleanup code arcpgu: - replace simple-display pipeline with regular atomic helpers aspeed: - replace simple-display pipeline with regular atomic helpers gm12u320: - replace simple-display pipeline with regular atomic helpers imagination: - various improvement to firmware interaction - add job to pvr_job_submit_fw trace event mali-dp: - fix runtime PM leak mcde: - replace simple-display pipeline with regular atomic helpers p111: - replace simple-display pipeline with regular atomic helpers - drop unsupported alpha formats from primary plane rcar-du: - support Renesas R-Car V4H DSC - dsi: Support larger transfers with AXI memory - Support RZ/G3L SoC repaper: - replace simple-display pipeline with regular atomic helpers rockchip: - dsi: improve PHY timings ssd130x: - support Solomon SSD1351 OLED plus DT bindings - ssd133x: upgrade color format to RGB565 - ssd132x: Set correct hardware window when clearing screen tve200: - replace simple-display pipeline with regular atomic helpers xen: - replace simple-display pipeline with regular atomic helpers v3d: - improve job handling under perfmon vc4: - reliably reset V3D - drop brcm,vc4-v3d compatible virtio: - fix damage handling - Support ARGB8888 for virtio primary planes. host1x: - Support partitioning imx: - Replace simple-display with regular atomic helpers mxsfb: - Support bus formats plus DT bindings tegra: - improve syncobj handling on submit panfrost: - add Mali-G68 support - improve GPU reset handling rockchip: - add HDMI 2.0 support to dw-hdmi-qp - improve HPD handling" * tag 'drm-next-2026-10-21' of https://gitlab.freedesktop.org/drm/kernel: (2324 commits) drm/adp: Switch to drm_atomic_helper_connector_create_state drm/malidp: Convert to atomic_create_state drm/vc4: Switch to drm_atomic_helper_connector_create_state drm/sysfb: Switch to drm_atomic_helper_connector_create_state drm/solomon: ssd130x: Switch to drm_atomic_helper_connector_create_state drm/imx: ipuv3: Switch to drm_atomic_helper_connector_create_state drm/hyperv: Switch to drm_atomic_helper_connector_create_state drm/bridge: megachips: Switch to drm_atomic_helper_connector_create_state drm/bridge: lontium-lt8912b: Switch to drm_atomic_helper_connector_create_state drm/arm: komeda: Switch to drm_atomic_helper_connector_create_state drm/appletbdrm: Fix incorrect error message on encoder attach failure drm/sun4i: hdmi: Convert to drmm_connector_hdmi_init() drm/rockchip: rk3066_hdmi: Advertise HDMI 1.4 capabilities drm/bridge: lontium-lt9611: Advertise HDMI 1.4 capabilities drm/bridge: ite-it66121: Advertise HDMI 1.2 capabilities drm/bridge: ite-it6263: Advertise HDMI 1.3 capabilities drm/bridge: ite-it6263: Drop redundant .mode_valid hook drm/bridge: inno-hdmi: Advertise HDMI 1.2 capabilities drm/bridge: adv7511: Advertise HDMI 1.2 capabilities drm/sched: Document entity->stopped is protected by entity->lock ...
2026-10-23cgroup: don't kill migrated processes through their exiting threadsSasha Levin
Since commit da13e0e72a73 ("cgroup: fix cgroup.kill for exited thread group leader"), __cgroup_kill() visits every thread in the cgroup and sends SIGKILL with PIDTYPE_TGID, so each visited thread takes down its whole thread group. A thread that has started exiting can stay in a cgroup after its process has been moved out of it. cgroup_migrate_add_task() skips PF_EXITING tasks, and the thread is only unlinked by cgroup_task_dead() after its final context switch. mm_release() clears the thread's clear_child_tid word and wakes the joiner well before that, so pthread_join() can return while the thread is still in the old cgroup. If cgroup.kill is written in that window, __cgroup_kill() finds the exiting thread and complete_signal() passes the group-wide SIGKILL on to a live thread, killing a process that already left the cgroup. The cgroup selftest test_stress.sh runs into this: test_cgcore_proc_migration() joins its threads, moves itself back to the root cgroup, and cg_destroy() falls back to cgroup.kill on the old cgroup when rmdir() fails with EBUSY, so test_core kills itself. KernelCI has been reporting "with_stress.sh: line 94: ... Killed" on several arm64 boards since the commit reached linux-next. Keep visiting every thread, but go back to sending SIGKILL to the thread that was found. A SIGKILL sent to a live thread still kills its whole thread group, so both cases fixed by that commit stay fixed, and task_can_queue_signal() drops one sent to a thread that is already exiting. Fixes: da13e0e72a73 ("cgroup: fix cgroup.kill for exited thread group leader") Cc: stable@vger.kernel.org # v5.14+ Link: https://lore.kernel.org/all/179272264722.329188.4773475583351189022@6e52bff8000c/ Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-23Merge tag 'modules-7.4-rc1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/modules/linux Pull module updates from Petr Pavlu: - Ignore local labels and mapping symbols during module load These symbols were already ignored during kallsyms address-to-symbol lookups. Filter them out earlier during module loading to reduce lookup overhead and prevent them from appearing in /proc/kallsyms. - Fix potentially incorrect access of SetOnce<T> when setting Rust module parameters. - Fix a potential read of an uninitialized count when printing taint information for unloaded modules. - Allocate the kmod argument array using kmalloc_array() in preparation for conversion to type-aware allocators. - Clean up includes in linux/kmod.h Add explicit includes of linux/umh.h to files that use usermode helpers. Then clean up the #include list in linux/kmod.h, notably removing the compatibility linux/umh.h include. - Remove the error-injection.h include from linux/module.h Add explicit includes of linux/error-injection.h to files that use the ALLOW_ERROR_INJECTION() macro and currently rely on it being provided indirectly via linux/module.h. Then remove the unnecessary linux/error-injection.h include from linux/module.h. - Replace __MODULE_STRING() with __stringify() in various drivers in preparation for removing the former macro. * tag 'modules-7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/modules/linux: drivers: Replace __MODULE_STRING() with __stringify() module: Fix uninitialized count read of unloaded taint tracking record rust: module_param: Fix potentially incorrect access of `SetOnce<T>` module: Remove the error-injection.h include from linux/module.h drm/xe: Include error-injection.h for ALLOW_ERROR_INJECTION() drm/i915: Include error-injection.h for ALLOW_ERROR_INJECTION() syscalls: Include error-injection.h for ALLOW_ERROR_INJECTION() net: Include error-injection.h for ALLOW_ERROR_INJECTION() block: Include error-injection.h for ALLOW_ERROR_INJECTION() module: Bring includes in linux/kmod.h up to date time/jiffies: Include linux/sysctl.h for proc_int_u2k_conv_uop(), ... umh, treewide: Explicitly include linux/umh.h where needed modules/kmod: Allocate argv with kmalloc_array() module/kallsyms: Ignore local labels and mapping symbols during module load module/kallsyms: Refactor current symbol filtering into is_core_symbol() module/kallsyms: Rename is_mapping_symbol() to is_ignored_kernel_symbol()
2026-10-23cgroup/cpuset: check effective mems for a movable-only configurationGregory Price (Meta)
check_insane_mems_config() warns and enables cpusets_insane_config_key when a cpuset's tasks can allocate only from movable memory. In cgroup v2 the request may name a memoryless node, which has no zones and cpusets reports it as a movable-only nodemask. For example, with node 2 a CPU-only node: # echo +cpuset > /sys/fs/cgroup/cgroup.subtree_control # mkdir /sys/fs/cgroup/test # echo 2 > /sys/fs/cgroup/test/cpuset.mems # cat /sys/fs/cgroup/test/cpuset.mems.effective 0-1 # dmesg Unsupported (movable nodes only) cpuset configuration detected (nmask=2)! This is a bit confusing, especially consider mems.effective ends up with a valid nodemask by-design (cpuset.mems is treated as-if it were empty). Check each new effective mask in update_nodemasks_hier() instead, as the hotplug path already does. The allocator's bail-out tests the task's mems_allowed, which comes from the effective mask. A request naming a movable-only not present in the parent's nodemask no longer warns (since, again, the cpuset.mems gets overridden and treated as empty). However, it will warn if the parent's mems include the node later. For Example, with node 3 a movable-only node: # echo 0 > /sys/fs/cgroup/test/cpuset.mems # echo 3 > /sys/fs/cgroup/test/child/cpuset.mems # cat /sys/fs/cgroup/test/child/cpuset.mems.effective 0 # dmesg { empty } # echo 0,3 > /sys/fs/cgroup/test/cpuset.mems # cat /sys/fs/cgroup/test/child/cpuset.mems.effective 3 # dmesg Unsupported (movable nodes only) cpuset configuration detected (nmask=3)! Fixes: 8ca1b5a49885 ("mm/page_alloc: detect allocation forbidden by cpuset and bail out early") Signed-off-by: Gregory Price (Meta) <gourry@gourry.net> Reviewed-by: Waiman Long <longman@redhat.com> Reviewed-by: Ridong Chen <ridong.chen@linux.dev> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-23Merge tag 'mm-stable-2026-10-19-21-33' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull MM updates from Andrew Morton: - "selftests/mm: improve khugepaged coverage" (Kiryl Shutsemau): Give khugepaged a proper workout. Extend the tests to all supported mTHP orders, check exactly which windows collapsed, and make scan completion deterministic. Race collapse against faults, pinning, fork and remap; verify that the contents survive. Add coverage for whole-page-table zaps and memory pressure too. - "selftests/mm: separate GUP microbenchmarking from functional testing" (Sarthak Sharma): Untangle GUP benchmarking from correctness testing. Move the benchmark into tools/mm, add a standalone harness-based GUP selftest, and move shared helpers into tools/lib/mm. Give each tool one job instead of mixing timing measurements with correctness checks. Cover the mapping, pinning and batch-size combinations explicitly. - "mm: Switch device DAX to section-based vmemmap optimization" (Muchun Song): Apply section-based vmemmap optimization to device DAX. Rework the backing metadata layout to save memory while preserving the handling of device memory hotplug and its different alignment and lifetime requirements. - "mm: Introduce section-based vmemmap optimization for HugeTLB" (Muchun Song): Cut the memory consumed by HugeTLB's struct page array with section-based vmemmap optimization. Arrange the metadata so large huge pages need fewer backing pages, while keeping initialization, teardown and the various hugepage configurations straight. - "mm/huge_memory: clean up and decouple the anon and file split helpers" (Kairui Song): Separate anonymous and file-backed folio splitting, which have rather different locking and bookkeeping requirements. Move the work into separate helpers, drop dead checks and tidy swap-cache handling. Fix error paths, including one that could leave a folio frozen. Keep the swap cache cases separate from file-cache splitting. - "mm: Unconditional per-VMA locks and cleanups" (Dave Hansen): Make per-VMA locking unconditional and clean up the old conditional code around it. This removes configuration-dependent plumbing and lets callers depend on one locking model rather than carrying both cases along. - "memcg: remove the v1 soft limit" (Shakeel Butt): Remove the old cgroup v1 memory soft-limit machinery. This has long since been superseded by the newer memory-control interfaces; hauling around a separate reclaim scheme for it no longer buys us much. - "read proc/pid/smaps_rollup under per-vma lock" (Suren Baghdasaryan): Read /proc/PID/smaps_rollup using per-VMA locks where possible, rather than holding mmap_lock across the entire walk. Retain the contention fallback and add tearing tests. This considerably reduces latency when another thread is busy changing its mappings. - "mm/damon: introduce data access-as-a-data attribute" (SJ Park): Teach DAMON to expose memory access observations as data attributes, making them available to the probe and filtering machinery. Extend the interfaces, documentation and tests so the new information can be used rather than merely collected. - "Remove PG_private by using page/folio->private checks instead" (Zi Yan): Get rid of PG_private and use the folio's private pointer to say whether private data is attached. Convert the filesystem users, including erofs, f2fs and md, and clean up the old accessors and flags. There are a few special cases, notably swapcache and hugetlb, which need explicit care. - "mm: make userland page table freeing RCU-safe" (Lorenzo Stoakes): Make freeing userspace page tables safe against RCU readers. Sort out the page-table lifetime and locking rules so walkers cannot stumble over freed tables while mappings are being torn down. - "make unused huge shrinker memcg aware" (Qi Zheng): Make shmem's unused-huge-folio shrinker aware of memory cgroups, so reclaim can find those folios under the right memcg. Wire up the per-memcg tracking and its lifetime rules, and give the shrinklist its own lockdep class to avoid spurious dependency reports. - "mm: Drop pxd_ERROR()" (Anshuman Khandual): Remove the old pxd_ERROR() page-table checks and their architecture-specific baggage. Simplify the common page-table interfaces and callers now that these checks no longer serve a useful purpose. - "memcg: group struct fields by access pattern" (Shakeel Butt): Rearrange memcg's hot and cold fields according to how they're actually accessed. Keep frequently updated counters and their neighboring state from fighting over cachelines and reduce needless cacheline traffic. - "mm/damon/core: fix the size charged for a filter-trimmed region" (Donggeun Yoo): Fix DAMOS quota accounting when a filter trims a monitoring region. Charge only the part left after filtering, not the original region, and add KUnit coverage. Previously the excluded memory could eat into the quota and make schemes run more slowly than requested. - "mm/damon: add kunit and selftests for probes and probe weights" (SJ Park): Add more tests for DAMON probes and their weights, including validation and the results reported to userspace. This is mostly about catching regressions in the newer probe infrastructure before they become harder to diagnose. - "mm: memcontrol: constify the read side of the read side of the memcg API" (Tal Zussman): Mark the read-only parts of the memcg interface const, from folio and lruvec accessors to counters and swap helpers. Most of this is mechanical, but it makes the contracts clearer and lets the compiler catch accidental writes. - "mm/damon/vaddr: support {prep,apply}_probes" (SJ Park): Support the new prepare/apply probe operations in DAMON's virtual-address backend. This brings vaddr monitoring into line with the probe infrastructure and makes the added probe types useful there too. - "mm/mglru: clean up folio counters and flag usage" (Kairui Song): Simplify MGLRU's folio counters and flag handling. Remove redundant accounting paths and make the state transitions easier to follow without changing the intended reclaim policy. - "arch, mm/execmem: resolve confusion about set_direct_map_valid_noflush()" (Mike Rapoport (Microsoft)): Clarify and straighten out the set_direct_map_valid_noflush() interface across architectures and execmem. Make the requested mapping changes and the callers' expectations agree, rather than relying on subtly different architecture behavior. - "mm: zswap: free cold writeback folios promptly" (Alexandre Ghiti): Free zswap writeback folios as soon as their disk write completes. They are cold temporary buffers, so putting them on the LRU only makes reclaim rediscover them later. Extend swap-cache dropbehind and let zswap keep these folios off the LRU altogether. - "mm: gup: cleanup gup_fast call chain" (Aristeu Rozanski): Clean up the GUP-fast call chain and simplify how the fast-path helpers pass their arguments and results around. Keep the page-pin behavior the same while making the call sequence easier to maintain. - "mm: optimize zone-device memmap initialization" (Li Zhe): Speed up initialization of zone-device struct pages. Avoid redundant work when setting up large device-memory ranges, where touching every descriptor can get expensive. - "mm/zswap: shrink zswap_entry via a pool id" (Jianyue Wu): Reduce the size of zswap_entry by storing a compact pool identifier instead of a full pointer. Keep the pool lookup and lifetime handling correct while saving per-entry metadata. - "mm/damon: support access monitoring of hugetlb-backed memory" (Krishna Iyer): Let DAMON monitor hugetlb-backed memory too. Extend the relevant access checks and add coverage so those mappings no longer fall through a hole in the monitoring machinery. - "mm/execmem: fixes and cleanups for the ROX cache" (Mike Rapoport): Tidy execmem's read-only-executable cache and fix its handling of the awkward allocation and release cases. The aim is to make the cache's ownership and permission transitions more predictable. - "zsmallc: remove old object read API" (Sergey Senozhatsky): Drop zsmalloc's old object-read API and move its remaining users to the supported access mechanism. This removes a duplicate interface which we no longer need. - "mm/mempolicy: stop copying state in the interleave paths" (Gregory Price): Stop copying mempolicy state in the interleave paths when a reference to the existing policy will do. This avoids extra work and makes the ownership rules less obscure. - "mm/mglru: speed up inc_min_seq() and fix cold/hot inversions" (Barry Song): Make MGLRU's generation advancement cheaper and fix cases where cold and hot folios end up in the wrong order. Adjust the counter and list handling so reclaim doesn't keep passing over the better eviction candidates. - "mm: Fix MF_DELAYED handling on memory failure" (Lisa Wang): Treat MF_DELAYED as a partially completed memory-failure operation, not an outright failure. Propagate that result correctly through shmem and the generic handler, fix the extra-reference accounting, and add tests for the resulting SIGBUS behavior. - "tools/mm/page_owner_sort: fix --sort, add module filter, improve usage" (Ye Liu): Fix page_owner_sort's --sort handling, add module-based filtering and make the usage output more helpful. This makes the tool less frustrating when tracking down who is holding on to pages. - "mm/damon: move damos filter range arguments validation to core" (SJ Park): Move DAMOS filter range validation into the core, where all callers can share it. Also close the sysfs read-twice races which could pass inconsistent ranges, and add tests for invalid input. - "mm/damon: misc cleanups" (Zenghui Yu): Clean up a collection of small DAMON rough edges in the core, interfaces and tests. No large behavior change here, just simpler code and fewer opportunities for confusion. - "mm/damon: cleanup code, add test cases, and update guidances in docs" (SJ Park): Clean up DAMON's core and interfaces, add missing test cases and bring the documentation into line with the code. These changes should make the newer facilities easier to understand and maintain. - "mm/damon: introduce hugepage_size probe filter" (SJ Park): Add a DAMON probe filter for hugepage size. Let users select a range of folio sizes through sysfs, validate the range and document the new interface, so monitoring can concentrate on memory backed by the desired hugepage sizes. - "selftests/mm: pagemap_ioctl test fixes and cleanups" (Zenghui Yu): Fix some pagemap ioctl test assumptions and clean up their setup and reporting. Make failures point to the operation that went wrong rather than to test harness bookkeeping. - "mm: avoid large folio splits when swap is unavailable" (Johannes Weiner): Avoid splitting large folios merely to attempt swap when no swap space is available. There is little point paying for an expensive split when reclaim cannot actually swap the resulting pages out. - "mm: workingset: fix the shadow node budget under MGLRU" (Hui Zhu): Fix the workingset shadow-node budget with MGLRU enabled. Keep the shadow entries within the intended limits so refault tracking does not quietly consume more metadata than it should. - "selftests/mm: TAP output and global-state fixes" (Song Hu): Fix selftests which produce misleading TAP output or fail to restore global settings after a run. A test ought to leave the system as it found it, particularly when a whole suite follows behind. - "mm: remove page_swap_entry()" (Tal Zussman): Remove page_swap_entry() and convert its remaining users to the folio and swap-entry helpers. Lose an old page-level interface and make the swap paths a little more consistent. - "mm: refactor zonelist constructors and iterators" (Gregory Price): Refactor the zonelist construction and walk helpers to reduce duplicated logic. Keep the allocation ordering rules intact while making it clearer how zones are selected. - "mm: kmemleak: batch the struct page scan" (Breno Leitao): Scan adjacent struct pages in batches instead of taking kmemleak_lock once for every single page. This preserves scan coverage and the lock-hold bound, while cutting overhead substantially on debug kernels. - "mm/damon: misc improvements in tests and documents" (Zenghui Yu): Fix some DAMON selftests which leave kdamond running or fail under Python safe-path mode, and improve the KUnit coverage and documentation. Nothing especially glamorous, but these are worth doing. - "mm/damon: introduce probe_hits_wsum DAMOS core filter" (SJ Park): Add a DAMOS core filter using weighted probe-hit counts. This gives schemes another way to select regions based on what the monitoring probes observed. - "mm/mglru: fix ineffective memory protection for non-kswapd reclaim" (Ridong Chen): Fix MGLRU's memory protection handling for reclaim outside kswapd. The protection settings weren't having the intended effect in these paths, leaving protected memory more exposed than requested. - "mm/damon: improve readability, clarity and test coverage" (SJ Park): Simplify a handful of DAMON core helpers, clarify their behavior and expand test coverage. There are no intended userspace-visible changes in this batch. - "mm: replace PF_KCOMPACTD/PF_KSWAPD with kthread_func()" (Kefeng Wang): Stop identifying reclaim and compaction workers through the PF_KSWAPD and PF_KCOMPACTD task flags. Use their kthread functions instead and remove the redundant flag plumbing. - "mm/hugetlb: fix surplus accounting and availability checks during demotion" (Longlong Xia): Fix HugeTLB surplus accounting and availability checks when demoting huge pages. Keep the counters and reservation decisions consistent as one huge page turns into several smaller ones. - "mm/damon: add kunit tests for probe_hits handling and probe params validation" (Jason Angelov): Add KUnit tests for DAMON probe-hit handling and invalid probe parameters. These cover edge cases that were too easy to miss in ordinary monitoring tests. - "mm/damon: fix misc bugs in kunit, quota goals and sysfs refresh_ms" (SJ Park): Fix a few DAMON bugs in KUnit tests, quota-goal handling and the sysfs refresh interval. Small fixes, but the incorrect results and control behavior were real. - "mm: zswap: optimize zswap invalidate and store" (Kefeng Wang): Simplify zswap's invalidate and store paths to avoid avoidable lookups and work. Keep the entry lifetime rules intact while making those frequent operations cheaper. - "mm: replace custom ratelimiting logic" (Pedro Falcato): Replace hand-rolled MM rate limiting with the common kernel helpers. There is no need to maintain another local version of the same mechanism. - "mm/vmalloc: minor cleanups" (Ye Liu): Tidy some vmalloc initialization and reporting code, and pull duplicated vmap-area insertion work into a helper. No functional changes are intended. - "mm/mglru: clean up isolate_folios for readability and clarity" (Ridong Chen): Clean up MGLRU's isolate_folios() so the decisions are easier to read and verify. Keep reclaim behavior unchanged while removing unnecessary twists in the control flow. - "Catch automatic storage in IDA and Maple Tree definitions" (Yury Norov): Catch IDA and maple-tree definitions placed in automatic storage where the lifetime rules are wrong or easy to get wrong. Add checks so such mistakes are harder to slip through unnoticed. - "mm, swap: some random fixes and cleanups" (Kemeng Shi): Fix and simplify a few unrelated swap paths, removing redundant work and tidying up error handling. A small collection of things which were better fixed than left lying around. - "mm/page_io: folio conversion cleanups" (Tal Zussman): Finish some page-to-folio conversions in swap I/O, removing obsolete wrappers and using folio operations consistently. The I/O behavior is intended to remain the same. - "mm/swap: skip empty clusters in the swapoff scan" (Youngjun Park): Skip empty swap clusters during swapoff instead of walking slots which have nothing to release. This avoids pointless work on sparsely used swap devices. - "mm: trace: decode architecture-specific VMA flags" (Meijing Zhao): Teach the VMA trace output to decode architecture-specific flags, rather than leaving them as unexplained bits. That should make tracing a good deal more useful when debugging unusual mappings. - "mm, swap: don't spin on a bad swap entry" (Breno Leitao): Stop repeatedly retrying a swap entry which is already known to be bad. Make the error path terminate cleanly instead of wasting CPU chasing an entry that cannot be used. - "mm/page_isolation: fix UBSAN shift-out-of-bounds in isolate_single_pageblock" (Qi Xi): Fix an out-of-range shift in isolate_single_pageblock() and tighten its associated boundary handling. UBSAN found the issue; the result should be more robust pageblock isolation. - "mm: khugepaged: fix tracepoint UAF" (Vernon Yang): Fix a use-after-free in khugepaged tracepoint handling. Make sure the information recorded by tracing stays valid for as long as the trace event needs it. - "mm/damon: fix DAMOS bugs in core, paddr and vaddr" (SJ Park): Fix several DAMOS problems across the core and physical- and virtual-address backends. The changes close error cases and bring the implementations back into agreement. - "mm/hugetlb_cgroup: move the per-node usage along with the folio" (Hongfu Li): Move HugeTLB per-node usage accounting when folios migrate between nodes or are reparented to another cgroup. Otherwise numa_stat can retain stale charges and even underflow when the folio is freed. - "docs/core-api: memory-allocation: add k[mz]alloc_obj() and clarify kmalloc" (Mike Rapoport): Document the typed k[mz]alloc_obj() allocation helpers and clear up some kmalloc guidance. Make it easier for callers to choose the right interface. - "selftests/mm: Validate selections and scope memfd_secret setup" (Tianyi Chen): Check selftest selections before running and confine memfd_secret setup to the cases that actually need it. Avoid confusing failures caused by the test harness itself. - "mm/damon: move zero damos quota target_value handling to the core layer" (SJ Park): Handle a zero DAMOS quota goal target in the core rather than making each user of the interface invent its own rule. Keep quota decisions consistent across callers. - "mm/damon: Introduce a huge page collapsing mechanism using auto tuning" (Asier Gutierrez): Add an auto-tuned DAMON mechanism for collapsing huge pages. Use monitoring feedback to decide when collapse is worthwhile rather than forcing one fixed policy onto every workload. - "kmemleak: fix stale documentation and raise the verbose default" (Breno Leitao): Bring the kmemleak documentation up to date and require a third confirming scan before verbose kernels print a suspected leak. The extra scan helps suppress transient false positives in noisy debug builds. - "mm/memory: improve insert_pages() error handling" (Avi Weiss): Tighten insert_pages() error handling so partial failures are reported and cleaned up consistently. The caller should not have to guess how far the operation got. - "mm/damon: allow NULL or unstarted damon_ctx parameter for damon_call()" (SJ Park): Make damon_call() handle NULL and not-yet-started contexts safely. Simplify the rules for callers and avoid an indefinite wait on an invalid state. - "mm/damon: introduce pgidle_set probe filter type" (SJ Park): Add a pgidle_set DAMON probe filter, giving monitoring users another way to select pages by their idle-tracking state. Wire it through the filter interfaces and tests. - "mm/damon/core: remove unused helper functions" (Cheng-Han Wu): Remove DAMON core helpers which no longer have callers. Keep the API surface smaller rather than retaining dead code just in case. - "mm: stop calling pmd_folio() on special PMDs" (Gregory Price): Avoid calling pmd_folio() on special PMD mappings that have no struct page. The old assumption could fault in procfs walkers; use the normal-folio checks and handle those mappings properly. - "mm: remove three unused helpers from mm.h" (Tal Zussman): Remove three unused helpers from mm.h. Fewer obsolete interfaces for the next person to stumble over. - "mm/damon: improvements in efficiency, error handling, documents" (Liew Rui Yan): Skip needless DAMOS quota work once the quota is full, propagate a missing sysfs error and tidy a couple of comments. The ignored error could make a monitoring-start request appear successful when it wasn't. - "docs/mm/damon/design: add explanation of nr_snapshots" (Liew Rui Yan): Explain DAMON's nr_snapshots accounting in the design documentation and straighten out the associated terminology. Make it clearer what users should expect from that counter. - "selftests/cgroup: fixes for test_zswap on single core VM" (Wilson Felipe Pereira): Fix two test_zswap failures on single-CPU VMs. Wait for the exiting child's cgroup to become unpopulated before reconfiguring it, and avoid another test assumption which breaks under constrained scheduling. - "docs/ksm: fix advisor documentation and comment" (Jinjiang Tu): Correct the KSM advisor documentation and a misleading code comment. Keep the description aligned with how the advisor actually works. - "Fix bugs in HugeTLB allocation when mem_cgroup_charge_hugetlb() fails" (Ackerley Tng): Fix HugeTLB allocation cleanup when the memcg charge fails. Make sure failure exits undo what was allocated rather than leaving stale accounting or resources behind. - "mm, memcg: fix memory.peak reset clobbering other fds' watermark" (Ridong Chen): Fix memory.peak reset handling so resetting one file descriptor doesn't overwrite the watermark kept for another. Keep those per-open measurements independent as intended. - "mm: reject zone device folios in more folio walkers" (Gregory Price): Reject zone-device folios in page walkers that cannot safely handle them. Extend the existing checks rather than assuming every folio has ordinary system-memory backing. - "percpu: remove code with no effect" (Sang-Heon Jeon): Remove a few pieces of percpu code which no longer do anything. There is no reason to keep maintaining dead paths. * tag 'mm-stable-2026-10-19-21-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (539 commits) mm: shmem: assign a lock class to the unused huge shrinklist mm: vmscan: put rotation-missed folios at the LRU tail mm: kmemleak: scan the struct page array in MAX_SCAN_SIZE batches mm: kmemleak: move the struct page scan into a helper mm/damon: document that a zero sample_interval is accepted mm/damon: fix typos in comments mm/damon/sysfs: propagate damon_call() error in turn_damon_on mm/damon/core: skip quota score setup when the quota is full mm/damon/tests/core-kunit: test the size charged for a filter-trimmed region mm/damon/core: charge only the part of a region the filter left mm: disallow raw PFN mappings of huge/shared zeropage selftests/mm: zap whole PTE tables in the khugepaged race harness selftests/mm: add memory-pressure threads to the khugepaged race harness selftests/mm: race the collapse of windows with holes selftests/mm: add khugepaged race harness selftests/mm: check that one khugepaged pass collapses one window selftests/mm: run every supported collapse order by default selftests/mm: cover a shared-source collapse write race selftests/mm: parameterize the mixed-source collapse case by source order selftests/mm: add order-parameterized khugepaged collapse cases ...
2026-10-22cgroup/cpuset: Fix task checking when enabling a partitionGuopeng Zhang
Creating a child partition can fail when tasks are present in its descendants, even though the parent has no tasks of its own. On a cgroup v2 system with CPUs 0-7 online: cd /sys/fs/cgroup echo +cpuset > cgroup.subtree_control mkdir A echo 1-2 > A/cpuset.cpus echo root > A/cpuset.cpus.partition echo +cpuset > A/cgroup.subtree_control mkdir A/B echo 1-2 > A/B/cpuset.cpus echo +cpuset > A/B/cgroup.subtree_control mkdir A/B/C echo 1-2 > A/B/C/cpuset.cpus echo $$ > A/B/C/cgroup.procs echo root > A/B/cpuset.cpus.partition cat A/B/cpuset.cpus.partition root invalid (Parent unable to distribute cpu downstream) cat A/cpuset.cpus.effective 1-2 B should be a valid root, leaving A with an empty effective CPU list. The task in C belongs to B's partition and does not need CPUs in A. partition_is_populated() is called with B as excluded_child, but continue only skips B in the pre-order walk. The walk still visits C and finds the task there, so A is considered populated. Fix this by skipping the whole excluded subtree with css_rightmost_descendant(), as is already done for valid child partitions. Also add a test case to test_cpuset_prs.sh with a task in a grandchild cpuset. Fixes: b1bcaed1e39a ("cpuset: Treat cpusets in attaching as populated") Cc: stable@vger.kernel.org # v6.19+ Signed-off-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-20cgroup: serialize migrations of newly forked processesDaehyeon Ko
With favordynmods enabled, a cgroup.procs migration takes the target threadgroup's cgroup_threadgroup_rwsem for writing. A process fork holds the forking threadgroup's semaphore for reading. These are different semaphores when fork creates a new process. copy_process() publishes the child's PID before cgroup_post_fork() links the child to its css_set. A writer can find the child in that interval, take its uncontended semaphore, and migrate it while child->cgroups still points to init_css_set and child->cg_list is empty. Migration links the node into init_css_set.mg_tasks. cgroup_post_fork() then adds the same node to the intended css_set as though it were unlinked, corrupting both task lists and their css_set ownership. This regresses the fork/migration race fixed by commit b12b533fa523 ("cgroups: add lock for child->cgroups in cgroup_post_fork()") and restored again in 2012 after the lock had been removed. The old task_lock protection was later superseded by global css_set and cgroup threadgroup locking. The change identified by the Fixes tag narrowed only the migration side back to a per-threadgroup lock, recreating the different-process exclusion failure. Take the new threadgroup's semaphore for reading before the child becomes visible and hold it through cgroup_post_fork(). Use a nested lock subclass because fork already holds the parent's semaphore. CLONE_THREAD children share the parent's signal_struct and remain covered by the existing lock. Migrations of unrelated, fully initialized threadgroups remain concurrent. An AI coding assistant was used to identify and reproduce the regression and to develop and test this fix. Fixes: 0568f89d4fb8 ("cgroup: replace global percpu_rwsem with per threadgroup resem when writing to cgroup.procs") Cc: stable@vger.kernel.org # 6.18+ Signed-off-by: Daehyeon Ko <4ncienth@gmail.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup: fix cgroup.kill for exited thread group leaderKir Kolyshkin
cgroup.kill does not work in case a process' thread group leader is gone but other threads are still running. This is because __cgroup_kill() only walks the thread group leaders and sends SIGKILL to each of them with send_sig(), which uses PIDTYPE_PID: 1. The exited leader is still listed by CSS_TASK_ITER_PROCS as long as the process has live threads, but a signal directed at a single exiting task never takes effect: complete_signal() does not look for another thread to deliver it to, and __send_signal_locked() now drops it altogether. The process keeps running. 2. The live threads of such a process can be migrated to another cgroup by writing any of their IDs to cgroup.procs, while the exited leader stays behind. The destination cgroup has no thread group leader to iterate, so writing to its cgroup.kill does nothing at all. In both cases the cgroup stays populated after cgroup.kill, and a process inside a container can use this to survive the container being killed. This also works with a private PID namespace when the init of the container is the process, as the PID namespace is not torn down while init has live threads. Case 1 can be fixed by using do_send_sig_info(..., PIDTYPE_TGID) instead of send_sig(), so that the signal is taken by a live thread even if the task found by the iterator is exiting. Note it was pointed out by Eric in [1] while reviewing v2 of cgroup.kill patchset, and v3 was meant to switch to it, but the change was only made in cgroup_post_fork(), leaving __cgroup_kill() with send_sig(). Case 2 can only be fixed by iterating over all threads, so this is what the fix also does. Once a thread group is being killed, its other threads are skipped since they already have a fatal signal pending. I ran some performance checks and the added cost is ~60 ns per every thread on my laptop (and less than a millisecond for 10K threads), which seems low. The time to "populated 0" seems unchanged. The issue was found in crun by Anthropic's OSS Scanner, and crun is getting a workaround for it [2] (which does what we needed to do before cgroup.kill was added). [1] https://lore.kernel.org/all/m1v97x6niq.fsf@fess.ebiederm.org/ [2] https://github.com/containers/crun/pull/2351 Fixes: 661ee6280931 ("cgroup: introduce cgroup.kill") Cc: stable@vger.kernel.org # v5.14+ Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Acked-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Don't allow a child root partition under an isolated parentWaiman Long
Although it is not typical for real users to create a complicated nested partition hierarchy, it is currently allowed to create nested root and isolated partitions. In addition, disabling or invalidating a partition should always be allowed especially when the cpuset is being deleted. With a nested root and isolated partition hierarchy, it is possible that all active housekeeping CPUs can be exhausted when disabling or invalidating a root partition and returning its CPUs to the isolated parent. One way to avoid this problem is to disallow the creation of a root partition under an isolated parent. It is indeed a new limitation imposed on the existing cpuset partition functionality. However an isolated partition is usually a left node where scheduling and some other kernel noises are disabled. It doesn't make much sense to create a child root partition under an isolated parent and remove some CPUs from the set of isolated CPUs again. Creation of a child isolated partition under a root or isolated parent is still allowed. If a partition with child root partitions is converted to isolated, the child root partitions will be invalidated. The cgroup-v2 documentation is also updated to reflect this new limitation. New test cases are added to test_cpuset_prs.sh to verify this new behavior. Some existing test cases are also updated accordingly. Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Properly disabling partition when partition state switching failsWaiman Long
Before commit 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping"), the partition state can be freely switched from "root" to "isolated" and vice versa. After that commit, the switch from "root" to "isolated" can fail if it exhausts all the housekeeping CPUs. Later on, the switch from "isolated" to "root" can also fail if some of the partition CPUs are boot-time isolated by "isolcpus". A local or remote partition is made invalid when the switch fails. However there are 2 problems with the existing invalidation code. 1) Both the subpartitions_cpus and isolated_cpus are not properly updated. 2) In the case of remote partition with remote_partition flag set, it is not cleared. Fix this by calling the proper partition disabling function in this case. On an x86 test system with boot option "isolcpus=10 cgroup_debug" set and more than 16 cores, the following commands were executed after boot. # cd /sys/fs/cgroup # echo +cpuset > cgroup.subtree_control # cat cpuset.cpus.isolated 10 # mkdir A # echo 10-12 > A/cpuset.cpus # echo isolated > A/cpuset.cpus.partition # cat A/cpuset.cpus.partition isolated Before the patch: # echo root > A/cpuset.cpus.partition # cat A/cpuset.cpus.partition root invalid (partition config conflicts with housekeeping setup) # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions 10-12 10-12 After the patch: # echo root > A/cpuset.cpus.partition # cat A/cpuset.cpus.partition root invalid (partition config conflicts with housekeeping setup) # cat cpuset.cpus.isolated .__DEBUG__.cpuset.cpus.subpartitions 10 Reported-by: Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/lkml/7f4c57b26ad120ab30adf35653f1dd94@kernel.org/ Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup") Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping") Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Do housekeeping check before converting invalid partition to ↵Waiman Long
valid When a hotplug event happens or when update_cpumasks_hier() is called, an invalid partition may be converted back to valid. However check against housekeeping conflict isn't being done which can lead to a partition state that violates the housekeeping rules. Update the code to allow reviving an invalid partition only if it can pass the housekeeping check. Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup") Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Consider all the exclusive CPUs when doing housekeeping checkWaiman Long
When prstate_housekeeping_conflict() is called to perform housekeeping check with cpumask changes, the cpumask changes are passed in via add_cpus and del_cpus parameters. With this arrangement, the exclusive CPUs owned by the child partitions and not affected by the cpumask change, if any, are excluded in the analysis. The only places where there are separate add and delete masks are when the cpuset.cpus or cpuset.cpus.exclusive is being changed. In both cases, partition_cpus_change() is a good place to perform the housekeeping check. If we allow a root partition to be created under an isolated partition, it is possible the cpumask change will still leave active housekeeping CPUs but then no housekeeping CPUs will be left when the child partitions are disabled or invalidated. As a first step to protect against this possibility, all the exclusive CPUs of the cpuset should be considered to be owned by the current cpuset when doing housekeeping check. At the same time, we do want to know if some housekeeping CPUs are going to be removed from the isolated partition that there will be no danger of losing all the active housekeeping CPUs. So the current add_cpus paremeter is renamed as new_cpus to indicate that it includes the complete set of effective xcpus. With the new check in partition_cpus_change(), the housekeeping check in update_parent_effective_cpumask() with partcmd_update and newmask is now duplicative and can be removed. The housekeeping check in remote_cpus_update(), however, will still be useful as this function can be called directly from update_cpumasks_hier() without going through partition_cpus_change(). The update_parent_effective_cpumask() call from update_cpumasks_hier() is partcmd_update with no cpumask which still have the housekeeping check and so is covered. Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping") Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Consolidate isolated_cpus_can_update() into ↵Waiman Long
prstate_housekeeping_conflict() The isolated_cpus_can_update() and prstate_housekeeping_conflict() are checking different aspects of upcoming cpumask changes that may conflict with the current setting of the housekeeping cpumasks. There are places where both are called together. There are also places where only one of them is called. That inconsistency can contribute to missing check where invalid cpumask changes may be allowed to move forward. Fix that by consolidating isolated_cpus_can_update() into prstate_housekeeping_conflict() and call prstate_housekeeping_conflict() in all the places where either one of them or both are called. The exception is the validate_partition() function where the prstate_housekeeping_conflict() call is removed. It is because validate_partition() is called only from partition_cpus_change() where prstate_housekeeping_conflict() will be called from either remote_cpus_update() or update_parent_effective_cpumask() with partcmd_update if not for partition invalidatation or disablement. Now prstate_housekeeping_conflict() will be called in the following locations: - when a partition is enabled in remote_partition_enable() or in update_parent_effective_cpumask() with partcmd_enable*. - when a cpumask is updated in remote_cpus_update() or in update_parent_effective_cpumask() with partcmd_update. - when a partition state changes from root to isolated or vice versa in update_prstate(). Fixes: 4a74e418881f ("cgroup/cpuset: Check partition conflict with housekeeping setup") Fixes: 103b08709e8a ("cgroup/cpuset: Fail if isolated and nohz_full don't leave any housekeeping") Reviewed-by: Guopeng Zhang <zhangguopeng@kylinos.cn> Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: Fix a missing isolated_cpus update in update_cpumasks_hier()Waiman Long
It is found that when a local isolated partition is being invalidated because its parent partition is no longer valid, the isolated_cpus cpumask may not be properly updated. Those isolated CPUs may or may not needed to be removed from isolated_cpus depending on if there is a local isolated partition ancestor that will now own those isolated CPUs instead. Fix that by calling a new ancestor_partition() helper to return the closeset valid partition root ancestor. If that ancestor is PRS_ROOT, the isolated CPUs will have to be removed from isolated_cpus. For a PRS_ISOLATED ancestor, the isolated CPUs are checked against the ancestor's ones to see if any of them should be removed as it is possible that the invalidation of the parent may be caused by change in the effective xcpus of that ancestor. If any of the isolated CPUs should be removed, isolated_cpus_update() will be called under callback_lock. An existing test case in test_cpuset_prs.sh is modifed and a new one added to test that. Fixes: 11e5f407b64a ("cgroup/cpuset: Keep track of CPUs in isolated partitions") Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-19cgroup/cpuset: use mems_allowed_seq when checking a policy nodemaskGregory Price (Meta)
A cpuset update changes the task's mems_allowed and rebinds its mempolicy in separate steps under mems_allowed_seq, but the check reads both without it. This can cause a spurious numa placement miss under some cpuset/mempolicy combinations. A reader that loads one mask before the update and the other after it can see a spurious empty intersection and drop the nodemask. For example, a task with cpuset.mems={0} and MPOL_BIND={0}. If the cpuset moves to mems={1,2}, the policy is rebound to {1}. The allocator can race and compare the old mems_allowed {0} against the new policy {1} - which will result in a dropped nodemask and a misplaced allocation on node 2. Simply retry the intersection until mems_allowed_seq is stable. This covers the task policy, which is rebound under mems_allowed_seq. VMA policies are rebound afterwards by mpol_rebind_mm(), outside mems_allowed_seq, and are not covered by this change. Fixes: cc9a6c877661 ("cpuset: mm: reduce large amounts of memory barrier related damage v3") Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Gregory Price (Meta) <gourry@gourry.net> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-12cgroup: Merge branch 'for-7.3-fixes' into for-7.4Tejun Heo
Pull to receive 1765a153d985 ("cgroup/pids: Restore pids.events notifications in local mode") for the pids.events notification selftest. Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-09BackMerge tag 'v7.3-rc6' into drm-nextDave Airlie
Linux 7.3-rc6 This was requested by Thomas to have a new base to start some branch from Signed-off-by: Dave Airlie <airlied@redhat.com>
2026-10-08mm/memcontrol: skip non-hierarchical memcg-wide stats when v1 is unavailableJoanne Koong
memcg_vmstats keeps a non-hierarchical copy of every memcg-wide stat item and event alongside the hierarchical one. The only readers however are the legacy memory.stat and memory.numa_stat, and reparenting on offline. All of them live under CONFIG_MEMCG_V1, and their accessors (memcg_page_state_local() and memcg_events_local()) are already compiled out with it. This means on a CONFIG_MEMCG_V1=n kernel, memcg_vmstats's non-hierarchical arrays are written to on every rstat flush, despite their values never being read / accessed. The same holds when the kernel does support v1 but the controller has been blocked from v1 hierarchies with the boot param cgroup_no_v1={memory,all}. A v1 mount is refused in that case, so the legacy memory.stat can never exist and the arrays are just as unread / unaccessed. Compile out the non-hierarchical memcg-wide arrays if CONFIG_MEMCG_V1 is not set. If it is set but cgroup_no_v1= has blocked the controller, skip updates on the arrays. This makes flushes cheaper. mem_cgroup_stat_aggregate() can now skip the read-modify-write of ac->local[i]. Nothing else accesses state_local or events_local, so those cachelines were getting pulled in solely for the writes and they are separate from the ones the loop is already walking. On an 80-cpu x86_64 machine with 500 cgroups each running a workload that dirties anon, file, dirty/writeback, slab, kmem, mlock, and reclaim counters, timing mem_cgroup_css_rstat_flush() in-kernel in TSC ticks per flush showed roughly before after delta memcg-wide aggregation 1231 1180 -4.1% overall flush function 2452 2397 -2.2% These numbers are from taking the median of 70 samples, one per 20s window on each kernel. The 95% intervals observed on the two deltas are [-5.41%, -1.76%] and [-4.53%, -0.14%]. The values above include the timing overhead itself, so only the delta is meaningful here. Counting the items that actually changed, a median of 1.5 of the 77 memcg-wide items (57 state + 20 events) had a non-zero per-cpu delta at each flush, which means the benchmarks above are with one or two fewer cachelines pulled in per flush. The count is low because the benchmark reads memory.stat in a loop to keep the flush rate up. For cases where flushes are triggered only by the 2s periodic worker, more changes will have accumulated between flushes, so more cachelines are skipped and the per-flush saving should be larger. Link: https://lore.kernel.org/20260903215616.1456239-1-joannelkoong@gmail.com Signed-off-by: Joanne Koong <joannelkoong@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Shakeel Butt <shakeel.butt@linux.dev> Reviewed-by: Yosry Ahmed <yosry@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Muchun Song <muchun.song@linux.dev> Cc: Roman Gushchin <roman.gushchin@linux.dev>
2026-10-06cgroup/cpuset: Call is_in_v2_mode() after acquiring cpuset_mutex in ↵Waiman Long
cpuset_handle_hotplug() It is reported by sashiko that calling is_in_v2_mode() outside of cpuset_mutex critical section in cpuset_handle_hotplug() can introduce a TOCTOU race where cgroup hierarchy may have changed from v1 to v2 or vice versa after is_in_v2_mode() is called leading to erroneously skip the allocation of tmpmasks or incorrectly modify cpus_allowed masks, resulting in cpuset state corruption. Fix that by calling is_in_v2_mode() after acquiring the cpuset_mutex. Fixes: b8d1b8ee93df ("cpuset: Allow v2 behavior in v1 cgroup") Signed-off-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-02cgroup/cpuset: Handle cpu hotplug race in guarantee_active_cpus()Waiman Long
With commit 2125c0034c5d ("cgroup/cpuset: Make cpuset hotplug processing synchronous"), the cpuset hotplug operation becomes synchronous. That commit also removes the code that handles race between cpuset_hotplug_work and cpu hotplug notifier with the assumption that race is now gone. Later commit 7a0aabd9ce69 ("cgroup/cpuset: Always use cpu_active_mask") updates the cpuset code to always use cpu_active_mask instead of cpu_online_mask in some places including guarantee_online_cpus() which is renamed to guarantee_active_cpus() in that commit. In the case of CPU offline operation, cpu_active_mask is updated first in sched_cpu_deactivate() to remove the offline CPU before cpuset_handle_hotplug() is called to update the effective_cpus of the affected cpusets. The cpu_online_mask is updated after that near the end of the offline operation to remove the offline CPU. As a result, the race comes back and the top cpuset may not have any active CPU leading to NULL pointer dereference during the race window when guarantee_active_cpus() is called after cpu_active_mask is updated to remove the CPU to be torn down but before cpuset_handle_hotplug() is able to properly update the effective_cpus of the top cpuset. Fix this by adding back the NULL cs check to avoid this problem. Fixes: 7a0aabd9ce69 ("cgroup/cpuset: Always use cpu_active_mask") Cc: stable@vger.kernel.org # v6.16+ Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu> Link: https://lore.kernel.org/lkml/CA+0ovChh3VjsKN1g+ZGjwwY2fGTpP7uD+aCCByLj5Qbymw=bfQ@mail.gmail.com Signed-off-by: Waiman Long <longman@redhat.com> Tested-by: Farhad Alemi <farhad.alemi@berkeley.edu> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-10-01cgroup: Track the effective css in each cgroupPeng Yu
cgroup_e_css() and cgroup_get_e_css() find the effective css by walking up the hierarchy until they reach a cgroup that has the subsystem enabled. Per-I/O users such as nvmet need this lookup to be O(1). Track the effective css in each cgroup and make cgroup_e_css() and cgroup_get_e_css() use it. When a css is brought online or offline, update the effective css of its cgroup and of all descendants. Signed-off-by: Peng Yu <yupeng0921@gmail.com> Assisted-by: Claude:claude-fable-5 [Claude Code] Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Tejun Heo <tj@kernel.org>
2026-09-30cgroup/cpuset: Move cpuset_update_flag() to cpuset-v1.cRidong Chen
cpuset_update_flag() is only used by the legacy (v1) interface: its callers are the write handlers of cpuset1_files via cpuset_write_u64(), and cpuset_css_offline() which invokes it only on the legacy hierarchy (!cpuset_v2() && is_sched_load_balance()). Move it into cpuset-v1.c, where its sole helper cpuset1_update_tasks_flags() already lives, so both can go back to being static and drop out of cpuset-internal.h. As a result: - dup_or_alloc_cpuset(), free_cpuset() and validate_change() are now referenced from cpuset-v1.c, so they lose their static qualifier and gain declarations in cpuset-internal.h. - The v2 branch in cpuset_update_flag() that called cpuset_force_rebuild() is dead on the v1-only call paths and is dropped; the remaining rebuild_sched_domains_locked() path is kept. callback_lock, a static in cpuset.c, is taken through the existing cpuset_callback_lock_irq() wrapper. - The open-coded "!cpuset_v2() && is_sched_load_balance()" test in cpuset_css_offline() is wrapped in a new cpuset1_offline_css() helper (a no-op stub when CONFIG_CPUSETS_V1=n), removing the last cpuset_v2() branch from that path in cpuset.c. No functional change intended. Assisted-by: LLM Signed-off-by: Ridong Chen <chenridong@xiaomi.com> Reviewed-by: Waiman Long <longman@redhat.com> Signed-off-by: Tejun Heo <tj@kernel.org>
2026-09-30cgroup/cpuset: Don't access cpuset_cgrp_subsys.root in is_in_v2_mode()Waiman Long
After seeing the patch [1] to guard is_in_v2_mode() with RCU, it makes me realize that is_in_v2_mode() may be called in a context where a new cgroup filesystem is being rebound with stale cpuset_cgrp_subsys.root pointer. Avoid this potential UaF situation by adding a new cpuset_v2_mode flag which is set when the cpuset_v2_mode mount option is used. This flag is written into only when cpuset_bind() is being called with a stable cpuset_cgrp_subsys.root value. The is_in_v2_mode() helper is modified to read the new cpuset_v2_mode flag instead of accessing cpuset_cgrp_subsys.root directly. [1] https://lore.kernel.org/lkml/20260929084124.626693-2-arighi@nvidia.com Fixes: b8d1b8ee93df ("cpuset: Allow v2 behavior in v1 cgroup") Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ridong Chen <ridong.chen@linux.dev> Reviewed-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org>