sched_ext: Specialize the DSQ hashtable compare

rhashtable is a generic container: it does not know what a key is, so it
carries the key's shape as data. dsq_hash_params declares key_len,
key_offset and head_offset and nothing else, so lookups fall back to
rhashtable_compare(), which reads both the offset and the length back out
of ht->p at runtime:

	memcmp(ptr + ht->p.key_offset, arg->key, ht->p.key_len)

The key is one naturally aligned u64, but neither the offset nor the
length is a compile-time constant there, so the compiler cannot narrow
the call and emits an out-of-line memcmp() for every element walked -
two loads, a call and a length dispatch to compare eight bytes. The
interpretation costs more than the comparison it is interpreting.

find_user_dsq() sits on the __schedule() path, and scx_layered calls
scx_bpf_dsq_nr_queued() once per layer per dispatch decision. On Meta's
fleet the lookup has a significant cost and shows up in fleet wide
profile.

Let's supply an obj_cmpfn. dsq_hash_params is a const object passed by
value into the __always_inline __rhashtable_lookup(), so
params.obj_cmpfn is a compile-time constant, the ternary that selects it
folds away and the callback inlines: the compare becomes a single cmp
against dsq->id. Every consumer of the result only tests it against
zero, so the ordering memcmp() also carries is never observed. No
functional change intended.

An in-kernel A/B over the DSQ ids scx_layered creates, both parameter
sets compiled into the same kernel, makes the lookup 2.9x faster.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Tejun Heo <tj@kernel.org>

0 files changed, 0 insertions, 0 deletions