diff options
| author | Emil Tsalapatis <emil@etsalapatis.com> | 2026-10-02 10:52:17 +0000 |
|---|---|---|
| committer | Alexei Starovoitov <ast@kernel.org> | 2026-10-02 13:00:59 +0000 |
| commit | 23290213e1abfc71731a6491ca278141e8cbbf3b (patch) | |
| tree | 407a1a4c11747b9249b597059ad63fc098adb9d7 /include | |
| parent | baf4e66f038fb1181dd0b1085c6b0913399c289f (diff) | |
bpf: Support per-call-site kfunc specialization
specialize_kfunc() currently updates the canonical kfunc descriptor in
place. Different call sites therefore cannot select different
specializations of the same kfunc, and the selected target depends on
verification order.
Keep canonical descriptors unchanged for verifier lookups. Specialize a
copy for each call site, then reuse or append an immutable target
descriptor and store its index in the finalized call instruction.
Allow space for one canonical and one specialized target per kfunc. This
is conservative because most kfuncs do not specialize. Adding specialized
kfunc versions does not require resorting the array because lookups are
only used for deduplication and func_model lookup. Deduplication for
specialized kfuncs is handled by the specialization process itself,
while all specializations of a function share the same proto and
func_model.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261002105218.6171-7-emil@etsalapatis.com
Diffstat (limited to 'include')
| -rw-r--r-- | include/linux/bpf_verifier.h | 12 |
1 files changed, 9 insertions, 3 deletions
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index b309ff7db493..79c4be5c33f4 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -1788,7 +1788,9 @@ enum bpf_reg_arg_type { }; #define MAX_KFUNC_DESCS 256 -static_assert(MAX_KFUNC_DESCS <= S16_MAX + 1); +/* Each kfunc can have its canonical and one specialized call target. */ +#define MAX_KFUNC_CALL_DESCS (MAX_KFUNC_DESCS * 2) +static_assert(MAX_KFUNC_CALL_DESCS <= S16_MAX + 1); struct bpf_kfunc_desc { struct btf_func_model func_model; @@ -1800,11 +1802,15 @@ struct bpf_kfunc_desc { struct bpf_kfunc_desc_tab { u32 nr_descs; + u32 nr_base_descs; /* Sorted by func_id (BTF ID) and offset (fd_array offset) during * verification. JITs use the descriptor index stored in the finalized - * call's off field. + * call's off field. The first nr_base_descs entries are the canonical + * descriptors used for verifier lookups. Call specialization may append + * immutable descriptors for additional targets. * - * Grown one entry at a time by bpf_add_kfunc_call(). + * Grown one entry at a time by bpf_add_kfunc_call() and during + * call specialization. */ struct bpf_kfunc_desc descs[]; }; |