LoongArch: BPF: Optimize 32-bit operations using native instructions

Currently, the LoongArch BPF JIT emits numerous redundant zero and sign
extensions across 32-bit operations, bloating the final JIT image size
and also leading to runtime overhead, particularly in the division and
modulo operations. Thus, it is necessary to optimize 32-bit operations
by emitting LoongArch native 32-bit word instructions.

For general ALU32 operations, utilize native 32-bit word instructions
for core calculations. Given that the BPF spec requires the upper 32
bits of the destination register to be zero-extended upon completion,
use a unified 'bstrpickd' instruction to clear the upper 32 bits at the
common exit of build_insn(), while immediate loads are optimized via
(u32) casting to bypass the trailing extension.

According to the LoongArch Reference Manual, native 32-bit division
and modulo instructions (DIV.W[U] and MOD.W[U]) require the upper 33
bits [63:31] of both input operands to be all 0s or all 1s (0x0 or
0x1ffffffff), that is to say, they need to be properly sign-extended
by the software on CPU cores without the DIV32 feature. Otherwise, the
32-bit division and modulo instructions will trigger undefined hardware
behaviors and return unexpected values.

To ensure both performance and hardware compatibility, leverage the
newly introduced 'cpu_has_div32' helper:

(1) on newer CPUs with DIV32, emit native 32-bit div/mod instructions
directly for optimal performance;

(2) on older CPUs without DIV32, sign-extend the destination register
in place and the source register into a temporary register via 'addw'
before executing the native 32-bit div/mod instructions. Note that the
JIT compiler must never alter the state of the src register because it
remains read-only for the duration of the instruction.

For JMP32 conditional branches, the source operands may originate from
64-bit contexts where the upper 32 bits contain arbitrary data, so use
6 independent case structures to prevent 64-bit comparison inversions.
Furthermore, use 'bstrpickd' to mask out upper bits for unsigned JMP32,
and 'addw' with GPR_ZERO to re-align sign bits for signed JMP32.

With this patch, the emit_{zext, sext}_32() and is_signed_bpf_cond()
helpers are completely wiped out from the header and source files,
delivering a native execution stream with a reduced JIT image size,
especially for division and mod operations.

All of the testcases of test_bpf passed on Loongson-3A6000 (with DIV32)
and Loongson-3A5000 (without DIV32), here are the test results:

$ sudo modprobe test_bpf
$ dmesg -t | grep Summary
test_bpf: Summary: 1061 PASSED, 0 FAILED, [1049/1049 JIT'ed]
test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED

For the BPF selftests test_verifier and test_progs, there are no any
regressions.

Acked-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>

0 files changed, 0 insertions, 0 deletions