linux-toradex.git/arch/arm64/net, branch v6.7

bpf: Use bpf_is_subprog to check for subprogs

2023-09-16T16:34:20+00:00

We would like to know whether a bpf_prog corresponds to the main prog or
one of the subprogs. The current JIT implementations simply check this
using the func_idx in bpf_prog->aux->func_idx. When the index is 0, it
belongs to the main program, otherwise it corresponds to some
subprogram.

This will also be necessary to halt exception propagation while walking
the stack when an exception is thrown, so we add a simple helper
function to check this, named bpf_is_subprog, and convert existing JIT
implementations to also make use of it.

Signed-off-by: Kumar Kartikeya Dwivedi 
Link: https://lore.kernel.org/r/20230912233214.1518551-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov

bpf, arm64: Support signed div/mod instructions

2023-08-18T13:46:35+00:00

Add JIT for signed div/mod instructions.

Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230815154158.717901-7-xukuohai@huaweicloud.com

bpf, arm64: Support 32-bit offset jmp instruction

2023-08-18T13:46:18+00:00

Add support for 32-bit offset jmp instructions. Given the arm64 direct jump
range is +-128MB, which is large enough for BPF prog, jumps beyond this range
are not supported.

Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230815154158.717901-6-xukuohai@huaweicloud.com

bpf, arm64: Support unconditional bswap

2023-08-18T13:46:09+00:00

Add JIT support for unconditional bswap instructions.

Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230815154158.717901-5-xukuohai@huaweicloud.com

bpf, arm64: Support sign-extension mov instructions

2023-08-18T13:45:58+00:00

Add JIT support for BPF sign-extension mov instructions with arm64
SXTB/SXTH/SXTW instructions.

Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230815154158.717901-4-xukuohai@huaweicloud.com

bpf, arm64: Support sign-extension load instructions

2023-08-18T13:45:49+00:00

Add JIT support for sign-extension load instructions.

Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230815154158.717901-3-xukuohai@huaweicloud.com

bpf, arm64: Fix BTI type used for freplace attached functions

2023-07-18T22:28:19+00:00

When running an freplace attached bpf program on an arm64 system w were
seeing the following issue:
  Unhandled 64-bit el1h sync exception on CPU47, ESR 0x0000000036000003 -- BTI

After a bit of work to track it down I determined that what appeared to be
happening is that the 'bti c' at the start of the program was somehow being
reached after a 'br' instruction. Further digging pointed me toward the
fact that the function was attached via freplace. This in turn led me to
build_plt which I believe is invoking the long jump which is triggering
this error.

To resolve it we can replace the 'bti c' with 'bti jc' and add a comment
explaining why this has to be modified as such.

Fixes: b2ad54e1533e ("bpf, arm64: Implement bpf_arch_text_poke() for arm64")
Signed-off-by: Alexander Duyck 
Acked-by: Xu Kuohai 
Link: https://lore.kernel.org/r/168926677665.316237.9953845318337455525.stgit@ahduyck-xeon-server.home.arpa
Signed-off-by: Alexei Starovoitov

bpf, arm64: Support struct arguments in the BPF trampoline

2023-05-15T19:17:22+00:00

This extends the BPF trampoline JIT to support attachment to functions
that take small structures (up to 128bit) as argument. This is trivially
achieved by saving/restoring a number of "argument registers" rather
than a number of arguments.

The AAPCS64 section 6.8.2 describes the parameter passing ABI.
"Composite types" (like C structs) below 16 bytes (as enforced by the
BPF verifier) are provided as part of the 8 argument registers as
explained in the section C.12.

Signed-off-by: Florent Revest 
Signed-off-by: Daniel Borkmann 
Acked-by: Yonghong Song 
Acked-by: Xu Kuohai 
Link: https://lore.kernel.org/bpf/20230511140507.514888-1-revest@chromium.org

bpf, arm64: Fixed a BTI error on returning to patched function

2023-04-03T15:44:03+00:00

When BPF_TRAMP_F_CALL_ORIG is set, BPF trampoline uses BLR to jump
back to the instruction next to call site to call the patched function.
For BTI-enabled kernel, the instruction next to call site is usually
PACIASP, in this case, it's safe to jump back with BLR. But when
the call site is not followed by a PACIASP or bti, a BTI exception
is triggered.

Here is a fault log:

 Unhandled 64-bit el1h sync exception on CPU0, ESR 0x0000000034000002 -- BTI
 CPU: 0 PID: 263 Comm: test_progs Tainted: GF
 Hardware name: linux,dummy-virt (DT)
 pstate: 40400805 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=-c)
 pc : bpf_fentry_test1+0xc/0x30
 lr : bpf_trampoline_6442573892_0+0x48/0x1000
 sp : ffff80000c0c3a50
 x29: ffff80000c0c3a90 x28: ffff0000c2e6c080 x27: 0000000000000000
 x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000050
 x23: 0000000000000000 x22: 0000ffffcfd2a7f0 x21: 000000000000000a
 x20: 0000ffffcfd2a7f0 x19: 0000000000000000 x18: 0000000000000000
 x17: 0000000000000000 x16: 0000000000000000 x15: 0000ffffcfd2a7f0
 x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
 x11: 0000000000000000 x10: ffff80000914f5e4 x9 : ffff8000082a1528
 x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0101010101010101
 x5 : 0000000000000000 x4 : 00000000fffffff2 x3 : 0000000000000001
 x2 : ffff8001f4b82000 x1 : 0000000000000000 x0 : 0000000000000001
 Kernel panic - not syncing: Unhandled exception
 CPU: 0 PID: 263 Comm: test_progs Tainted: GF
 Hardware name: linux,dummy-virt (DT)
 Call trace:
  dump_backtrace+0xec/0x144
  show_stack+0x24/0x7c
  dump_stack_lvl+0x8c/0xb8
  dump_stack+0x18/0x34
  panic+0x1cc/0x3ec
  __el0_error_handler_common+0x0/0x130
  el1h_64_sync_handler+0x60/0xd0
  el1h_64_sync+0x78/0x7c
  bpf_fentry_test1+0xc/0x30
  bpf_fentry_test1+0xc/0x30
  bpf_prog_test_run_tracing+0xdc/0x2a0
  __sys_bpf+0x438/0x22a0
  __arm64_sys_bpf+0x30/0x54
  invoke_syscall+0x78/0x110
  el0_svc_common.constprop.0+0x6c/0x1d0
  do_el0_svc+0x38/0xe0
  el0_svc+0x30/0xd0
  el0t_64_sync_handler+0x1ac/0x1b0
  el0t_64_sync+0x1a0/0x1a4
 Kernel Offset: disabled
 CPU features: 0x0000,00034c24,f994fdab
 Memory Limit: none

And the instruction next to call site of bpf_fentry_test1 is ADD,
not PACIASP:

:
	bti     c
	nop
	nop
	add     w0, w0, #0x1
	paciasp

For BPF prog, JIT always puts a PACIASP after call site for BTI-enabled
kernel, so there is no problem. To fix it, replace BLR with RET to bypass
the branch target check.

Fixes: efc9909fdce0 ("bpf, arm64: Add bpf trampoline for arm64")
Reported-by: Florent Revest 
Signed-off-by: Xu Kuohai 
Signed-off-by: Daniel Borkmann 
Tested-by: Florent Revest 
Acked-by: Florent Revest 
Link: https://lore.kernel.org/bpf/20230401234144.3719742-1-xukuohai@huaweicloud.com

bpf: Remove prog->active check for bpf_lsm and bpf_iter

2022-10-26T06:11:46+00:00

The commit 64696c40d03c ("bpf: Add __bpf_prog_{enter,exit}_struct_ops for struct_ops trampoline")
removed prog->active check for struct_ops prog.  The bpf_lsm
and bpf_iter is also using trampoline.  Like struct_ops, the bpf_lsm
and bpf_iter have fixed hooks for the prog to attach.  The
kernel does not call the same hook in a recursive way.
This patch also removes the prog->active check for
bpf_lsm and bpf_iter.

A later patch has a test to reproduce the recursion issue
for a sleepable bpf_lsm program.

This patch appends the '_recur' naming to the existing
enter and exit functions that track the prog->active counter.
New __bpf_prog_{enter,exit}[_sleepable] function are
added to skip the prog->active tracking. The '_struct_ops'
version is also removed.

It also moves the decision on picking the enter and exit function to
the new bpf_trampoline_{enter,exit}().  It returns the '_recur' ones
for all tracing progs to use.  For bpf_lsm, bpf_iter,
struct_ops (no prog->active tracking after 64696c40d03c), and
bpf_lsm_cgroup (no prog->active tracking after 69fd337a975c7),
it will return the functions that don't track the prog->active.

Signed-off-by: Martin KaFai Lau 
Link: https://lore.kernel.org/r/20221025184524.3526117-2-martin.lau@linux.dev
Signed-off-by: Alexei Starovoitov