Pull bpf fixes from Alexei Starovoitov:
 "Fix various bugs in bpf core and bpf-lwt. Looks like the speed of us
  fixing bugs is higher than the number of new bugs found. Not out of
  the woods yet. sockmap fixes will come later.

   - Various fixes (Alexei Starovoitov):
       - Use one may_goto time slice per prog invocation instead of one
         per stack frame to fix a soft lockup in loops that nest across
         calls
       - Fix use-after-free in btf__add_type() in libbpf
       - Invalidate a dynptr slice when its stack buffer is written or
         when the frame of the buffer returns
       - Fix the tracking of pointers to the stack in SCEV
       - Keep the valid bytes of a packet pointer in SCEV
       - Fix bpf_throw() in a prog with a private stack in x86 JIT
       - Compare the arguments of an extension prog with those of its
         target
       - Don't accept memory of the prog for struct arguments of
         KF_ACQUIRE kfuncs and for arguments of an empty struct type
       - Fix mmap() of an arena: refuse address 0 and a length above
         max_entries pages, etc.
       - Don't let bpf_skb_adjust_room() raise gso_size to GSO_BY_FRAGS
       - Fix lwt progs: reset skb->cb and don't leave a stale MAC header
         after a prog pushes an IP header, require BPF_LWT_REROUTE when
         an lwt_in prog changes the protocol, don't call into IPv6 when
         it is disabled, don't look for the SRH again in the End.BPF
         helpers, and set a dst for test runs of lwt_in progs

   - Fix bpf_dynptr_copy() for overlapping ranges in one packet
     (Ayushman Rout)

   - Various fixes (Chris Mason):
       - Reject memory of the prog for kfunc projection type arguments
       - Require prog fd or capability for cgroup prog detach
       - Keep the skb network offset signed in cgroup skb progs
       - Reject non-IPv6 packets in BPF_LWT_ENCAP_SEG6
       - Limit the SRH length and move the MAC header in
         bpf_lwt_seg6_adjust_srh()
       - Move the transport header together with the headers that
         bpf_skb_adjust_room() and bpf_lwt_seg6_adjust_srh() move, and
         use the inner transport header after a decap
       - Check the offset in bpf_skb_generic_{push,pop}()

   - Mark jmp mode of ftrace direct trampolines on x86 by clearing bit
     63 of the address instead of setting bit 0, since an ftrace
     callback can start at an odd address (Daniel Borkmann)

   - Fix psock update-side data races on sk->sk_prot and progs in
     sockmap (Jakub Sitnicki)

   - Fix skb_pull() BUG in __bpf_redirect_neigh(): make sure that the
     MAC and IP headers are there and are linear (Jamal Hadi Salim)

   - Fix jited line info that is off by the CFI preamble on x86-64,
     arm64 and riscv64 (Jose Fernandez)

   - Clamp the trace length in __bpf_get_stack() to fix a write after
     the end of the buffer (Josef Bacik)

   - Drop percpu_counter from the htab element count, since its lock can
     deadlock with NMI and tracing progs that update the same map
     (Puranjay Mohan)

   - Propagate scalar precision through stack arguments (Xu Yunxiang)

   - Fix callback recursion selftest with llvm24 (Yonghong Song)

   - Free the skb when bpf_xmit() fails to expand the headroom (Yuqiang
     Sun)"

* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (61 commits)
  libbpf: Fix use-after-free in btf__add_type
  selftests/bpf: Exercise stack argument precision on state reuse
  bpf: Propagate scalar precision through stack arguments
  selftests/bpf: Fix callback recursion test with noipa global func
  selftests/bpf: Test one may_goto time slice per prog invocation
  bpf: End bpf_loop() when the time of may_goto is up
  bpf: One may_goto time slice per prog invocation
  bpf: Pass a pointer to arch_bpf_timed_may_goto()
  s390/bpf: Support internal-only MOV instruction to resolve per-CPU addrs
  selftests/bpf: End the loop of parallel_test_bitmap after 50 ms
  selftests/bpf: Give the first loop of cond_break5 an end
  selftests/bpf: Don't cast a pointer to .bss in arena_htab_asm
  bpf: Emit timed may_goto for JITed progs only
  selftests/bpf: Test writes to the buffer of bpf_dynptr_slice_rdwr()
  bpf: Invalidate a dynptr slice when its stack buffer is written
  selftests/bpf: Test a callee that overwrites a spilled pointer of its caller
  bpf: Don't trust pointers spilled in slots that a callee can write
  selftests/bpf: Test the join of fp+0 with a value that is not a pointer to stack
  bpf: Fix the join of fp+0 with a value that is not a pointer to stack
  selftests/bpf: Run arena_mmap serially and test MAP_FIXED addresses
  ...

Add two asm regressions for the precision of the sixth argument, which
is passed in a stack argument slot and used as a map value offset.

The first test reuses a safe state after storing arg6 from a register.
Precision must propagate from that slot to the register at an earlier
checkpoint, so a later out-of-bounds path cannot reuse that checkpoint.

The second test triggers conservative precision fallback by backtracking
across a store through a register that later becomes a scalar through
pointer subtraction. The caller's argument slot must become precise so
an out-of-bounds argument cannot reuse the safe state.

Both programs are load-only rejection tests and use the existing stack
argument compiler and architecture guards.

The two test programs were supplied by Alexei Starovoitov during review
and are included unchanged.

AI assistance was used in preparing this patch.

Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/DMJI3GWXQOC9.UA3K9X139AR@gmail.com/
Link: https://patch.msgid.link/20261102190350.788433-3-xyx2021@mail.ustc.edu.cn

With llvm24 after commit 28b4a2032f97 ("[BPF] Mark global functions
noipa"), the test recursion_via_global_func_and_callback fails:

  13: (85) call bpf_loop#181 4: frame1: R1=scalar() R10=fp0 cb
  ; return static_calls_global(i); @ verifier_global_subprogs.c:548
  4: (85) call pc+1
  ...
  ; return global_calls_loop(x); @ verifier_global_subprogs.c:543
  6: (85) call pc+1
  Func#3 ('global_calls_loop') is global and assumed valid.
  7: frame2: R0=scalar()
  7: (95) exit
  ...
  ; return static_calls_global(i); @ verifier_global_subprogs.c:548
  5: (95) exit
  At callback return the register R0 has unknown scalar value should have been in [0, 1]
  =============
  EXPECTED   SUBSTR: 'recursive call from global_calls_loop() to loop_cb_calls_static()'
   #669/9   verifier_global_subprogs/recursion_via_global_func_and_callback:FAIL

The callback loop_cb_calls_static() returns the result of
static_calls_global(), which in turn returns the result of the global
function global_calls_loop().

Before the above llvm24 commit, IPSCCP (interprocedural sparse
conditional constant propagation) could fold the return value of
global_calls_loop() into its callers. global_calls_loop() is not
static, but it has a non-weak definition in the same compilation unit,
so IPSCCP may assume that definition is the one called. It found that
global_calls_loop() always returns 0 and replaced the uses of the call
result with 0, which made static_calls_global() and then
loop_cb_calls_static() return a constant 0 as well. The calls stay, only
their return values are replaced. So the callback set R0 to 0 itself,
passed the bpf_loop callback return range check, and verification went
on to report the expected recursion.

The llvm24 commit marks global functions noipa, since the verifier
checks a global function only against its BTF prototype and freplace
can replace it. With noipa, IPSCCP no longer propagates the return
value of global_calls_loop() to callers. The verifier does not look
into a global function either, so R0 after the call is an unknown
scalar, and the callback return check fails before the recursion is
detected.

Make static_calls_global() return void and let loop_cb_calls_static()
return 0 explicitly. The call chain the test is about stays the same,
and the test passes with both llvm24 and earlier versions (e.g. llvm21
to llvm23).

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://github.com/llvm/llvm-project/pull/237690
Link: https://patch.msgid.link/20261102192727.762933-1-yonghong.song@linux.dev

Most tests call a function that spins until may_goto stops it, four times.
With one time slice for an invocation of the prog only the first call
spins. With a time slice for each frame all four calls spin, and the test
fails after 1 sec.

  group           tests
  --------------  ------------------------------------------------------
  the problem     9 insns, three levels of short loops, a subprog with
                  65534 iterations, bpf_loop() with a callback that spins
                  or makes 60000 iterations
  ways to a loop  call, call through a function without a loop, global
                  function, callx, bpf_loop() inlined and not,
                  bpf_for_each_map_elem(), bpf_user_ringbuf_drain(),
                  bpf_find_vma(), bpf_rbtree_add()
  bpf_loop()      ends when the time is up: inlined and not, in a callback
                  of bpf_loop(). Goes on when BPF code calls the callback
                  too
  count           a loop that is on the stack ends when the time is up,
                  65535 calls that make one iteration read the clock
  new time slice  timer, wq, task_work, call_rcu, call_rcu_tasks_trace,
                  each after a prog on the cpu that used up its time. Tail
                  call, extension prog
  same one        a timer callback that interrupts the prog doesn't give it
                  more time, the exception callback
  registers       may_goto takes a register that is dead: R0-RN are live,
                  for each N. A register that is live through the back
                  edge only, at the target only, behind the may_goto only.
                  All of that when it calls C and when the time is up.
                  16-byte return value
  other           tracing prog with frames in a private stack, may_goto in
                  dead code

Two tests lock up a cpu on a kernel with a time slice for each frame: the
9 insns and the three levels of short loops. All other tests give up and
fail there, or pass. The tests are for x86-64 and arm64 with the JIT only.

may_goto_far gets four subtests with the may_goto in a subprog.

The test is serial. Its progs spin for 250 ms in an RCU read-side critical
section, one after the other. Next to them in 'test_progs -j' the test
"mmap" gives up its wait for the end of a map. The test takes 10 sec.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-10-alexei.starovoitov@gmail.com

The count and the timestamp of may_goto are in the stack frame. So the
limit of 250 ms is for one call of a subprog, not for one invocation of the
prog, and loops that nest across calls multiply it. These 9 insns cause a
soft lockup:

  main:                 sub:
  1: call sub           1: may_goto 2f
     may_goto 2f           goto 1b
     goto 1b            2: r0 = 0
  2: r0 = 1                exit
     exit

Use one count and one timestamp for all may_goto of a prog, in per-cpu
memory of the prog. The main prog sets them in its first insns. When the
time is up each loop ends at its next may_goto, in any function.

  rX = address of the count
  rX = &(void __percpu *)(rX)
  r12 = *(u64 *)(rX +0)
  if r12 == 0x0 goto target
  r12 -= 1
  if r12 != 0x0 goto pc+2
  r12 = rX
  call arch_bpf_timed_may_goto
  *(u64 *)(rX +0) = r12

rX is a register that is dead at the may_goto. When all ten are live R0 is
saved in the stack.

An async callback can interrupt the prog on the same cpu. So its caller
sets count and timestamp, and puts the old values back at the end. A second
invocation of the prog on the same cpu gives more time to the first one.

  x86-64 (Zen 5), nsec                             before  after
  iteration, empty loop                             0.36    0.44
  iteration, the body takes 4 cycles                0.89    0.89
  call of a subprog and 1 iteration                 1.21    1.16

The 56 progs of scx with may_goto get 1.7% more JITed code and use 25% less
stack.

Fixes: e723608bf428 ("bpf: Add verifier support for timed may_goto")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-8-alexei.starovoitov@gmail.com

arch_bpf_timed_may_goto() gets the offset of count and timestamp in the
stack frame in BPF_REG_AX. Arch code adds the frame pointer to it, and
calls bpf_check_timed_may_goto() with the result. On arm64, riscv,
loongarch, powerpc and s390 the asm function does it. On x86-64 the JIT
does it, because the frame pointer is in another register when the prog has
a private stack.

The next patch moves count and timestamp out of the stack frame. So pass
the pointer in BPF_REG_AX, and let the verifier emit the addition:

  before                           after

  r12 = -16                        r12 = r10
  call arch_bpf_timed_may_goto     r12 += -16
                                   call arch_bpf_timed_may_goto

The JITs know which register r10 is. The asm functions have a move in place
of the addition, and the special case in the x86-64 JIT goes away.

These insns run when the count is used up: in each 65535th iteration. The
cost of an iteration does not change.

Three tests have the insns of may_goto in them. Update them.

Tested on x86-64, and on arm64 and s390 in QEMU. riscv, loongarch and
powerpc are compile tested only.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-7-alexei.starovoitov@gmail.com

The two progs of parallel_test_bitmap change bits of the same words in a
loop of 10000000 iterations with can_loop. The loop goes on until may_goto
stops it. The functions that it calls retry a cmpxchg that fails:

  do {
          ...
          actual = cmpxchg(&bmp->bits[idx], old, new);
          if (actual == old)
                  return true;
  } while (can_loop);
  return false;

The retry is always possible only while each stack frame has a time slice
of its own. With one time slice for all may_goto of a prog, a function that
is called after the time is up makes no iteration. The bit does not change,
and the test fails with -EINVAL, in 53 of 900 runs.

Leave the loop after 50 ms.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-5-alexei.starovoitov@gmail.com

The main prog of cond_break5 spins until may_goto stops the loop. Then it
calls a subprog and expects that the loop of the subprog makes iterations.
This is true only while each stack frame has a time slice of its own.

Stop the first loop after 100 iterations. The test still makes sure that
may_goto works in the main prog and in a subprog of one prog.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-4-alexei.starovoitov@gmail.com

arena_htab_asm is arena_htab.c with BPF_ARENA_FORCE_ASM. In this variant
__arena has no address space, so arr1 is in .bss, and cast_kern() is a real
insn. cast_kern(arr) turns the pointer to .bss into an offset in the arena.
Each of the 100000 stores to arr[i] faults, and the fault handler of the
arena skips it.

Remove the cast. In the other variant it is a nop. The prog takes 67 ms in
place of 950 ms on a kernel with lockdep.

With one time slice for all may_goto of a prog, may_goto stops the loops of
the old prog after 250 ms. The test does not see that, because it looks at
the first keys only.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-3-alexei.starovoitov@gmail.com

The timed form of may_goto calls arch_bpf_timed_may_goto(). This function
takes its argument and returns its result in BPF_REG_AX, in a register of
the cpu. Only JITed code can call it.

The verifier emits the timed form when the arch has a JIT that supports it.
It does not look at the prog. With CONFIG_BPF_JIT_ALWAYS_ON=n the
interpreter can run such a prog: the JIT is off, or it fails. The
interpreter calls the function like a helper. The function takes what the C
code left in that register for a pointer, and bpf_check_timed_may_goto()
reads and writes through it.

Emit the timed form only when the prog asks for a JIT, and make the JIT
mandatory for such a prog. All other progs get the form that counts down
from BPF_MAX_LOOPS.

One test of verifier_may_goto_1 shows the insns of the timed form. Load it
only when progs are JITed.

Fixes: e723608bf428 ("bpf: Add verifier support for timed may_goto")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261031215543.682284-2-alexei.starovoitov@gmail.com