linux-toradex.git/arch/s390/net, branch v5.16-rc3

s390: introduce nospec_uses_trampoline()

2021-10-26T13:21:29+00:00

and replace all of the "__is_defined(CC_USING_EXPOLINE) && !nospec_disable"
occurrences.

Signed-off-by: Sven Schnelle 
Reviewed-by: Heiko Carstens 
Signed-off-by: Vasily Gorbik

bpf, s390: Fix potential memory leak about jit_data

2021-10-04T07:49:10+00:00

Make sure to free jit_data through kfree() in the error path.

Fixes: 1c8f9b91c456 ("bpf: s390: add JIT support for multi-function programs")
Signed-off-by: Tiezhu Yang 
Acked-by: Ilya Leoshkevich 
Reviewed-by: Christian Borntraeger 
Signed-off-by: Christian Borntraeger 
Signed-off-by: Vasily Gorbik

s390/bpf: Fix optimizing out zero-extensions

2021-09-16T08:39:55+00:00

Currently the JIT completely removes things like `reg32 += 0`,
however, the BPF_ALU semantics requires the target register to be
zero-extended in such cases.

Fix by optimizing out only the arithmetic operation, but not the
subsequent zero-extension.

Reported-by: Johan Almbladh 
Fixes: 054623105728 ("s390/bpf: Add s390x eBPF JIT compiler backend")
Reviewed-by: Heiko Carstens 
Signed-off-by: Ilya Leoshkevich 
Signed-off-by: Vasily Gorbik

s390/bpf: Fix 64-bit subtraction of the -0x80000000 constant

2021-09-16T08:39:55+00:00

The JIT uses agfi for subtracting constants, but -(-0x80000000) cannot
be represented as a 32-bit signed binary integer. Fix by using algfi in
this particular case.

Reported-by: Johan Almbladh 
Fixes: 054623105728 ("s390/bpf: Add s390x eBPF JIT compiler backend")
Reviewed-by: Heiko Carstens 
Signed-off-by: Ilya Leoshkevich 
Signed-off-by: Vasily Gorbik

s390/bpf: Fix branch shortening during codegen pass

2021-09-16T08:39:55+00:00

EMIT6_PCREL() macro assumes that the previous pass generated 6 bytes
of code, which is not the case if branch shortening took place. Fix by
using jit->prg, like all the other EMIT6_PCREL_*() macros.

Reported-by: Johan Almbladh 
Fixes: 4e9b4a6883dd ("s390/bpf: Use relative long branches")
Signed-off-by: Ilya Leoshkevich 
Signed-off-by: Vasily Gorbik

bpf: Introduce BPF nospec instruction for mitigating Spectre v4

2021-07-28T22:20:56+00:00

In case of JITs, each of the JIT backends compiles the BPF nospec instruction
/either/ to a machine instruction which emits a speculation barrier /or/ to
/no/ machine instruction in case the underlying architecture is not affected
by Speculative Store Bypass or has different mitigations in place already.

This covers both x86 and (implicitly) arm64: In case of x86, we use 'lfence'
instruction for mitigation. In case of arm64, we rely on the firmware mitigation
as controlled via the ssbd kernel parameter. Whenever the mitigation is enabled,
it works for all of the kernel code with no need to provide any additional
instructions here (hence only comment in arm64 JIT). Other archs can follow
as needed. The BPF nospec instruction is specifically targeting Spectre v4
since i) we don't use a serialization barrier for the Spectre v1 case, and
ii) mitigation instructions for v1 and v4 might be different on some archs.

The BPF nospec is required for a future commit, where the BPF verifier does
annotate intermediate BPF programs with speculation barriers.

Co-developed-by: Piotr Krysiuk 
Co-developed-by: Benedict Schlueter 
Signed-off-by: Daniel Borkmann 
Signed-off-by: Piotr Krysiuk 
Signed-off-by: Benedict Schlueter 
Acked-by: Alexei Starovoitov

s390/bpf: Perform r1 range checking before accessing jit->seen_reg[r1]

2021-07-15T17:47:25+00:00

Currently array jit->seen_reg[r1] is being accessed before the range
checking of index r1. The range changing on r1 should be performed
first since it will avoid any potential out-of-range accesses on the
array seen_reg[] and also it is more optimal to perform checks on r1
before fetching data from the array. Fix this by swapping the order
of the checks before the array access.

Fixes: 054623105728 ("s390/bpf: Add s390x eBPF JIT compiler backend")
Signed-off-by: Colin Ian King 
Signed-off-by: Daniel Borkmann 
Tested-by: Ilya Leoshkevich 
Acked-by: Ilya Leoshkevich 
Link: https://lore.kernel.org/bpf/20210715125712.24690-1-colin.king@canonical.com

s390/bpf: Implement new atomic ops

2021-03-16T19:18:49+00:00

Implement BPF_AND, BPF_OR and BPF_XOR as the existing BPF_ADD. Since
the corresponding machine instructions return the old value, BPF_FETCH
happens by itself, the only additional thing that is required is
zero-extension.

There is no single instruction that implements BPF_XCHG on s390, so use
a COMPARE AND SWAP loop.

BPF_CMPXCHG, on the other hand, can be implemented by a single COMPARE
AND SWAP. Zero-extension is automatically inserted by the verifier.

Signed-off-by: Ilya Leoshkevich 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20210304233002.149096-1-iii@linux.ibm.com

bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

2021-01-15T02:34:29+00:00

A subsequent patch will add additional atomic operations. These new
operations will use the same opcode field as the existing XADD, with
the immediate discriminating different operations.

In preparation, rename the instruction mode BPF_ATOMIC and start
calling the zero immediate BPF_ADD.

This is possible (doesn't break existing valid BPF progs) because the
immediate field is currently reserved MBZ and BPF_ADD is zero.

All uses are removed from the tree but the BPF_XADD definition is
kept around to avoid breaking builds for people including kernel
headers.

Signed-off-by: Brendan Jackman 
Signed-off-by: Alexei Starovoitov 
Acked-by: Björn Töpel 
Link: https://lore.kernel.org/bpf/20210114181751.768687-5-jackmanb@google.com

s390/bpf: Fix multiple tail calls

2020-09-15T01:21:31+00:00

In order to branch around tail calls (due to out-of-bounds index,
exceeding tail call count or missing tail call target), JIT uses
label[0] field, which contains the address of the instruction following
the tail call. When there are multiple tail calls, label[0] value comes
from handling of a previous tail call, which is incorrect.

Fix by getting rid of label array and resolving the label address
locally: for all 3 branches that jump to it, emit 0 offsets at the
beginning, and then backpatch them with the correct value.

Also, do not use the long jump infrastructure: the tail call sequence
is known to be short, so make all 3 jumps short.

Fixes: 6651ee070b31 ("s390/bpf: implement bpf_tail_call() helper")
Signed-off-by: Ilya Leoshkevich 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20200909232141.3099367-1-iii@linux.ibm.com