linux-toradex.git/kernel/extable.c, branch v5.19-rc1

lkdtm: Really write into kernel text in WRITE_KERN

2022-02-16T12:25:12+00:00

WRITE_KERN is supposed to overwrite some kernel text, namely
do_overwritten() function.

But at the time being it overwrites do_overwritten() function
descriptor, not function text.

Fix it by dereferencing the function descriptor to obtain
function text pointer. Export dereference_function_descriptor()
for when LKDTM is built as a module.

And make do_overwritten() noinline so that it is really
do_overwritten() which is called by lkdtm_WRITE_KERN().

Signed-off-by: Christophe Leroy 
Acked-by: Kees Cook 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/31e58eaffb5bc51c07d8d4891d1982100ade8cfc.1644928018.git.christophe.leroy@csgroup.eu

asm-generic: Refactor dereference_[kernel]_function_descriptor()

2022-02-16T12:25:11+00:00

dereference_function_descriptor() and
dereference_kernel_function_descriptor() are identical on the
three architectures implementing them.

Make them common and put them out-of-line in kernel/extable.c
which is one of the users and has similar type of functions.

Signed-off-by: Christophe Leroy 
Reviewed-by: Kees Cook 
Reviewed-by: Arnd Bergmann 
Acked-by: Helge Deller 
Signed-off-by: Michael Ellerman 
Link: https://lore.kernel.org/r/449db09b2eba57f4ab05f80102a67d8675bc8bcd.1644928018.git.christophe.leroy@csgroup.eu

extable: use is_kernel_text() helper

2021-11-09T18:02:51+00:00

The core_kernel_text() should check the gate area, as it is part of kernel
text range, use is_kernel_text() in core_kernel_text().

Link: https://lkml.kernel.org/r/20210930071143.63410-9-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang 
Reviewed-by: Sergey Senozhatsky 
Cc: Steven Rostedt 
Cc: Alexander Potapenko 
Cc: Alexei Starovoitov 
Cc: Andrey Konovalov 
Cc: Andrey Ryabinin 
Cc: Arnd Bergmann 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Christophe Leroy 
Cc: "David S. Miller" 
Cc: Dmitry Vyukov 
Cc: Ingo Molnar 
Cc: Ivan Kokshaysky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Paul Mackerras 
Cc: Petr Mladek 
Cc: Richard Henderson 
Cc: Thomas Gleixner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

sections: move is_kernel_inittext() into sections.h

2021-11-09T18:02:50+00:00

The is_kernel_inittext() and init_kernel_text() are with same
functionality, let's just keep is_kernel_inittext() and move it into
sections.h, then update all the callers.

Link: https://lkml.kernel.org/r/20210930071143.63410-5-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang 
Reviewed-by: Sergey Senozhatsky 
Cc: Ingo Molnar 
Cc: Thomas Gleixner 
Cc: Arnd Bergmann 
Cc: Alexander Potapenko 
Cc: Alexei Starovoitov 
Cc: Andrey Konovalov 
Cc: Andrey Ryabinin 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Christophe Leroy 
Cc: "David S. Miller" 
Cc: Dmitry Vyukov 
Cc: Ivan Kokshaysky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Paul Mackerras 
Cc: Petr Mladek 
Cc: Richard Henderson 
Cc: Steven Rostedt 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

sections: move and rename core_kernel_data() to is_kernel_core_data()

2021-11-09T18:02:50+00:00

Move core_kernel_data() into sections.h and rename it to
is_kernel_core_data(), also make it return bool value, then update all the
callers.

Link: https://lkml.kernel.org/r/20210930071143.63410-4-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang 
Reviewed-by: Sergey Senozhatsky 
Cc: Arnd Bergmann 
Cc: Steven Rostedt 
Cc: Ingo Molnar 
Cc: "David S. Miller" 
Cc: Alexander Potapenko 
Cc: Alexei Starovoitov 
Cc: Andrey Konovalov 
Cc: Andrey Ryabinin 
Cc: Benjamin Herrenschmidt 
Cc: Borislav Petkov 
Cc: Christophe Leroy 
Cc: Dmitry Vyukov 
Cc: Ivan Kokshaysky 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Michal Simek 
Cc: Paul Mackerras 
Cc: Petr Mladek 
Cc: Richard Henderson 
Cc: Thomas Gleixner 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

mm: create a new system state and fix core_kernel_text()

2021-11-06T20:30:38+00:00

core_kernel_text() considers that until system_state in at least
SYSTEM_RUNNING, init memory is valid.

But init memory is freed a few lines before setting SYSTEM_RUNNING, so
we have a small period of time when core_kernel_text() is wrong.

Create an intermediate system state called SYSTEM_FREEING_INIT that is
set before starting freeing init memory, and use it in
core_kernel_text() to report init memory invalid earlier.

Link: https://lkml.kernel.org/r/9ecfdee7dd4d741d172cb93ff1d87f1c58127c9a.1633001016.git.christophe.leroy@csgroup.eu
Signed-off-by: Christophe Leroy 
Cc: Gerald Schaefer 
Cc: Kefeng Wang 
Cc: Benjamin Herrenschmidt 
Cc: Paul Mackerras 
Cc: Heiko Carstens 
Signed-off-by: Andrew Morton 
Signed-off-by: Linus Torvalds

kernel/extable.c: use address-of operator on section symbols

2020-04-07T17:43:42+00:00

Clang warns:

../kernel/extable.c:37:52: warning: array comparison always evaluates to
a constant [-Wtautological-compare]
        if (main_extable_sort_needed && __stop___ex_table > __start___ex_table) {
                                                          ^
1 warning generated.

These are not true arrays, they are linker defined symbols, which are just
addresses.  Using the address of operator silences the warning and does
not change the resulting assembly with either clang/ld.lld or gcc/ld
(tested with diff + objdump -Dr).

Suggested-by: Nick Desaulniers 
Signed-off-by: Nathan Chancellor 
Signed-off-by: Andrew Morton 
Reviewed-by: Andrew Morton 
Link: https://github.com/ClangBuiltLinux/linux/issues/892
Link: http://lkml.kernel.org/r/20200219202036.45702-1-natechancellor@gmail.com
Signed-off-by: Linus Torvalds

bpf: Remove bpf_image tree

2020-03-13T19:49:52+00:00

Now that we have all the objects (bpf_prog, bpf_trampoline,
bpf_dispatcher) linked in bpf_tree, there's no need to have
separate bpf_image tree for images.

Reverting the bpf_image tree together with struct bpf_image,
because it's no longer needed.

Also removing bpf_image_alloc function and adding the original
bpf_jit_alloc_exec_page interface instead.

The kernel_text_address function can now rely only on is_bpf_text_address,
because it checks the bpf_tree that contains all the objects.

Keeping bpf_image_ksym_add and bpf_image_ksym_del because they are
useful wrappers with perf's ksymbol interface calls.

Signed-off-by: Jiri Olsa 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20200312195610.346362-13-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov

bpf: Allow to resolve bpf trampoline and dispatcher in unwind

2020-01-25T15:12:40+00:00

When unwinding the stack we need to identify each address
to successfully continue. Adding latch tree to keep trampolines
for quick lookup during the unwind.

The patch uses first 48 bytes for latch tree node, leaving 4048
bytes from the rest of the page for trampoline or dispatcher
generated code.

It's still enough not to affect trampoline and dispatcher progs
maximum counts.

Signed-off-by: Jiri Olsa 
Signed-off-by: Alexei Starovoitov 
Link: https://lore.kernel.org/bpf/20200123161508.915203-3-jolsa@kernel.org

bpf: Add support for BTF pointers to x86 JIT

2019-10-17T14:44:36+00:00

Pointer to BTF object is a pointer to kernel object or NULL.
Such pointers can only be used by BPF_LDX instructions.
The verifier changed their opcode from LDX|MEM|size
to LDX|PROBE_MEM|size to make JITing easier.
The number of entries in extable is the number of BPF_LDX insns
that access kernel memory via "pointer to BTF type".
Only these load instructions can fault.
Since x86 extable is relative it has to be allocated in the same
memory region as JITed code.
Allocate it prior to last pass of JITing and let the last pass populate it.
Pointer to extable in bpf_prog_aux is necessary to make page fault
handling fast.
Page fault handling is done in two steps:
1. bpf_prog_kallsyms_find() finds BPF program that page faulted.
   It's done by walking rb tree.
2. then extable for given bpf program is binary searched.
This process is similar to how page faulting is done for kernel modules.
The exception handler skips over faulting x86 instruction and
initializes destination register with zero. This mimics exact
behavior of bpf_probe_read (when probe_kernel_read faults dest is zeroed).

JITs for other architectures can add support in similar way.
Until then they will reject unknown opcode and fallback to interpreter.

Since extable should be aligned and placed near JITed code
make bpf_jit_binary_alloc() return 4 byte aligned image offset,
so that extable aligning formula in bpf_int_jit_compile() doesn't need
to rely on internal implementation of bpf_jit_binary_alloc().
On x86 gcc defaults to 16-byte alignment for regular kernel functions
due to better performance. JITed code may be aligned to 16 in the future,
but it will use 4 in the meantime.

Signed-off-by: Alexei Starovoitov 
Signed-off-by: Daniel Borkmann 
Acked-by: Andrii Nakryiko 
Acked-by: Martin KaFai Lau 
Link: https://lore.kernel.org/bpf/20191016032505.2089704-10-ast@kernel.org