| Age | Commit message (Collapse) | Author |
|---|
|
Cache the dont_correlate() result once per symbol at the start of
correlate_symbols(). This reduces klp diff time on an arm64 LTO
vmlinux.o from 2m51s to 35s.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Only create prefix symbols for functions that have
__patchable_function_entries entries, since those are the only C
functions where prefix NOPs are intentional.
This both simplifies the detection and makes it more accurate.
Note that assembly functions using SYM_TYPED_FUNC_START() can also have
prefixed NOPs, but that macro already creates their __cfi_ symbols.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
With CFI+CALL_PADDING, Clang places .Ltmp labels at the start of the NOP
padding (offset 5) between the __cfi_ prefix and the function entry
point. get_func_prefix() only checks the immediately previous symbol,
so the intervening .Ltmp label causes it to miss the __cfi_ prefix
symbol.
This results in klp-diff not cloning the kCFI type hash into the
livepatch module, causing a CFI failure at module load when calling
callback functions through indirect calls:
CFI failure at __klp_enable_patch+0xab/0x140
(target: pre_patch_callback+0x0/0x80 [livepatch_combined];
expected type: 0xde073954)
Instead of walking backward through the section's symbol list, just use
find_func_containing() for the byte before the function. This works now
that __cfi_ symbols are being grown by objtool to fill the padding.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
For all CONFIG_CFI+CONFIG_CALL_PADDING configs, for C functions, the
__cfi_ symbols only cover the 5-byte kCFI type hash. After that there
also N bytes of NOP padding between the hash and the function entry
which aren't associated with any symbol.
The NOPs can be replaced with actual code at runtime. Without a symbol,
unwinders and tooling have no way of knowing where those bytes belong.
Grow the existing __cfi_* symbols to fill that gap.
Note that assembly functions with SYM_TYPED_FUNC_START() aren't affected
by this issue, their __cfi_ symbols also cover the padding.
Also, CONFIG_PREFIX_SYMBOLS has no reason to exist: CONFIG_CALL_PADDING
is what causes the compiler to emit NOP padding before function entry
(via -fpatchable-function-entry), so it's the right condition for
creating prefix symbols.
Remove CONFIG_PREFIX_SYMBOLS, as it's no longer needed. Simplify the
LONGEST_SYM_KUNIT_TEST dependency accordingly. Rework objtool's
arguments a bit to handle the variety of prefix/cfi-related cases.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
When computing klp checksums, instructions with non-relocated jump/call
destination offsets are problematic because the offset values can change
when surrounding code has moved, causing the function to be incorrectly
marked as changed.
Specifically, that includes jumps from alternatives to the end of the
alternative, which from objtool's perspective are jumps to the end of
the alternative instruction block in the original function.
Note that 'jump_dest' jumps don't include sibling calls (those use
call_dest), nor do they include jumps to/from .cold sub functions (those
are cross-section and need a reloc).
Fix it by hashing the opcode bytes (excluding the immediate operand)
along with a position-independent representation of the destination.
For calls, use the function name, and for jumps, use the destination's
offset within its function.
[Note the "9 bit hole" comment was wrong: it has been 8 bits since
commit 70589843b36f ("objtool: Add option to trace function validation")
added the 'trace' field. Adding the 4-bit 'immediate_len' field now
leaves a 4-bit hole.]
Fixes: 0d83da43b1e1 ("objtool/klp: Add --checksum option to generate per-function checksums")
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Alternative replacement instructions awkwardly have insn->sym set to the
function they get patched to rather than the symbol (or rather lack
thereof) they belong to in the file.
This makes it difficult to know where a given instruction actually
lives.
Add a new insn_sym() helper which preserves the existing semantic of
insn->sym. Rename insn->sym to insn->_sym, which contains the actual
ELF binary symbol (or NULL, for alternative replacements) an instruction
lives in.
The private insn->_sym value will be needed for a subsequent patch.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Add debugging messages to show how duplicate symbols get correlated, and
split the --debug feature into --debug-correlate and --debug-clone.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Rewrite the symbol correlation code, using a tiered list of
deterministic strategies in a loop. For duplicately named symbols, each
tier applies a filter with the goal of finding a 1:1 deterministic
correlation between the original and patched version of the symbol.
The three matching strategies are:
find_twin(): A funnel of progressively tighter filters. Candidates
with the same demangled name are counted at four levels: name, scope
(local-vs-global), file (strict file association), and checksum
(unchanged functions). The widest level that yields a 1:1 match wins,
narrower levels are only tried when the wider level is ambiguous.
find_twin_suffixed(): Uses already-correlated LLVM symbol pairs to map
.llvm.<hash> suffixes from orig to patched. Because all promoted
symbols from the same TU share the same hash, one correlated pair
seeds the mapping for the entire TU.
find_twin_positional(): Last resort, matches symbols by position among
same-named candidates, similar to livepatch sympos. Used for data
objects like __quirk variables where no deterministic filter can
distinguish the candidates.
Overall this works much better than the existing algorithm, particularly
with LTO kernels.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Start checksumming data objects in preparation for revamping the
correlation algorithm.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
The checksum functionality has been moved to "objtool klp checksum"
which is now used by klp-build. Remove the now-dead --checksum and
--debug-checksum options from the default objtool command.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Move the checksum functionality out of the main objtool command into a
new "objtool klp checksum" subcommand.
This has the benefit of making the code (and the patch generation
process itself) more modular.
For bisectability, both "objtool --checksum" and "objtool klp checksum"
work for now. The former will be removed after klp-build has been
converted to use the new subcommand.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
decode_sections() relies on CFI and cfi_hash initialization done
separately in check(), making it unusable outside of check().
Consolidate the initialization into decode_sections() and rename it to
decode_file(), and make it global along with free_insns() and
insn_reloc() for use by other objtool components -- namely, the checksum
code which will be moving to another file.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
In preparation for porting the checksum code to other arches, make its
functionality independent from the CFG reverse engineering code.
Move it into a standalone calculate_checksums() function which iterates
all functions and instructions directly, rather than being called inline
from do_validate_branch().
Since checksum_update_insn() is no longer called during CFG traversal,
it needs to manually iterate the alternatives.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Add an is_cold_func() helper. No functional changes intended.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Improve readability with a new is_alias_sym() helper.
No functional changes intended.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Clang generates anonymous data sections named .data..Lanon.<hash>.
These need section-symbol references in the same way as .data..Lubsan
(GCC) and .data..L__unnamed_ (Clang UBSAN) sections. Without this,
convert_reloc_sym() fails when processing relocations that reference
these sections.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
If an object file has no functions, objtool has nothing to checksum, so
it doesn't create the .discard.sym_checksum symbol.
Then when 'objtool klp diff' reads symbol checksums, it errors out due
to the missing .discard.sym_checksum section.
Instead, just create an empty checksum section to signal to
read_sym_checksums() that the file has been processed.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Instead of installing libsubcmd headers to a build output directory and
including from there, include directly from tools/lib/ where they
already exist. This fixes clangd indexing which otherwise can't find
libsubcmd headers.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Section symbols aren't grouped after their corresponding FILE symbols.
Their sym->file should really be NULL rather than whatever random FILE
happened to be last.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
In find_reloc_by_dest_range(), hash collisions can cause a high-offset
relocation to appear when probing a low-offset hash bucket.
Only return early when the best match found so far genuinely belongs to
the current bucket (its offset is within the bucket's stride range).
Otherwise, continue scanning later buckets which may contain
lower-offset matches.
This ensures the first reloc in the range gets returned.
Fixes: 74b873e49d92 ("objtool: Optimize find_rela_by_dest_range()")
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Use the section symbol's index instead of the old symbol's index when
updating the ELF relocation entry in convert_reloc_sym_to_secsym().
Found by Sashiko review.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Clang aggregates UBSAN type descriptors into shared anonymous
.data..L__unnamed_* sections. This data is used by UBSAN trap handlers.
When a changed function has an UBSAN bounds check, klp-diff clones the
entire UBSAN data section associated with the TU. Relocations within
the cloned section that reference named rodata objects in .rodata.cst*
(like 'exponent', 'pirq_ali_set.irqmap') become KLP relocations because
those objects now get correlated.
That results in a .klp.rela.vmlinux..data section which can easily have
thousands of KLP relocs, most of which are completely superfluous, used
by functions which aren't cloned to the patch module.
The .rodata.cst* sections are SHF_MERGE constant pool sections
containing small fixed-size data (lookup tables, bitmasks) that is only
read by value. Pointer identity is never relevant for these objects, so
correlating them is unnecessary.
Exclude .rodata.cst* objects from correlation so they get cloned as
local data instead of generating KLP relocations.
It might be possible to someday treat UBSAN data sections as special
sections, and only extract the few needed entries. But this works for
now.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
klp-diff treats all rodata as uncorrelated, so any reference to it uses
a duplicated copy rather than using a KLP reloc.
For the contents of the data itself, a duplicated copy is fine.
However, pointer comparisons (e.g., f->f_op == &foo_ops) are broken.
Fix it by correlating non-anonymous rodata objects.
Also, use a new find_symbol_containing_inclusive() helper for matching
the end of a symbol so bounds calculations don't get broken, for the
case where an array or other symbol's ending address is used as part of
a bounds calculation.
While these are really two distinct changes, they need to be done in the
same patch so as to avoid introducing bisection regressions.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Inline section_reference_needed() and is_reloc_allowed() into
convert_reloc_sym() and remove the redundant is_reloc_allowed() check in
clone_reloc().
Move the is_sec_sym() checks into the convert callees so they become
no-ops when the reloc is already in the right format. This allows
convert_reloc_sym() to unconditionally dispatch to the right converter
based on section type.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Move the sec->rodata marking from check.c to elf.c so it's set during
ELF reading rather than during the check pipeline. This makes the
rodata flag available to all objtool users, including klp-diff which
reads ELF files directly without running check().
Add an is_rodata_sec() helper to elf.h for consistency with
is_text_sec() and is_string_sec().
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Objtool has some hacks which NOP out certain calls/jumps and replace
their relocations with R_X86_64_NONE. The klp-diff relocation
extraction code will error out when trying to copy these relocations due
to their negative addend, which would only makes sense for a PC-relative
branch instruction. Just ignore them.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
.kcfi_traps contains references to kCFI trap instruction locations.
When a KCFI type check fails at an indirect call, the trap handler looks
up the faulting address in this section.
Add it to the special sections list so the entries get extracted for the
changed functions they reference.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Objtool is failing to extract text annotations which reference
.altinstr_replacement instructions:
1) Alternative replacement fake symbols are NOTYPE rather than FUNC,
and they don't have sym->included set, thus they aren't recognized
by should_keep_special_sym().
2) .discard.annotate_insn gets processed before .altinstr_replacement,
so the referenced (fake) symbols don't have clones yet.
Fix the first issue by checking for a valid clone instead of
sym->included and by accepting NOTYPE symbols when processing
.discard.annotate_insn.
Fix the second issue by deferring text annotation processing until after
the other special sections have been cloned.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
The XXH3 state allocated in checksum_init() is never freed. Free it in
checksum_finish().
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Fix NULL dereference when cloning a symbol from an empty section.
sec->data is only populated for sections with non-zero size.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
When a section is empty (e.g. only zero-length alternative
replacements), there are no symbols to convert a section symbol
reference to. Skip the reloc instead of erroring out.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
find_symbol_by_name() only returns the first match, so
--debug-checksum=<func> silently ignores any subsequent duplicately
named functions after the first.
Fix that, along with a new for_each_sym_by_name() helper.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Convert the callback-based iterate_sym_by_demangled_name() with a new
for_each_sym_by_demangled_name() macro. This eliminates the callback
struct/function and makes the code more compact and readable.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
create_fake_symbols() has two phases: creating symbols from
ANNOTATE_DATA_SPECIAL entries, and a fallback that uses sh_entsize for
special sections like .static_call_sites.
When .discard.annotate_data is absent, the function returns early,
skipping the entsize fallback and silently allowing unsupported
module-local static call keys through.
Fix it by jumping to the entsize phase instead of returning early.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Assisted-by: Claude:claude-4-opus
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Clang emits various .L-prefixed local symbols beyond .Ltmp*, such as
.L__const.* for local constant data. These are assembler-local labels
not present in kallsyms, so they can never be resolved at module load
time.
Broaden the check from .Ltmp* to all .L* symbols so they get cloned into
the patch module instead.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Clang LTO uses __UNIQUE_ID() to generate some uniquely named wrapper
functions, like initstubs. If they're uncorrelated, prevent them from
being reported as new functions and included unnecessarily.
Note that dont_correlate() already includes prefix functions, so prefix
functions are still being ignored here.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
With LTO, the initcall infrastructure generates __initstub__kmod_*
wrapper functions in .init.text. These are the LTO equivalent of
__initcall__kmod_* data pointers, which are already excluded from
correlation.
These are __init functions whose memory is freed after boot, so there's
no reason to include or reference them in a livepatch module.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Some arch/x86/crypto/*.S files define local .set/.equ constants that get
duplicated in vmlinux.o. This causes klp-diff to fail with "Multiple
correlation candidates" errors since it can't uniquely match these
between orig and patched builds.
Skip ABS symbols in dont_correlate(). They're purely compile-time
assembly constants that are never referenced by relocations, so they
don't need correlation.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Symbols created by __ADDRESSABLE() are only used to convince the
toolchain not to optimize out the referenced symbol.
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
While there was once a section named .data.once, it has since been
renamed to .data..once with commit dbefa1f31a91 ("Rename .data.once to
.data..once to fix resetting WARN*_ONCE"). Fix it.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
For naming function-local static locals, GCC uses <var>.<id>, e.g.
__already_done.15, while Clang uses <func>.<var> with optional .<id>,
e.g. create_worker.__already_done.111
The existing is_uncorrelated_static_local() check only matches the GCC
convention where the variable name is a prefix. Handle both cases by
checking for a prefix match (GCC) and by checking after the first dot
separator (Clang).
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull objtool updates from Ingo Molnar:
- KLP support updates and fixes (Song Liu)
- KLP-build script updates and fixes (Joe Lawrence)
- Support Clang RAX DRAP sequence, to address clang false positive
(Josh Poimboeuf)
- Reorder ORC register numbering to match regular x86 register
numbering (Josh Poimboeuf)
- Misc cleanups (Wentong Tian, Song Liu)
* tag 'objtool-core-2026-04-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
objtool/x86: Reorder ORC register numbering
objtool: Support Clang RAX DRAP sequence
livepatch/klp-build: report patch validation fuzz
livepatch/klp-build: add terminal color output
livepatch/klp-build: provide friendlier error messages
livepatch/klp-build: improve short-circuit validation
livepatch/klp-build: fix shellcheck complaints
livepatch/klp-build: add Makefile with check target
livepatch/klp-build: add grep-override function
livepatch/klp-build: switch to GNU patch and recountdiff
livepatch/klp-build: support patches that add/remove files
objtool/klp: Correlate locals to globals
objtool/klp: Match symbols based on demangled_name for global variables
objtool/klp: Remove .llvm suffix in demangle_name()
objtool/klp: Also demangle global objects
objtool/klp: Use sym->demangled_name for symbol_name hash
objtool/klp: Remove trailing '_' in demangle_name()
objtool/klp: Remove redundant strcmp() in correlate_symbols()
objtool: Use section/symbol type helpers
|
|
This series cleans up some of the special user copy functions naming and
semantics. In particular, get rid of the (very traditional) double
underscore names and behavior: the whole "optimize away the range check"
model has been largely excised from the other user accessors because
it's so subtle and can be unsafe, but also because it's just not a
relevant optimization any more.
To do that, a couple of drivers that misused the "user" copies as kernel
copies in order to get non-temporal stores had to be fixed up, but that
kind of code should never have been allowed anyway.
The x86-only "nocache" version was also renamed to more accurately
reflect what it actually does.
This was all done because I looked at this code due to a report by Jann
Horn, and I just couldn't stand the inconsistent naming, the horrible
semantics, and the random misuse of these functions. This code should
probably be cleaned up further, but it's at least slightly closer to
normal semantics.
I had a more intrusive series that went even further in trying to
normalize the semantics, but that ended up hitting so many other
inconsistencies between different architectures in this area (eg
'size_t' vs 'unsigned long' vs 'int' as size arguments, and various
iovec check differences that Vasily Gorbik pointed out) that I ended up
with this more limited version that fixed the worst of the issues.
Reported-by: Jann Horn <jannh@google.com>
Tested-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/all/CAHk-=wgg1QVWNWG-UCFo1hx0zqrPnB3qhPzUTrWNft+MtXQXig@mail.gmail.com/
* nocache-cleanup:
x86-64/arm64/powerpc: clean up and rename __copy_from_user_flushcache
x86: rename and clean up __copy_from_user_inatomic_nocache()
x86-64: rename misleadingly named '__copy_user_nocache()' function
|
|
This function was a masterclass in bad naming, for various historical
reasons.
It claimed to be a non-cached user copy. It is literally _neither_ of
those things. It's a specialty memory copy routine that uses
non-temporal stores for the destination (but not the source), and that
does exception handling for both source and destination accesses.
Also note that while it works for unaligned targets, any unaligned parts
(whether at beginning or end) will not use non-temporal stores, since
only words and quadwords can be non-temporal on x86.
The exception handling means that it _can_ be used for user space
accesses, but not on its own - it needs all the normal "start user space
access" logic around it.
But typically the user space access would be the source, not the
non-temporal destination. That was the original intention of this,
where the destination was some fragile persistent memory target that
needed non-temporal stores in order to catch machine check exceptions
synchronously and deal with them gracefully.
Thus that non-descriptive name: one use case was to copy from user space
into a non-cached kernel buffer. However, the existing users are a mix
of that intended use-case, and a couple of random drivers that just did
this as a performance tweak.
Some of those random drivers then actively misused the user copying
version (with STAC/CLAC and all) to do kernel copies without ever even
caring about the exception handling, _just_ for the non-temporal
destination.
Rename it as a first small step to actually make it halfway sane, and
change the prototype to be more normal: it doesn't take a user pointer
unless the caller has done the proper conversion, and the argument size
is the full size_t (it still won't actually copy more than 4GB in one
go, but there's also no reason to silently truncate the size argument in
the caller).
Finally, use this now sanely named function in the NTB code, which
mis-used a user copy version (with STAC/CLAC and all) of this interface
despite it not actually being a user copy at all.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Reorder the ORC register values so their ordering matches the x86
instruction set register encodings.
No functional change intended.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
|
|
Recent Clang can use RAX as a temporary register for the DRAP stack
alignment sequence. Add support for that.
Fixes the following warning:
vmlinux.o: error: objtool: vmw_host_printf+0xd: unknown CFA base reg 0
Closes: https://lore.kernel.org/cefefdd1-7b82-406d-8ff4-e4b167e45ee6@app.fastmail.com
Reported-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/3f33dc720b83dc6d3a2b7094f75a5c90a0b1cbc5.1773708458.git.jpoimboe@kernel.org
|
|
With Clang, there can be a conditional forward jump between the load of
the jump table address and the indirect branch.
Fixes the following warning:
vmlinux.o: warning: objtool: ___bpf_prog_run+0x1c5: sibling call from callable instruction with modified stack frame
Reported-by: Arnd Bergmann <arnd@arndb.de>
Closes: https://lore.kernel.org/a426d669-58bb-4be1-9eaa-6f3d83109e2d@app.fastmail.com
Link: https://patch.msgid.link/7d8600caed08901b6679767488acd639f6df9688.1773071992.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
The elf_create_file() function fails with EINVAL when the build directory
path is long enough to truncate the "XXXXXX" suffix in the 256-byte
tmp_name buffer.
Simplify the code to remove the unnecessary dirname()/basename() split
and concatenation. Instead, allocate the exact number of bytes needed for
the path.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/20260310203751.1479229-3-joe.lawrence@redhat.com
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
Commit 356e4b2f5b80 ("objtool: Fix data alignment in elf_add_data()")
corrected the alignment of data within a section (honoring the section's
sh_addralign). Apply the same alignment when klp-diff mode clones a
symbol, adjusting the new symbol's offset for the output section's
sh_addralign.
Fixes: dd590d4d57eb ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/20260310203751.1479229-2-joe.lawrence@redhat.com
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull objtool fixes from Ingo Molnar:
- Fix cross-build bug by using HOSTCFLAGS for HAVE_XXHASH test
- Fix klp bug by fixing detection of corrupt static branch/call entries
- Handle unsupported pr_debug() usage more gracefully
- Fix hypothetical klp bug by avoiding NULL pointer dereference when
printing code symbol name
- Fix data alignment bug in elf_add_data() causing mangled strings
- Fix confusing ERROR_INSN() error message
- Handle unexpected Clang RSP musical chairs causing false positive
warnings
- Fix another objtool stack overflow in validate_branch()
* tag 'objtool-urgent-2026-03-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
objtool: Fix another stack overflow in validate_branch()
objtool: Handle Clang RSP musical chairs
objtool: Fix ERROR_INSN() error message
objtool: Fix data alignment in elf_add_data()
objtool: Use HOSTCFLAGS for HAVE_XXHASH test
objtool/klp: Avoid NULL pointer dereference when printing code symbol name
objtool/klp: Disable unsupported pr_debug() usage
objtool/klp: Fix detection of corrupt static branch/call entries
|
