linux-toradex.git/scripts/setlocalversion, branch v5.17-rc4

kbuild: do not quote string values in include/config/auto.conf

2022-01-08T09:03:57+00:00

The previous commit fixed up all shell scripts to not include
include/config/auto.conf.

Now that include/config/auto.conf is only included by Makefiles,
we can change it into a more Make-friendly form.

Previously, Kconfig output string values enclosed with double-quotes
(both in the .config and include/config/auto.conf):

    CONFIG_X="foo bar"

Unlike shell, Make handles double-quotes (and single-quotes as well)
verbatim. We must rip them off when used.

There are some patterns:

  [1] $(patsubst "%",%,$(CONFIG_X))
  [2] $(CONFIG_X:"%"=%)
  [3] $(subst ",,$(CONFIG_X))
  [4] $(shell echo $(CONFIG_X))

These are not only ugly, but also fragile.

[1] and [2] do not work if the value contains spaces, like
   CONFIG_X=" foo bar "

[3] does not work correctly if the value contains double-quotes like
   CONFIG_X="foo\"bar"

[4] seems to work better, but has a cost of forking a process.

Anyway, quoted strings were always PITA for our Makefiles.

This commit changes Kconfig to stop quoting in include/config/auto.conf.

These are the string type symbols referenced in Makefiles or scripts:

    ACPI_CUSTOM_DSDT_FILE
    ARC_BUILTIN_DTB_NAME
    ARC_TUNE_MCPU
    BUILTIN_DTB_SOURCE
    CC_IMPLICIT_FALLTHROUGH
    CC_VERSION_TEXT
    CFG80211_EXTRA_REGDB_KEYDIR
    EXTRA_FIRMWARE
    EXTRA_FIRMWARE_DIR
    EXTRA_TARGETS
    H8300_BUILTIN_DTB
    INITRAMFS_SOURCE
    LOCALVERSION
    MODULE_SIG_HASH
    MODULE_SIG_KEY
    NDS32_BUILTIN_DTB
    NIOS2_DTB_SOURCE
    OPENRISC_BUILTIN_DTB
    SOC_CANAAN_K210_DTB_SOURCE
    SYSTEM_BLACKLIST_HASH_LIST
    SYSTEM_REVOCATION_KEYS
    SYSTEM_TRUSTED_KEYS
    TARGET_CPU
    UNUSED_KSYMS_WHITELIST
    XILINX_MICROBLAZE0_FAMILY
    XILINX_MICROBLAZE0_HW_VER
    XTENSA_VARIANT_NAME

I checked them one by one, and fixed up the code where necessary.

Signed-off-by: Masahiro Yamada

kbuild: do not include include/config/auto.conf from shell scripts

2022-01-08T08:46:36+00:00

Richard Weinberger pointed out the risk of sourcing the kernel config
from shell scripts [1], and proposed some patches [2], [3]. It is a good
point, but it took a long time because I was wondering how to fix this.

This commit goes with simple grep approach because there are only a few
scripts including the kernel configuration.

scripts/link_vmlinux.sh has references to a bunch of CONFIG options,
all of which are boolean. I added is_enabled() helper as
scripts/package/{mkdebian,builddeb} do.

scripts/gen_autoksyms.sh uses 'eval', stating "to expand the whitelist
path". I removed it since it is the issue we are trying to fix.

I was a bit worried about the cost of invoking the grep command over
again. I extracted the grep parts from it, and measured the cost. It
was approximately 0.03 sec, which I hope is acceptable.

[test code]

  $ cat test-grep.sh
  #!/bin/sh

  is_enabled() {
          grep -q "^$1=y" include/config/auto.conf
  }

  is_enabled CONFIG_LTO_CLANG
  is_enabled CONFIG_LTO_CLANG
  is_enabled CONFIG_STACK_VALIDATION
  is_enabled CONFIG_UNWINDER_ORC
  is_enabled CONFIG_FTRACE_MCOUNT_USE_OBJTOOL
  is_enabled CONFIG_VMLINUX_VALIDATION
  is_enabled CONFIG_FRAME_POINTER
  is_enabled CONFIG_GCOV_KERNEL
  is_enabled CONFIG_LTO_CLANG
  is_enabled CONFIG_RETPOLINE
  is_enabled CONFIG_X86_SMAP
  is_enabled CONFIG_LTO_CLANG
  is_enabled CONFIG_VMLINUX_MAP
  is_enabled CONFIG_KALLSYMS_ALL
  is_enabled CONFIG_KALLSYMS_ABSOLUTE_PERCPU
  is_enabled CONFIG_KALLSYMS_BASE_RELATIVE
  is_enabled CONFIG_DEBUG_INFO_BTF
  is_enabled CONFIG_KALLSYMS
  is_enabled CONFIG_DEBUG_INFO_BTF
  is_enabled CONFIG_BPF
  is_enabled CONFIG_BUILDTIME_TABLE_SORT
  is_enabled CONFIG_KALLSYMS

  $ time ./test-grep.sh
  real    0m0.036s
  user    0m0.027s
  sys     m0.009s

[1]: https://lore.kernel.org/all/1919455.eZKeABUfgV@blindfold/
[2]: https://lore.kernel.org/all/20180219092245.26404-1-richard@nod.at/
[3]: https://lore.kernel.org/all/20210920213957.1064-2-richard@nod.at/
Signed-off-by: Masahiro Yamada 
Reviewed-by: Nicolas Schier

scripts/setlocalversion: fix a bug when LOCALVERSION is empty

2021-07-18T14:48:14+00:00

The commit 042da426f8eb ("scripts/setlocalversion: simplify the short
version part") reduces indentation. Unfortunately, it also changes behavior
in a subtle way - if the user has empty "LOCALVERSION" variable, the plus
sign is appended to the kernel version. It wasn't appended before.

This patch reverts to the old behavior - we append the plus sign only if
the LOCALVERSION variable is not set.

Fixes: 042da426f8eb ("scripts/setlocalversion: simplify the short version part")
Signed-off-by: Mikulas Patocka 
Signed-off-by: Masahiro Yamada

scripts/setlocalversion: simplify the short version part

2021-05-26T19:01:51+00:00

Reduce the indentation.

Signed-off-by: Masahiro Yamada 
Reviewed-by: Greg Kroah-Hartman 
Reviewed-by: Nico Schottelius

scripts/setlocalversion: factor out 12-chars hash construction

2021-05-26T19:01:51+00:00

Both of if and else parts append exactly 12 hex chars, but in
different ways.

Factor out the else part because we need to support it without relying
on git-describe. Remove the --abbrev=12 option since we do not use the
hash from git-describe anyway.

Signed-off-by: Masahiro Yamada 
Reviewed-by: Greg Kroah-Hartman 
Reviewed-by: Nico Schottelius

scripts/setlocalversion: add more comments to -dirty flag detection

2021-05-26T19:01:51+00:00

This script stumbled on the read-only source tree over again:

 - a2bb90a08cb3 ("kbuild: fix delay in setlocalversion on readonly
   source")

 - cdf2bc632ebc ("scripts/setlocalversion on write-protected source
   tree")

 - 8ef14c2c41d9 ("Revert "scripts/setlocalversion: git: Make -dirty
   check more robust"")

 - ff64dd485730 ("scripts/setlocalversion: Improve -dirty check with
   git-status --no-optional-locks")

Add comments to clarify that this script should never ever try to write
to the source tree.

'git describe --dirty' might look as a simple solution for appending
the -dirty string, but we cannot use it because it creates the
.git/index.lock file.

Signed-off-by: Masahiro Yamada 
Reviewed-by: Greg Kroah-Hartman 
Reviewed-by: Nico Schottelius

scripts/setlocalversion: remove workaround for old make-kpkg

2021-05-26T19:01:51+00:00

This reverts commit b052ce4c840e ("kbuild: fix false positive -dirty
tag caused by make-kpkg").

If I understand correctly, this problem occurred in very old versions
of make-kpkg. When I tried a newer version, make-kpkg did not touch
scripts/package/Makefile.

Anyway, Debian uses 'make deb-pkg' instead of make-kpkg these days.

Debian handbook [1] mentions it as "the good old days":

 "CULTURE The good old days of kernel-package

  Before the Linux build system gained the ability to build proper
  Debian packages, the recommended way to build such packages was to
  use make-kpkg from the kernel-package package."

[1]: https://debian-handbook.info/browse/stable/sect.kernel-compilation.html

Signed-off-by: Masahiro Yamada 
Reviewed-by: Greg Kroah-Hartman 
Reviewed-by: Nico Schottelius

scripts/setlocalversion: remove mercurial, svn and git-svn supports

2021-05-26T19:01:51+00:00

The mercurial, svn, git-svn supports were added by the following commits:

 - 3dce174cfcba ("kbuild: support mercurial in setlocalversion")

 - ba3d05fb6369 ("kbuild: add svn revision information to setlocalversion")

 - ff80aa97c9b4 ("setlocalversion: add git-svn support")

They did not explain why they are useful for the kernel source tree.
Let's revert all of them, and see if somebody will complain about it.

Signed-off-by: Masahiro Yamada 
Reviewed-by: Greg Kroah-Hartman 
Reviewed-by: Nico Schottelius

kbuild: replace LANG=C with LC_ALL=C

2021-05-01T15:43:35+00:00

LANG gives a weak default to each LC_* in case it is not explicitly
defined. LC_ALL, if set, overrides all other LC_* variables.

  LANG  <  LC_CTYPE, LC_COLLATE, LC_MONETARY, LC_NUMERIC, ...  <  LC_ALL

This is why documentation such as [1] suggests to set LC_ALL in build
scripts to get the deterministic result.

LANG=C is not strong enough to override LC_* that may be set by end
users.

[1]: https://reproducible-builds.org/docs/locales/

Signed-off-by: Masahiro Yamada 
Acked-by: Michael Ellerman  (powerpc)
Reviewed-by: Matthias Maennich 
Acked-by: Matthieu Baerts  (mptcp)
Reviewed-by: Greg Kroah-Hartman

scripts/setlocalversion: make git describe output more reliable

2020-09-24T17:28:12+00:00

When building for an embedded target using Yocto, we're sometimes
observing that the version string that gets built into vmlinux (and
thus what uname -a reports) differs from the path under /lib/modules/
where modules get installed in the rootfs, but only in the length of
the -gabc123def suffix. Hence modprobe always fails.

The problem is that Yocto has the concept of "sstate" (shared state),
which allows different developers/buildbots/etc. to share build
artifacts, based on a hash of all the metadata that went into building
that artifact - and that metadata includes all dependencies (e.g. the
compiler used etc.). That normally works quite well; usually a clean
build (without using any sstate cache) done by one developer ends up
being binary identical to a build done on another host. However, one
thing that can cause two developers to end up with different builds
[and thus make one's vmlinux package incompatible with the other's
kernel-dev package], which is not captured by the metadata hashing, is
this `git describe`: The output of that can be affected by

(1) git version: before 2.11 git defaulted to a minimum of 7, since
2.11 (git.git commit e6c587) the default is dynamic based on the
number of objects in the repo
(2) hence even if both run the same git version, the output can differ
based on how many remotes are being tracked (or just lots of local
development branches or plain old garbage)
(3) and of course somebody could have a core.abbrev config setting in
~/.gitconfig

So in order to avoid `uname -a` output relying on such random details
of the build environment which are rather hard to ensure are
consistent between developers and buildbots, make sure the abbreviated
sha1 always consists of exactly 12 hex characters. That is consistent
with the current rule for -stable patches, and is almost always enough
to identify the head commit unambigously - in the few cases where it
does not, the v5.4.3-00021- prefix would certainly nail it down.

Signed-off-by: Rasmus Villemoes 
Signed-off-by: Masahiro Yamada