linux-toradex.git/tools/testing/kunit, branch v5.9-rc6

Merge tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

2020-08-05T17:07:39+00:00

Pull kunit updates from Shuah Khan:

 - Add a generic kunit_resource API extending it to support resources
   that are passed in to kunit in addition kunit allocated resources. In
   addition, KUnit resources are now refcounted to avoid passed in
   resources being released while in use by kunit.

 - Add support for named resources.

 - Important bug fixes from Brendan Higgins and Will Chen

* tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
  kunit: tool: fix improper treatment of file location
  kunit: tool: fix broken default args in unit tests
  kunit: capture stderr on all make subprocess calls
  Documentation: kunit: Remove references to --defconfig
  kunit: add support for named resources
  kunit: generalize kunit_resource API beyond allocated resources

kunit: tool: fix improper treatment of file location

2020-07-17T20:17:49+00:00

Commit 01397e822af4 ("kunit: Fix TabError, remove defconfig code and
handle when there is no kunitconfig") and commit 45ba7a893ad8 ("kunit:
kunit_tool: Separate out config/build/exec/parse") introduced two
closely related issues which built off of each other: they excessively
created the build directory when not present and modified a constant
(constants in Python only exist by convention).

Together these issues broken a number of unit tests for KUnit tool, so
fix them.

Fixed up commit log to fic checkpatch commit description style error.
Shuah Khan 

Fixes: 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig")
Fixes: 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse")
Signed-off-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: tool: fix broken default args in unit tests

2020-07-17T20:17:42+00:00

Commit ddbd60c779b4 ("kunit: use --build_dir=.kunit as default") changed
the default build directory for KUnit tests, but failed to update
associated unit tests for kunit_tool, so update them.

Fixes: ddbd60c779b4 ("kunit: use --build_dir=.kunit as default")
Signed-off-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: capture stderr on all make subprocess calls

2020-07-17T20:13:19+00:00

Direct stderr to subprocess.STDOUT so error messages get included in the
subprocess.CalledProcessError exceptions output field. This results in
more meaningful error messages for the user.

This is already being done in the make_allyesconfig method. Do the same
for make_mrproper, make_olddefconfig, and make methods.

With this, failures on unclean trees [1] will give users an error
message that includes:
"The source tree is not clean, please run 'make ARCH=um mrproper'"

[1] https://bugzilla.kernel.org/show_bug.cgi?id=205219

Signed-off-by: Will Chen 
Reviewed-by: Brendan Higgins 
Tested-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: kunit_tool: Fix invalid result when build fails

2020-06-26T20:29:31+00:00

When separating out different phases of running tests[1]
(build/exec/parse/etc), the format of the KunitResult tuple changed
(adding an elapsed_time variable). This is not populated during a build
failure, causing kunit.py to crash.

This fixes [1] to probably populate the result variable, causing a
failing build to be reported properly.

[1]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=45ba7a893ad89114e773b3dc32f6431354c465d6

Signed-off-by: David Gow 
Reviewed-by: Brendan Higgins 
Tested-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: show error if kunit results are not present

2020-06-26T20:29:10+00:00

Currently, if the kernel is configured incorrectly or if it crashes before any
kunit tests are run, kunit finishes without error, reporting
that 0 test cases were run.

To fix this, an error is shown when the tap header is not found, which
indicates that kunit was not able to run at all.

Signed-off-by: Uriel Guajardo 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: kunit_config: Fix parsing of CONFIG options with space

2020-06-26T20:27:35+00:00

Commit 8b59cd81dc5e ("kbuild: ensure full rebuild when the compiler is
updated") introduced a new CONFIG option CONFIG_CC_VERSION_TEXT. On my
system, this is set to "gcc (GCC) 10.1.0" which breaks KUnit config
parsing which did not like the spaces in the string.

Fix this by updating the regex to allow strings containing spaces.

Fixes: 8b59cd81dc5e ("kbuild: ensure full rebuild when the compiler is updated")
Signed-off-by: Rikard Falkeborn 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig

2020-06-01T20:14:07+00:00

The identation before this code
(`if not os.path.exists(cli_args.build_dir):``)
was with spaces instead of tabs after fixed up merge conflits,
this commit revert spaces to tabs:

[iha@bbking linux]$ tools/testing/kunit/kunit.py run
  File "tools/testing/kunit/kunit.py", line 247
    if not linux:
                ^
TabError: inconsistent use of tabs and spaces in indentation

[iha@bbking linux]$ tools/testing/kunit/kunit.py run
Traceback (most recent call last):
  File "tools/testing/kunit/kunit.py", line 338, in 
    main(sys.argv[1:])
  File "tools/testing/kunit/kunit.py", line 215, in main
    add_config_opts(config_parser)

[iha@bbking linux]$ tools/testing/kunit/kunit.py run
Traceback (most recent call last):
  File "tools/testing/kunit/kunit.py", line 337, in 
    main(sys.argv[1:])
  File "tools/testing/kunit/kunit.py", line 255, in main
    result = run_tests(linux, request)
  File "tools/testing/kunit/kunit.py", line 133, in run_tests
    request.defconfig,
AttributeError: 'KunitRequest' object has no attribute 'defconfig'

Handles when there is no .kunitconfig, the error due to merge conflicts
between the following:

commit 9bdf64b35117 ("kunit: use KUnit defconfig by default")
commit 45ba7a893ad8 ("kunit: kunit_tool: Separate out
	config/build/exec/parse")

[iha@bbking linux]$ tools/testing/kunit/kunit.py run
Traceback (most recent call last):
  File "tools/testing/kunit/kunit.py", line 335, in 
    main(sys.argv[1:])
  File "tools/testing/kunit/kunit.py", line 246, in main
    linux = kunit_kernel.LinuxSourceTree()
  File "../tools/testing/kunit/kunit_kernel.py", line 109, in __init__
    self._kconfig.read_from_file(kunitconfig_path)
  File "t../ools/testing/kunit/kunit_config.py", line 88, in read_from_file
    with open(path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '.kunit/.kunitconfig'

Signed-off-by: Vitor Massaru Iha 
Signed-off-by: Shuah Khan

kunit: use KUnit defconfig by default

2020-05-22T22:23:28+00:00

To improve the usability of KUnit, defconfig is used
by default if no kunitconfig is present.

 * https://bugzilla.kernel.org/show_bug.cgi?id=205259

Fixed up minor merge conflicts - Shuah Khan 

Signed-off-by: Vitor Massaru Iha 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan

kunit: use --build_dir=.kunit as default

2020-05-22T20:52:06+00:00

To make KUnit easier to use, and to avoid overwriting object and
.config files, the default KUnit build directory is set to .kunit

 * Related bug: https://bugzilla.kernel.org/show_bug.cgi?id=205221

Fixed up minor merge conflicts - Shuah Khan 

Signed-off-by: Vitor Massaru Iha 
Reviewed-by: Brendan Higgins 
Signed-off-by: Shuah Khan