| Age | Commit message (Collapse) | Author |
|---|
|
Enhance the ut command to accept a comma-separated list of test suites
to run. Report the summary information for these at the end.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Some suites need things to be set up before they can run. Add a way to
declare an init function using the UNIT_TEST_INIT() macro. The init
function is just like any other test, but is always placed first so that
it runs before all the other test functions in the suite.
Add an uninit function as well, to clean up after the test.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
When tests are all in the same suite it is annoying to have to read all
the common text after each name. Skip this to help the user.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Show the average duration of a test, so we can keep track of how it is
trending. Report the suite with the longest average test to encourage
people to improve it.
Add a function to update the stats based on the results from a single
suite and another to show the summary information.
Make this optional, since sandbox's SPL tests do not have a timer driver
and people may want to print results without times.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Show the time taken by each test suite with 'ut all' and the total time
for all suites.
Take care to remove any sandbox time-offset from the values.
Fix the comment-format on timer_test_add_offset() while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Drop the unwanted asterisk in the comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
With 'ut all' multiple test suites are run. Add a way to collect totals
and show them at the end.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Add a function to show the stats, so we can decide when to print it.
This slightly adjusts the output, so that any 'test not found' message
appears on its own line after all other output.
The 'failures' message now appears in lower case so update pytest
accordingly.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
This is useful information and is not always the same as the 'count' arg
to ut_run_list() so add it as a separate stat.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Use a struct to hold the stats, since we also want to have the same
stats for all runs as we have for each suite.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Add a Python test which runs 'ut all' and then checks that the expected
suites are present and all tests in each suite are run.
This can help to check that nothing is missing.
Update 'ut info' to ignore the 'all' suite when counting the number of
suites, since that is really just a combination of all the other suites.
Adjust the message for skipped tests so that appears even if no
particular test was selected. This helps the new 'test_suite' test see
what is going on.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Pass this into the function so that callers can inspect the state
afterwards.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Move these operations into separate functions so that it is clearer what
is needed. These functions can also be called from somewhere other than
ut_run_list().
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Rename this function and test_set_state() so use the same ut_ prefix as
other functions in ut.h
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Simon Glass <sjg@chromium.org> says:
Some tests do not use the unit-test framework. Others are in a suite of
their own, for no obvious reason.
This series tidies this up.
Link: https://lore.kernel.org/r/20241102193715.432529-1-sjg@chromium.org
|
|
When mentioning a test name, add single quotes to make it easier to see.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
|
|
This should show the test name, not the selected name, since the user
may be running all tests, in which case 'select_name' is NULL
Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
|
|
Tests which create a new bloblist overwrite the existing one in sandbox.
Provide a flag for tests to declare this behaviour. Save and restore the
bloblist pointer so that other tests remain unaffected.
Note that when sandbox is running normally, the bloblist has been
relocated to high in memory. The existing bloblist tests create a new
bloblist low in memory, so they do not conflict.
Correct a build error on coreboot by using accessors for gd->bloblist:
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
This function assumes that all tests in a suite are being run. This
means that it can sometimes call dm_test_restore() when it should not.
The impact of this is that it is not possible, for example, to run
'ut bootstd bootflow_cros' and then check the state of bootstd
afterwards, since all devices are removed and recreated.
Update the function to take account of any selected test, to avoid this
problem.
Add a comment for test_insert while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
A comment in test-main.c was not updated with the recent rename. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
|
|
When a test returns -EAGAIN this should not be considered a failure.
Fix what seems to be a problem case, where the pytests see a failure
when a test has merely been skipped.
We cannot squash the -EAGAIN error in ut_run_test() since the failure
count is incremented by its caller, ut_run_test_live_flat()
The specific example here is on snow, where a test is compiled into the
image but cannot run, so returns -EAGAIN to skip:
test/py/tests/test_ut.py sssnow # ut bdinfo bdinfo_test_eth
Test: bdinfo_test_eth: bdinfo.c
Skipping: Console recording disabled
test/test-main.c:486, ut_run_test_live_flat(): 0 == ut_run_test(uts,
test, test->name): Expected 0x0 (0), got 0xfffffff5 (-11)
Test bdinfo_test_eth failed 1 times
Skipped: 1, Failures: 1
snow # F+u-boot-test-reset snow snow
The fix is simply to respect the return code from ut_run_test(), so do
that.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Complete this rename for all directories outside arch/ board/ drivers/
and include/
Use the new symbol to refer to any 'SPL' build, including TPL and VPL
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Clear any USB-keyboard devices before running a unit test, to avoid
using a stale udevice pointer in stdio. Add a long comment to explain
this situation and why this solution seems best, at least for now.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The _REC suffix doesn't add much. Really what we want to know is whether
the test uses the console, so rename this flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
|
|
The UT_TESTF_ macros read as 'unit test test flags' which is not right.
Rename to UTF ('unit test flags').
This has the benefit of being shorter, which helps keep UNIT_TEST()
declarations on a single line.
Give the enum a name and reference it from the UNIT_TEST() macros while
we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
As part of bringing the master branch back in to next, we need to allow
for all of these changes to exist here.
Reported-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Tom Rini <trini@konsulko.com>
|
|
When bringing in the series 'arm: dts: am62-beagleplay: Fix Beagleplay
Ethernet"' I failed to notice that b4 noticed it was based on next and
so took that as the base commit and merged that part of next to master.
This reverts commit c8ffd1356d42223cbb8c86280a083cc3c93e6426, reversing
changes made to 2ee6f3a5f7550de3599faef9704e166e5dcace35.
Reported-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Tom Rini <trini@konsulko.com>
|
|
Remove <common.h> from all "test/" files and when needed add
missing include files directly.
Signed-off-by: Tom Rini <trini@konsulko.com>
|
|
Sandbox unit tests in U-Boot proper load a test device tree to have some
devices to work with. In order to do the same in SPL, we must enable
SPL_OF_REAL. However, we already have SPL_OF_PLATDATA enabled. When
generating platdata from a devicetree, it is expected that we will not need
devicetree access functions (even though SPL_OF_CONTROL is enabled). This
expectation does not hold for sandbox, so allow user control of
SPL_OF_REAL.
There are several places in the tree where conditions involving OF_PLATDATA
or OF_REAL no longer function correctly when both of these options can be
selected at the same time. Adjust these conditions accordingly.
Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
SPL doesn't have OF_LIVE enabled, so we can only run tests with a flat
tree. Don't skip them even if they don't use the devicetree.
Fixes: 6ec5178c0ef ("test: Skip flat-tree tests if devicetree is not used")
Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
Many tests don't actually use the devicetree at all so there is no point
in running the tests both with livetree and flat tree. Check for this and
skip the flat tree test in that case.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
|
|
dm_test_restore() is called after dm unit test is run.
But this function does not scan any nodes under /firmware since
it calls dm_scan_fdt().
This causes an issue. For instance, scmi_sandbox_agent device
will disappear after running 'ut dm scmi_sandbox_agent'.
So call dm_extended_scan() instead. This change will be coherent
with what dm_scan() and test_pre_run() does.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
This is pretty messy at present since it relies on a SPL_SANDBOX option
that does not exist. Use the normal options instead, so that it will work
with split config.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Most tests don't want these and they can create a lot of noise. Add a way
to disable them. Use that in tests, with a flag provided to enable them
for tests that need this feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Most tests don't want these and can create a lot of noise. Add a way to
disable them. Use that in tests, with a flag provided to enable them for
tests that need this feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
We must call dm_scan_other() after devices from the device tree have been
created, since that function behaves differently if there is no bootstd
device.
Adjust the logic to achieve this.
Also fix the bootflow_system() test which was relying on this broken
behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
This code is present twice. Fix it so that it is only executed once.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
When running unit tests, some may have side effects which cause a
subsequent test to break. This can sometimes be seen when using 'ut dm'
or similar.
Add a new argument which allows a particular (failing) test to be run
immediately after a certain number of tests have run. This allows the
test causing the failure to be determined.
Update the documentation also.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The select_name parameter is not used anymore. Drop it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Some tests access data in block devices and so cause the cache to fill
up. This results in memory being allocated.
Some tests check the malloc usage at the beginning and then again at the
end, to ensure there is no memory leak caused by the test. The block cache
makes this difficult, since the any test may cause entries to be allocated
or even freed, if the cache becomes full.
It is simpler to clear the block cache after each test. This ensures that
it will not introduce noise in tests which check malloc usage.
Add the logic to clear the cache, using the existing blkcache_invalidate()
function. Drop the duplicate code at the same time.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Currently, we must call cyclic_init() at some point before
cyclic_register() becomes possible. That turns out to be somewhat
awkward, especially with SPL, and has resulted in a watchdog callback
not being registered, thus causing the board to prematurely reset.
We already rely on gd->cyclic reliably being set to NULL by the asm
code that clears all of gd. Now that the cyclic list is a hlist, and
thus an empty list is represented by a NULL head pointer, and struct
cyclic_drv has no other members, we can just as well drop a level of
indirection and put the hlist_head directly in struct
global_data. This doesn't increase the size of struct global_data,
gets rid of an early malloc(), and generates slightly smaller code.
But primarily, this avoids having to call cyclic_init() early; the cyclic
infrastructure is simply ready to register callbacks as soon as we
enter C code.
We can still end up with schedule() being called from asm very early,
so we still need to check that gd itself has been properly initialized
[*], but once it has, gd->cyclic_list is perfectly fine to access, and
will just be an empty list.
As for cyclic_uninit(), it was never really the opposite of
cyclic_init() since it didn't free the struct cyclic_drv nor set
gd->cyclic to NULL. Rename it to cyclic_unregister_all() and use that
in test/, and also insert a call at the end of the board_init_f
sequence so that gd->cyclic_list is a fresh empty list before we enter
board_init_r().
A small piece of ugliness is that I had to add a cast in
cyclic_get_list() to silence a "discards 'volatile' qualifier"
warning, but that is completely equivalent to the existing handling of
the uclass_root_s list_head member.
[*] I'm not really sure where we guarantee that the register used for
gd contains 0 until it gets explicitly initialized, but that must be
the case, otherwise testing gd for being NULL would not make much sense.
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Reviewed-by: Stefan Roese <sr@denx.de>
Tested-by: Stefan Roese <sr@denx.de>
Tested-by: Tim Harvey <tharvey@gateworks.com> # imx8mm-venice-*
|
|
At present we normally write tests either in Python or in C. But most
Python tests end up doing a lot of checks which would be better done in C.
Checks done in C are orders of magnitude faster and it is possible to get
full access to U-Boot's internal workings, rather than just relying on
the command line.
The model is to have a Python test set up some things and then use C code
(in a unit test) to check that they were done correctly. But we don't want
those checks to happen as part of normal test running, since each C unit
tests is dependent on the associate Python tests, so cannot run without
it.
To acheive this, add a new UT_TESTF_MANUAL flag to use with the C 'check'
tests, so that they can be skipped by default when the 'ut' command is
used. Require that tests have a name ending with '_norun', so that pytest
knows to skip them.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present it is possible for a test to skip itself by returning -EAGAIN
but this is not recorded. An existing example is in test_pre_run() with
the "Console recording disabled" check.
Keep a track of skipped tests and report the total at the end.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
|
|
We already have a function for probing all devices of a specific class,
use it.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
At present there is only one device tree used by the ofnode functions,
except for some esoteric use of live tree. In preparation for supporting
more than one, add a way to reset the list of device trees.
For now this does nothing.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Add a test flag which indicates that the 'other' FDT should be set up
ready for use. Handle this by copying in the FDT, unflattening it for
livetree tests. Free the structures when the tests have run.
We cannot use the other FDT unless we are using live tree or
OFNODE_MULTI_TREE is enabled, since only one tree is supported by the
ofnode interface in that case. Add this condition into
ut_run_test_live_flat() and update the comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
If the device tree changes during a test and we cannot restore it, mark
it as such so that future tests which need the live tree are skipped.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
When the flat device tree changes it can mess up the live tree since that
uses the flat tree for its strings. This affects only a few sandbox tests
which modify the device tree, but the number will grow as ofnode support
for writing improves.
While the control FDT is not intended to change while U-Boot is running,
some tests do so. For example, the ofnode interface only supports
modifying properties in the control FDT, so tests must use that.
To solve this problem, keep a copy of the FDT and restore it as needed
when the test is finished. The copy only happens on sandbox (except SPL
builds), to reduce memory usage and because these tests are not useful on
other boards. For other boards, a checksum is taken to ensure that nothing
changes.
It would be possible to always checksum the FDT on sandbox and only
restore it if needed, but this is slightly slower than restoring it every
time, at least with crc8.
Move the code which checks for success to the very end, for clarity.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
It is helpful to test that out-of-memory checks work correctly in code
that calls malloc().
Add a simple way to force failure after a given number of malloc() calls.
Fix a header guard to avoid a build error on sandbox_vpl.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Sean Anderson <seanga2@gmail.com>
|
|
At present the live tree tests are not run on sandbox. This bug is in two
parts, with a duplicate flag value and incorrect logic in the test runner.
This was not noticed because the bug was fixed in a later commit and does
not cause test failures.
Fix this.
Fixes: 7b1dfc9fd7e ("dm: core: Prepare for updating the device tree with ofnode")
Signed-off-by: Simon Glass <sjg@chromium.org>
|
