| Age | Commit message (Collapse) | Author |
|---|
|
This function is sometimes useful outside tests. Also it can affect how
terminal output is done, e.g. whether ANSI characters should be emitted
or not.
Move it out of the test_util package and into terminal.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:
- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
virtualenv.
Signed-off-by: Tom Rini <trini@konsulko.com>
|
|
Recent versions of this module call time.perf_counter() so add a patch
for this also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
|
|
It is convenient to use ~ to represent the home directory in the
settings file. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
For most boards, the device-tree compiler is built in-tree, ignoring the
system version. Add a special option to skip this build. This can be
useful when the system dtc is up-to-date, as it speeds up the build.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
When building for sandbox, self.cross is empty.
In MakeEnvironment(), CROSS_COMPILE is defined to be self.cross (with
or without a full path), optionally prefixed by the toolchain wrapper
defined in ~/.buildman. This is fine when self.cross is not empty, but
it doesn't make sense when it is:
- Either there is no wrapper and we end up with an empty CROSS_COMPILE
which is the same as not defining it (the host compiler will be used),
- Or there is a wrapper and CROSS_COMPILE will contain only the wrapper
which obviously is not a valid compiler, hence an error.
Test case:
$ sudo apt install ccache
$ grep -q toolchain-wrapper ~/.buildman || \
printf "[toolchain-wrapper]\nwrapper = ccache\n" >>~/.buildman
$ make mrproper
$ ./tools/buildman/buildman sandbox_noinst
$ ./tools/buildman/buildman sandbox_noinst
Building current source for 1 boards (1 thread, 24 jobs per thread)
sandbox: + sandbox_noinst
+arch/sandbox/lib/reloc_sandbox_efi.c:10:15: error: operator '==' has no left operand
+ 10 | #if HOST_ARCH == HOST_ARCH_X86_64
+ | ^~
[...]
The GetEnvArgs function is modified too, since the VAR_CROSS_COMPILE
case has the same issue.
In tools/buildman/test.py, testGetEnvArgs is extended and
testMakeEnvironment is added. They check the 'arm' and 'sandbox'
toolchains, with and without a wrapper.
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
The Python virtualenv tool sets up a few things in the environment,
putting its path first in the PATH environment variable and setting up
a sys.prefix different from the sys.base_prefix value.
At present buildman puts the toolchain path first in PATH so that it can
be found easily during the build. For sandbox this causes problems since
/usr/bin/gcc (for example) results in '/usr/bin' being prepended to the
PATH variable. As a result, the venv is partially disabled.
The result is that sandbox builds within a venv ignore the venv, e.g.
when looking for packages.
Correct this by detecting the venv and adding the toolchain path after
the venv path.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Newer versions of filelock use time.monotonic() instead of time.time().
Update the test the handle this.
It would be better if filelock had support for writing unit tests which
use locking.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.
This reverts commit 6c0a3cf75f72370deec3ee516a9dd377397af207.
Signed-off-by: Tom Rini <trini@konsulko.com>
|
|
The feature to set the toolchain path does not seem to be needed. It
causes problems with venv (see [1]). Let's remove it.
Add some tests while we are here.
It does not look like any docs changes are needed for this.
[1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
|
|
Buildman uses all available CPUs by default, so running more than one or
two concurrent processes is not normally useful.
However in some CI cases we want to be able to run several jobs at once
to save time. For example, in a lab situation we may want to run a test
on 20 boards at a time, since only the build step actually takes much
CPU.
Add an option which allows such a limit. When buildman starts up, it
waits until the number of running processes goes below the limit, then
claims a spot in the list. The list is maintained with a temporary file.
Note that the temp file is user-specific, since it is hard to create a
locked temporary file which can be accessed by any user. In most cases,
only one user is running jobs on a machine, so this should not matter.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The methods `unittest.assertEquals()` and
`unittest.assertRegexpMatches()` are marked deprecated[1].
In Python 3.12 these aliases have been removed, so do a sed to replace
them with their new names.
[1] https://docs.python.org/3.11/library/unittest.html#deprecated-aliases
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
CC: Simon Glass <sjg@chromium.org>
CC: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
|
|
%s/arguemnts/arguemnts/
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
|
|
Make this a public memory since it is used outside the class.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Use a separate file for the Boards class so that its name matches the
module name.
Fix up the function names to match the pylint style and fix some other
warnings.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
We want to create a module called 'boards' so avoid use of this variable
name in this module. Change the global to be capitalised, as required by
Python style.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
We have a module called 'board'. Sometimes buildman uses 'brd' as an
instance variable but sometimes it uses 'board', which is confusing and
can mess with the module handling. Update the code to use 'brd'
consistently, making it easier for tools to determine when the module
is being referenced.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Fix documentation path in deprecated warning message about device
driver.
Signed-off-by: Johannes Krottmayer <krjdev@gmail.com>
Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
This method has the same name as its class which is confusing. It is also
annoying when searching the code.
It builds a string with a colour, so rename it to build().
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert this file to snake case and update all files which use it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present the only straightforward way to write tests that need a
slightly different configuration is to create a new board with its own
configuration. This is cumbersome.
It would be useful if buildman could adjust the configuration of a build
on the fly. In preparation for this, add a utility library which can
modify a .config file according to various parameters passed to it.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present even if only a single thread is in use, buildman still uses
threading.
For some debugging it is helpful to do everything in the main process.
Allow -T0 to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
There are few places where the path of the current modules is calculated
but not used. Drop them.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
This current fails with an error. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: 7664b03ffc5 ("buildman: Remove _of_#_ from results directory paths")
|
|
Currently, the following scenario will rebuild the first commit even
though it is not really necessary - the commit sha or the position in the
patchset did not change:
$ git am <local-patch-0001>
$ tools/buildman/buildman -P -E -W -b master mx6
<do some more development work>
$ git am <local-patch-0002>
$ tools/buildman/buildman -P -E -W -b master mx6 <- will rebuild the first
commit as well, even
though nothing has
changed about it.
This is due to the fact that previous results directories get removed
when the number of commits change. By removing the _of_#_ part of the
directory path, the commits will be rebuilt only if the commit sha or the
position in the patchset changes. Also, update the testcase to reflect this
change.
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
|
|
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.
We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.
The special paths for finding pylibfdt remain.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present buildman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move buildman to use absolute imports. Also adjust moveconfig.py too since
it uses some buildman modules and cannot work without this.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The naming is slightly different on kernel.org now. Update the regex so
that the test still passes.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
These are becoming more common now. They cause boards to show warnings
which can be mistaking for compiler warnings.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Unfortunately the plague of device-tree warnings has not lifted. These
warnings infiltrate almost every build, adding noise and confusion.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
When buildman finishes it leaves the last summary line visible, which
shows the number of successful builds, builds with warnings and builds
with errors.
It is useful also to see how many builds were done in total along with
the time taken. Show these on a separate line before buildman finishes.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present the board names shown with -l are separated by commas. This
makes it hard to double-click to select a particular board. Also it is not
possible to select all boards and paste them as arguments to a subsequent
buildman run, since buildman requires spaces to separate the list on the
command line, not commas.
Change the output format to use spaces instead of commas.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
It is quite hard to see the list of board for each error line since the
colour is the same as the actual error line. Show the board list in
magenta so that it is easier to distinguish them.
There is no point in checking the colour of the overall line, since there
are now multiple colours. So drop those tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present warnings are shown in yellow in the summary (-s) but magenta in
the detail listing (-e). Use yellow in both.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Add a test to cover this flag, which adds the name of each board to each
error/warning line.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
The split/join code is repeated in a lot of places. Add a function to
handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
We want to add a few more tests similar to testOutput(). Split its logic
into a function which runs buildman to get the output and another which
checks the output. This will make it easier to reuse the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Rather than having a few tests handle this themselves, create the
temporary directory in the setUp() method and remove it in tearDown().
This will make it easier to add more tests.
Only testOutput and testGit() actually need it, but it doesn't add to the
test time noticeably to do this for all tests in this file.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Rather than using the absolute array index, use an interator to work
through the expected output lines. This is easier to follow.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Buildman should output the right colours for each error/warning line. Some
of these checks are missing. Add them.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
At present buildman removes any directory it doesn't intend to write
output into. This is overly expansive since if the output directory
happens to be somewhere with existing files, they may be removed. Using
an existing directory for buildman is not a good practice, but since the
result might be catastrophic, it is best to guard against it.
A previous commit[1] fixed this by refusing to write to a subdirectory
of the current directory, assumed to have U-Boot source code. But we can
do better by only removing directories that look like the ones buildman
creates.
Update the code to do this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
[1] 409fc029c40 tools: buildman: Don't use the working dir as build dir
|
|
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE
Signed-off-by: Simon Glass <sjg@chromium.org>
|
|
Convert buildman to Python 3 and make it use that, to meet the 2020
deadline.
Signed-off-by: Simon Glass <sjg@chromium.org>
|
