summaryrefslogtreecommitdiff
path: root/tools/buildman/builderthread.py
AgeCommit message (Collapse)Author
2025-07-08Merge patch series "kbuild: Update Makefile.extrawarn to 5.1"Tom Rini
This series from Ilias Apalodimas <ilias.apalodimas@linaro.org> largely finishes the re-sync with the Linux Kernel v5.1 kbuild system. Link: https://lore.kernel.org/r/20250627185723.342553-1-ilias.apalodimas@linaro.org
2025-07-08kbuild: Bump the build system to 5.1Ilias Apalodimas
Our last sync with the kernel was 5.1. Even that was a partial one as some patches from 4.x kernels were already missing making the transition to a modern kbuild infeasible. We are so out of sync now, that tracking the patches and backporting them one by one makes little sense and it's going to take ages. This is an attempt to sync up Makefile[.lib/.kbuild]. Unfortunately due to sheer amount of patches this is not easy to review, but that's what we decided during a community call. One of the biggest changes is get rid of partial linking entirely and build .a archives isntead of .o. We diaviate from the kernel on that. Instead of calling a custom script to create the archive symbol table, we call ar with rcTP (isntead of rcSTP) since we want a resulting archive that's sauble with the linker. The only affected platforms are PPC ones. Unfortunately I don't have any of them around to test, but the objdump of the resulting files -- arch/powerpc/lib/built-in.[oa] looks identical. Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
2025-07-01buildman: Support an in-tree build in the current dirSimon Glass
Allow -w to be used with -i to do a build without a separate output directory. Signed-off-by: Simon Glass <sjg@chromium.org>
2025-07-01buildman: Correct behaviour of --in-treeSimon Glass
This option doesn't work as expected since it sets the cwd to the work directory, which does not necessarily hold the source code. It should be left unset, so that the current directory is the source directory. Signed-off-by: Simon Glass <sjg@chromium.org>
2025-05-27buildman: Add a way to build a particular targetSimon Glass
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files. Add a --target option to help. Also add a comment indicating which letters are free for new options. Signed-off-by: Simon Glass <sjg@chromium.org>
2025-03-04Merge patch series "tools: Minor clean-ups for the command library"Tom Rini
Simon Glass <sjg@chromium.org> says: This series adds comments and fixes pylint warnings in the command library. It also introduces a new, simpler way of running a single command. Link: https://lore.kernel.org/r/20250203162704.627469-1-sjg@chromium.org
2025-03-04u_boot_pylib: Add a function to run a single commandSimon Glass
Add a helper to avoid needing to use a list within a list for this simple case. Update existing users of runpipe() to use this where possible. Signed-off-by: Simon Glass <sjg@chromium.org>
2025-02-17u_boot_pylib: Move gitutil into the librarySimon Glass
Move this file into U-Boot's Python library, so that it is no-longer part of patman. This makes a start on: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/35 Signed-off-by: Simon Glass <sjg@chromium.org>
2025-01-25buildman: Record an error if a toolchain is missingSimon Glass
Buildman has always treated the lack of a toolchain as an infrastructure problem rather than a build failure. However the logic for this is not correct, since it does not write a 'done' file in this case. As a result, one of two things can happen. 1. If a previous build ran in the same (output) directory, the outcome of *that* build is recorded as the outcome of this one 2. Otherwise, no outcome is recorded Obviously this inconsistency is not ideal. While (2) is rare, it can be very confusing as the build sort-of fails but does not produce any summary output with 'buildman -s' Overall it seems better to attribute a toolchain issue to the boards that it affects. This results in clear failures which can be examined, no matter what happened in the .bm-work directory previously. So write a 'done' file for each build when a toolchain is missing. The end result of this patch is to make missing toolchains much more obvious. It should be things a bit easier for novice users. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-09-26buildman: Allow skipping the dtc buildSimon Glass
For most boards, the device-tree compiler is built in-tree, ignoring the system version. Add a special option to skip this build. This can be useful when the system dtc is up-to-date, as it speeds up the build. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-09-26buildman: Retry the build for current sourceSimon Glass
Buildman retries a failed build when processing a branch, but does not do this when building current source. It is useful to do this retry in both cases, so add the logic for it. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-08Revert "buildman: Always use the full path in CROSS_COMPILE"Tom Rini
There are operations in buildman that result in running the cross-tools (such as performing size checks) and now that we have not modified PATH to know where our tools are, these operations fail. This reverts commit 6c0a3cf75f72370deec3ee516a9dd377397af207. Signed-off-by: Tom Rini <trini@konsulko.com>
2024-07-03buildman: Always use the full path in CROSS_COMPILESimon Glass
The feature to set the toolchain path does not seem to be needed. It causes problems with venv (see [1]). Let's remove it. Add some tests while we are here. It does not look like any docs changes are needed for this. [1] https://patchwork.ozlabs.org/project/uboot/patch/20240621131423.2363294-6-sjg@chromium.org/ Signed-off-by: Simon Glass <sjg@chromium.org> Suggested-by: Tom Rini <trini@konsulko.com> Reviewed-by: Tom Rini <trini@konsulko.com> Reviewed-by: Andrejs Cainikovs <andrejs.cainikovs@toradex.com>
2024-07-03buildman: Add a flag to force mrproper on failureSimon Glass
When a file is removed by a commit (e.g. include/common.h yay!) it can cause incremental build failures since one of the dependency files from a previous build may mention the file. Add an option to run 'make mrproper' automatically when a build fails. This can be used to automatically resolve the problem, without always adding the large overhead of 'make mrproper' to every build. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03buildman: Avoid rebuilding when --mrproper is usedSimon Glass
When this flag is enabled, 'make mrproper' is always used when reconfiguring, so there is no point in doing it again. Update this. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03buildman: Make mrproper an argument to run_commit()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03buildman: Make mrproper an argument to _config_and_build()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <sjg@chromium.org>
2024-07-03buildman: Make mrproper an argument to _reconfigure()Simon Glass
Pass this in so the caller can change it independently of the member variable. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-11-07buildman: Use oldconfig when adjusting the configSimon Glass
We cannot be sure that the new config is consistent, particularly when changing a major item like CONFIG_CMDLINE. Use 'make oldconfig' to check that and avoid any such problems. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-09-23buildman: Keep all common output filesSimon Glass
Make a list of common output extensions and use it to ensure that the -k option preserves all of these. Signed-off-by: Simon Glass <sjg@chromium.org> Suggested-by: Tom Rini <trini@konsulko.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2023-07-24buildman: Move copy_files() out ot BuilderThread classSimon Glass
This does not need to be in the class. Move it out to avoid a pylint warning. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Tidy up some comments in builderthreadSimon Glass
Make sure all functions have full argument documentation. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Tidy up reporting of a toolchain errorSimon Glass
Provide the text of the exception when something goes wrong. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Avoid passing result into _read_done_file()Simon Glass
Move the creating of the result object into the function which sets it up, to simplify the code. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Create a function to handle config and buildSimon Glass
Move this code into a _config_and_build() function, so reduce the size of run_commit(). Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move checkout code to a separate functionSimon Glass
Put this in its own function to reduce the size of the run_commit() function Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move code to decide output dirsSimon Glass
Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move code to remove old outputsSimon Glass
Put this in its own function to reduce the size of the run_commit() function. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move reading of the done file into a functionSimon Glass
Move this logic into its own function to reduce the size of the run_commt() function. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move bulid code into its own functionSimon Glass
Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move reconfigure code into its own functionSimon Glass
Split this into its own function so reduce the size of run_commit(). Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Convert config_out to string IOSimon Glass
This is probably a little more efficient and it allows passing the object to another function to write data. Convert config_out to use a string I/O device. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move more things into _build_args()Simon Glass
Move more of the argument-building code into this function. Fix a missing assignment for out_rel_dir too. Rename the function since it now builds all the arguments. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Move setting of toolchain arguments to _build_args()Simon Glass
Move a few more pieces to this new function. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Start a function to set up the make argumentsSimon Glass
Move some of this code into a new funciion, to help reduce the size of the run_commits() function. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Drop unnecessary assignment of config_outSimon Glass
This is already set up earlier in the function, so drop the extra assignment. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Correct invalid use of out_dir variableSimon Glass
This variable has a different meaning in the outer scope. Use a different name to avoid confusion, or bugs. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Export _get_output_dir() to avoid warningsSimon Glass
Make this a public memory since it is used outside the class. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Correct most pylint warnings in builderthreadSimon Glass
Fix the easy warnings in this file. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Convert camel case in builderthread.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-07-24buildman: Convert camel case in builder.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-05-03buildman: Pass -Werror to the host compiler tooSimon Glass
The host compiler is not failing on warnings at present, when the -E flag is used in buildman. Add the required flag to fix this. Signed-off-by: Simon Glass <sjg@chromium.org> Tested-by: Tom Rini <trini@konsulko.com>
2023-03-08patman: Move library functions into a library directorySimon Glass
The patman directory has a number of modules which are used by other tools in U-Boot. This makes it hard to package the tools using pypi since the common files must be copied along with the tool that uses them. To address this, move these files into a new u_boot_pylib library. This can be packaged separately and listed as a dependency of each tool. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-03-08buildman: Add a flag for reproducible buildsSimon Glass
This is quite a useful thing to use when building since it avoids small size changes between commits. Add a -r flag for it. Also undefine CONFIG_LOCALVERSION_AUTO since this appends the git hash to the version string, causing every build to be slightly different. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-03-08buildman: Support disabling LTOSimon Glass
This cuts down build performance considerably and is not always needed, when checking for build errors, etc. Add a flag to disable it. Signed-off-by: Simon Glass <sjg@chromium.org>
2023-03-08buildman: Write out the build command usedSimon Glass
It is sometimes useful to see the exact 'make' command used by buildman for a commit. Add an output file for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2022-11-22buildman: Add --allow-missing flag to allow missing blobsTom Rini
Add a new flag to buildman so that we will in turn pass BINMAN_ALLOW_MISSING=1 to 'make'. Make use of this flag in CI. Allow the settings file to control this. Cc: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Cc: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Simon Glass <sjg@chromium.org>
2022-11-22buildman: Detect binman reporting missing blobsSimon Glass
Buildman should consider a build as a success (with warnings) if missing blobs have been dealt with by binman, even though buildman itself returns and error code overall. This is how other warnings are dealt with. We cannot easily access the 103 exit code, so detect the problem in the output. With this change, missing blobs result in an exit code of 101, although they still indicate failure. Signed-off-by: Simon Glass <sjg@chromium.org>
2022-08-05buildman: Avoid using board as a variableSimon Glass
We have a module called 'board'. Sometimes buildman uses 'brd' as an instance variable but sometimes it uses 'board', which is confusing and can mess with the module handling. Update the code to use 'brd' consistently, making it easier for tools to determine when the module is being referenced. Signed-off-by: Simon Glass <sjg@chromium.org>
2022-02-09patman: Convert camel case in gitutil.pySimon Glass
Convert this file to snake case and update all files which use it. Signed-off-by: Simon Glass <sjg@chromium.org>