linux-toradex.git/scripts/spdxcheck.py, branch v6.14-rc2

scripts/spdxcheck: Handle license identifiers in Jinja comments

2025-01-08T14:38:33+00:00

Commit 4b132aacb076 ("tools: Add xdrgen") adds a tool, which uses Jinja
template files, i.e., files with the j2 file extension, for its lightweight
code generation.

These template files for this tool have proper headers with the SPDX
License information, which are included as Jinja comments by enclosing the
text with '{#' and '#}'. Sofar, the spdxcheck script does not support to
properly parse this license information in Jinja comments and it reports
back with 'Invalid token: #}'.

Parse Jinja comments properly by stripping the known Jinja comment suffix.

Signed-off-by: Lukas Bulwahn 
Link: https://lore.kernel.org/r/20250108125207.57486-1-lukas.bulwahn@redhat.com
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Parse j2 comments correctly

2025-01-08T14:38:28+00:00

j2 files use '#}' as comment closure, which trips up the SPDX
parser:

 tools/.../definition.j2: 1:36 Invalid token: #}

Handle those comments correctly by removing the closure.

Signed-off-by: Thomas Gleixner 
Link: https://lore.kernel.org/r/878qt2xr46.ffs@tglx
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Add count of missing files to stats output

2024-05-04T17:00:54+00:00

Add a count of files missing an SPDX header to the stats
output.  This is useful detailed information for working
on SPDX header additions.

Signed-off-by: Tim Bird 
Link: https://lore.kernel.org/r/SA3PR13MB6372DB9F9F2C09F8A1E1B99BFD1A2@SA3PR13MB6372.namprd13.prod.outlook.com
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Put excluded files and directories into a separate file

2022-05-18T13:34:33+00:00

The files and directories which are excluded from scanning are currently
hard coded in the script. That's not maintainable and not accessible for
external tools.

Move the files and directories which should be excluded into a file.  The
default file is scripts/spdxexclude. This can be overridden with the
'-e $FILE' command line option.

The file format and syntax is similar to the .gitignore file.

Signed-off-by: Thomas Gleixner 
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Add option to display files without SPDX

2022-05-18T13:32:38+00:00

Makes life easier when chasing the missing ones. Is activated with '-f'
on the command line.

# scripts/spdxcheck.py -f kernel/
Files without SPDX:
    ./kernel/cpu.c
    ./kernel/kmod.c
    ./kernel/relay.c
    ./kernel/bpf/offload.c
    ./kernel/bpf/preload/.gitignore
    ./kernel/bpf/preload/iterators/README
    ./kernel/bpf/ringbuf.c
    ./kernel/cgroup/cgroup.c
    ./kernel/cgroup/cpuset.c
    ./kernel/cgroup/legacy_freezer.c
    ./kernel/debug/debug_core.h
    ./kernel/debug/kdb/Makefile
    ./kernel/debug/kdb/kdb_bp.c
    ./kernel/debug/kdb/kdb_bt.c
    ./kernel/debug/kdb/kdb_cmds
    ./kernel/debug/kdb/kdb_debugger.c
    ./kernel/debug/kdb/kdb_io.c
    ./kernel/debug/kdb/kdb_keyboard.c
    ./kernel/debug/kdb/kdb_main.c
    ./kernel/debug/kdb/kdb_private.h
    ./kernel/debug/kdb/kdb_support.c
    ./kernel/locking/lockdep_states.h
    ./kernel/locking/mutex-debug.c
    ./kernel/locking/spinlock_debug.c
    ./kernel/sched/pelt.h

With the optional -D parameter the directory depth can be limited:

# scripts/spdxcheck.py -f -D 0 kernel/
Files without SPDX:
    ./kernel/cpu.c
    ./kernel/kmod.c
    ./kernel/relay.c

Signed-off-by: Thomas Gleixner 
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Add [sub]directory statistics

2022-05-18T13:31:45+00:00

Add functionality to display [sub]directory statistics. This is enabled by
adding '-d' to the command line. The optional -D parameter allows to limit
the directory depth. If supplied the subdirectories are accumulated

# scripts/spdxcheck.py -d kernel/
Incomplete directories: SPDX in Files
    ./kernel                         :   111 of   114   97%
    ./kernel/bpf                     :    43 of    45   95%
    ./kernel/bpf/preload             :     4 of     5   80%
    ./kernel/bpf/preload/iterators   :     4 of     5   80%
    ./kernel/cgroup                  :    10 of    13   76%
    ./kernel/configs                 :     0 of     9    0%
    ./kernel/debug                   :     3 of     4   75%
    ./kernel/debug/kdb               :     1 of    11    9%
    ./kernel/locking                 :    29 of    32   90%
    ./kernel/sched                   :    38 of    39   97%

The result can be accumulated by restricting the depth via the new command
line option '-d $DEPTH':

# scripts/spdxcheck.py -d -D1
Incomplete directories: SPDX in Files
    ./                               :     6 of    13   46%
    ./Documentation                  :  4096 of  8451   48%
    ./arch                           : 13476 of 16402   82%
    ./block                          :   100 of   101   99%
    ./certs                          :    11 of    14   78%
    ./crypto                         :   145 of   176   82%
    ./drivers                        : 24682 of 30745   80%
    ./fs                             :  1876 of  2110   88%
    ./include                        :  5175 of  5757   89%
    ./ipc                            :    12 of    13   92%
    ./kernel                         :   493 of   527   93%
    ./lib                            :   393 of   524   75%
    ./mm                             :   151 of   159   94%
    ./net                            :  1713 of  1900   90%
    ./samples                        :   211 of   273   77%
    ./scripts                        :   341 of   435   78%
    ./security                       :   241 of   250   96%
    ./sound                          :  2438 of  2503   97%
    ./tools                          :  3810 of  5462   69%
    ./usr                            :     9 of    10   90%

Signed-off-by: Thomas Gleixner 
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Add directory statistics

2022-05-18T13:31:28+00:00

For better insights.

Directories accounted:     4646
Directories complete:      2565  55%

Signed-off-by: Thomas Gleixner 
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck: Add percentage to statistics

2022-05-18T13:30:49+00:00

Files checked:            75856
Lines checked:           294516
Files with SPDX:          59410  78%
Files with errors:            0

Signed-off-by: Thomas Gleixner 
Signed-off-by: Greg Kroah-Hartman

spdxcheck.py: Fix a type error

2022-02-04T15:43:01+00:00

remove unused variable "col", otherwise there will be a type error
as below:

typeerror: not all arguments converted during string formatting

Signed-off-by: Ding Xiang 
Link: https://lore.kernel.org/r/20220114024058.74536-1-dingxiang@cmss.chinamobile.com
Signed-off-by: Greg Kroah-Hartman

scripts/spdxcheck.py: Strictly read license files in utf-8

2021-07-12T15:56:50+00:00

Commit bc41a7f36469 ("LICENSES: Add the CC-BY-4.0 license")
unfortunately introduced LICENSES/dual/CC-BY-4.0 in UTF-8 Unicode text
While python will barf at it with:

FAIL: 'ascii' codec can't decode byte 0xe2 in position 2109: ordinal not in range(128)
Traceback (most recent call last):
  File "scripts/spdxcheck.py", line 244, in 
    spdx = read_spdxdata(repo)
  File "scripts/spdxcheck.py", line 47, in read_spdxdata
    for l in open(el.path).readlines():
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2109: ordinal not in range(128)

While it is indeed debatable if 'Licensor.' used in the license file
needs unicode quotes, instead, force spdxcheck to read utf-8.

Reported-by: Rahul T R 
Signed-off-by: Nishanth Menon 
Reviewed-by: Thomas Gleixner 
Link: https://lore.kernel.org/r/20210707204840.30891-1-nm@ti.com
Signed-off-by: Jonathan Corbet