summaryrefslogtreecommitdiff
path: root/tools/lib/python/kdoc/parse_data_structs.py
diff options
context:
space:
mode:
authorJonathan Corbet <corbet@lwn.net>2026-01-23 11:46:08 -0700
committerJonathan Corbet <corbet@lwn.net>2026-01-23 11:46:08 -0700
commit330367bdc176a8f52cc4c5065ba0312277202dee (patch)
tree4e7965522177b29a5f51c3fb85f285e8b69585ff /tools/lib/python/kdoc/parse_data_structs.py
parentffb569d59c253399efb2345ddfefe7929cd7e2a8 (diff)
parentef6aa110d8888a14dfb2e843794097263c45a06b (diff)
Merge branch 'mauro' into docs-mw
Mauro's work to include documentation from our Python modules. His cover letter follows: This is an extended version of: https://lore.kernel.org/linux-doc/cover.1768488832.git.mchehab+huawei@kernel.org/ It basically adds everything we currently have inside libs/tool/python to "tools" book inside documentation. This version should be independent of the other series yet to be merged, (including the jobserver one). The vast amount of changes here are docstring cleanups and additions. They mainly consists on: - ensuring that every phrase will end with a period, making it uniform along all files; - cleaning ups to better uniform docstrings; - variable descriptions now use "#:" markup, as it allows autodoc to add them inside the documentation; - added some missing docstrings; - some new blank lines at comments to make ReST syntax parser happy; - add a couple of sphinx markups (mainly, code blocks). Most of those are minor changes, affecting only comments. It also has one patch per libarary type, adding them to docs. For kernel-doc, I did the cleanups first, as there is one code block inside tools/lib/python/kdoc/latex_fonts.py that would cause a Sphinx crash without such markups. The series actually starts with 3 fixes: - avoid "*" markups on indexes with deep> 3 to override text - a variable rename to stop abusing doctree name - don't rely on cwd to get Documentation/ location patch 4 adds support to document scripts either at: - tools/ - scripts/ patch 5 contains a CSS to better display autodoc html output. For those who want to play with documentation, documenting a python file is very simple. All it takes is to use: .. automodule:: lib.python.<dir+name> Usually, we add a couple of control members to it to adjust the desired documentation scope (add/remove members, showing class inheritance, showing members that currently don't have docstrings, etc). That's why we're using: .. automodule:: lib.python.kdoc.enrich_formatter :members: :show-inheritance: :undoc-members: (and similar) inside tools/kdoc*.rst. autodoc allows filtering in/out members, file docstrings, etc. It also allows documenting just some members or functions with directives like: ..autofunction: ..automember: Sphinx also has a helper script to generate .rst files with documentation: $ sphinx-apidoc -o foobar tools/lib/python/ which can be helpful to discover what should be documented, although changes are needed to use what it produces.
Diffstat (limited to 'tools/lib/python/kdoc/parse_data_structs.py')
-rwxr-xr-xtools/lib/python/kdoc/parse_data_structs.py62
1 files changed, 39 insertions, 23 deletions
diff --git a/tools/lib/python/kdoc/parse_data_structs.py b/tools/lib/python/kdoc/parse_data_structs.py
index 25361996cd20..9941cd19032e 100755
--- a/tools/lib/python/kdoc/parse_data_structs.py
+++ b/tools/lib/python/kdoc/parse_data_structs.py
@@ -9,12 +9,12 @@ Parse a source file or header, creating ReStructured Text cross references.
It accepts an optional file to change the default symbol reference or to
suppress symbols from the output.
-It is capable of identifying defines, functions, structs, typedefs,
-enums and enum symbols and create cross-references for all of them.
+It is capable of identifying ``define``, function, ``struct``, ``typedef``,
+``enum`` and ``enum`` symbols and create cross-references for all of them.
It is also capable of distinguish #define used for specifying a Linux
ioctl.
-The optional rules file contains a set of rules like:
+The optional rules file contains a set of rules like::
ignore ioctl VIDIOC_ENUM_FMT
replace ioctl VIDIOC_DQBUF vidioc_qbuf
@@ -34,8 +34,8 @@ class ParseDataStructs:
It is meant to allow having a more comprehensive documentation, where
uAPI headers will create cross-reference links to the code.
- It is capable of identifying defines, functions, structs, typedefs,
- enums and enum symbols and create cross-references for all of them.
+ It is capable of identifying ``define``, function, ``struct``, ``typedef``,
+ ``enum`` and ``enum`` symbols and create cross-references for all of them.
It is also capable of distinguish #define used for specifying a Linux
ioctl.
@@ -43,13 +43,13 @@ class ParseDataStructs:
allows parsing an exception file. Such file contains a set of rules
using the syntax below:
- 1. Ignore rules:
+ 1. Ignore rules::
ignore <type> <symbol>`
Removes the symbol from reference generation.
- 2. Replace rules:
+ 2. Replace rules::
replace <type> <old_symbol> <new_reference>
@@ -58,22 +58,22 @@ class ParseDataStructs:
- A simple symbol name;
- A full Sphinx reference.
- 3. Namespace rules
+ 3. Namespace rules::
namespace <namespace>
Sets C namespace to be used during cross-reference generation. Can
be overridden by replace rules.
- On ignore and replace rules, <type> can be:
- - ioctl: for defines that end with _IO*, e.g. ioctl definitions
- - define: for other defines
- - symbol: for symbols defined within enums;
- - typedef: for typedefs;
- - enum: for the name of a non-anonymous enum;
- - struct: for structs.
+ On ignore and replace rules, ``<type>`` can be:
+ - ``ioctl``: for defines that end with ``_IO*``, e.g. ioctl definitions
+ - ``define``: for other defines
+ - ``symbol``: for symbols defined within enums;
+ - ``typedef``: for typedefs;
+ - ``enum``: for the name of a non-anonymous enum;
+ - ``struct``: for structs.
- Examples:
+ Examples::
ignore define __LINUX_MEDIA_H
ignore ioctl VIDIOC_ENUM_FMT
@@ -83,13 +83,15 @@ class ParseDataStructs:
namespace MC
"""
- # Parser regexes with multiple ways to capture enums and structs
+ #: Parser regex with multiple ways to capture enums.
RE_ENUMS = [
re.compile(r"^\s*enum\s+([\w_]+)\s*\{"),
re.compile(r"^\s*enum\s+([\w_]+)\s*$"),
re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*\{"),
re.compile(r"^\s*typedef\s*enum\s+([\w_]+)\s*$"),
]
+
+ #: Parser regex with multiple ways to capture structs.
RE_STRUCTS = [
re.compile(r"^\s*struct\s+([_\w][\w\d_]+)\s*\{"),
re.compile(r"^\s*struct\s+([_\w][\w\d_]+)$"),
@@ -97,11 +99,13 @@ class ParseDataStructs:
re.compile(r"^\s*typedef\s*struct\s+([_\w][\w\d_]+)$"),
]
- # FIXME: the original code was written a long time before Sphinx C
+ # NOTE: the original code was written a long time before Sphinx C
# domain to have multiple namespaces. To avoid to much turn at the
# existing hyperlinks, the code kept using "c:type" instead of the
# right types. To change that, we need to change the types not only
# here, but also at the uAPI media documentation.
+
+ #: Dictionary containing C type identifiers to be transformed.
DEF_SYMBOL_TYPES = {
"ioctl": {
"prefix": "\\ ",
@@ -158,6 +162,10 @@ class ParseDataStructs:
self.symbols[symbol_type] = {}
def read_exceptions(self, fname: str):
+ """
+ Read an optional exceptions file, used to override defaults.
+ """
+
if not fname:
return
@@ -242,9 +250,9 @@ class ParseDataStructs:
def store_type(self, ln, symbol_type: str, symbol: str,
ref_name: str = None, replace_underscores: bool = True):
"""
- Stores a new symbol at self.symbols under symbol_type.
+ Store a new symbol at self.symbols under symbol_type.
- By default, underscores are replaced by "-"
+ By default, underscores are replaced by ``-``.
"""
defs = self.DEF_SYMBOL_TYPES[symbol_type]
@@ -276,12 +284,16 @@ class ParseDataStructs:
self.symbols[symbol_type][symbol] = (f"{prefix}{ref_link}{suffix}", ln)
def store_line(self, line):
- """Stores a line at self.data, properly indented"""
+ """
+ Store a line at self.data, properly indented.
+ """
line = " " + line.expandtabs()
self.data += line.rstrip(" ")
def parse_file(self, file_in: str, exceptions: str = None):
- """Reads a C source file and get identifiers"""
+ """
+ Read a C source file and get identifiers.
+ """
self.data = ""
is_enum = False
is_comment = False
@@ -433,7 +445,7 @@ class ParseDataStructs:
def gen_toc(self):
"""
- Create a list of symbols to be part of a TOC contents table
+ Create a list of symbols to be part of a TOC contents table.
"""
text = []
@@ -464,6 +476,10 @@ class ParseDataStructs:
return "\n".join(text)
def write_output(self, file_in: str, file_out: str, toc: bool):
+ """
+ Write a ReST output file.
+ """
+
title = os.path.basename(file_in)
if toc: