From 22cba31bae9dc357622f09d22b82ca5a112b4fde Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Thu, 19 May 2016 15:14:05 +0300 Subject: Documentation/sphinx: add basic working Sphinx configuration and build Add basic configuration and makefile to build documentation from any .rst files under Documentation using Sphinx. For starters, there's just the placeholder index.rst. At the top level Makefile, hook Sphinx documentation targets alongside (but independent of) the DocBook toolchain, having both be run on the various 'make *docs' targets. All Sphinx processing is placed into Documentation/Makefile.sphinx. Both that and the Documentation/DocBook/Makefile are now expected to handle all the documentation targets, explicitly ignoring them if they're not relevant for that particular toolchain. The changes to the existing DocBook Makefile are kept minimal. There is graceful handling of missing Sphinx and rst2pdf (which is needed for pdf output) by checking for the tool and python module, respectively, with informative messages to the user. If the Read the Docs theme (sphinx_rtd_theme) is available, use it, but otherwise gracefully fall back to the Sphinx default theme, with an informative message to the user, and slightly less pretty HTML output. Sphinx can now handle htmldocs, pdfdocs (if rst2pdf is available), epubdocs and xmldocs targets. The output documents are written into per output type subdirectories under Documentation/output. Finally, you can pass options to sphinx-build using the SPHINXBUILD make variable. For example, 'make SPHINXOPTS=-v htmldocs' for more verbose output from Sphinx. This is based on the original work by Jonathan Corbet, but he probably wouldn't recognize this as his own anymore. Signed-off-by: Jani Nikula --- Documentation/DocBook/Makefile | 7 +- Documentation/Makefile.sphinx | 61 +++++++ Documentation/conf.py | 387 +++++++++++++++++++++++++++++++++++++++++ Documentation/index.rst | 23 +++ 4 files changed, 474 insertions(+), 4 deletions(-) create mode 100644 Documentation/Makefile.sphinx create mode 100644 Documentation/conf.py create mode 100644 Documentation/index.rst (limited to 'Documentation') diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile index d70f9b68174e..e0c7e1e0590b 100644 --- a/Documentation/DocBook/Makefile +++ b/Documentation/DocBook/Makefile @@ -33,10 +33,6 @@ PDF_METHOD = $(prefer-db2x) PS_METHOD = $(prefer-db2x) -### -# The targets that may be used. -PHONY += xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs cleandocs - targets += $(DOCBOOKS) BOOKS := $(addprefix $(obj)/,$(DOCBOOKS)) xmldocs: $(BOOKS) @@ -63,6 +59,9 @@ installmandocs: mandocs sort -k 2 -k 1 | uniq -f 1 | sed -e 's: :/:' | \ xargs install -m 644 -t /usr/local/man/man9/ +# no-op for the DocBook toolchain +epubdocs: + ### #External programs used KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx new file mode 100644 index 000000000000..25a7970bfe5c --- /dev/null +++ b/Documentation/Makefile.sphinx @@ -0,0 +1,61 @@ +# -*- makefile -*- +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXBUILD = sphinx-build +SPHINXOPTS = +PAPER = +BUILDDIR = $(obj)/output + +# User-friendly check for sphinx-build +HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi) + +ifeq ($(HAVE_SPHINX),0) + +.DEFAULT: + $(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.) + @echo " SKIP Sphinx $@ target." + +else # HAVE_SPHINX + +# User-friendly check for rst2pdf +HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi) + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . + +quiet_cmd_sphinx = SPHINX $@ + cmd_sphinx = $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 + +htmldocs: + $(call cmd,sphinx,html) + +pdfdocs: +ifeq ($(HAVE_RST2PDF),0) + $(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.) + @echo " SKIP Sphinx $@ target." +else # HAVE_RST2PDF + $(call cmd,sphinx,pdf) +endif # HAVE_RST2PDF + +epubdocs: + $(call cmd,sphinx,epub) + +xmldocs: + $(call cmd,sphinx,xml) + +# no-ops for the Sphinx toolchain +sgmldocs: +psdocs: +mandocs: +installmandocs: + +cleandocs: + $(Q)rm -rf $(BUILDDIR) + +endif # HAVE_SPHINX diff --git a/Documentation/conf.py b/Documentation/conf.py new file mode 100644 index 000000000000..0f0e20757bb1 --- /dev/null +++ b/Documentation/conf.py @@ -0,0 +1,387 @@ +# -*- coding: utf-8 -*- +# +# The Linux Kernel documentation build configuration file, created by +# sphinx-quickstart on Fri Feb 12 13:51:46 2016. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys +import os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +#sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [] + +# Gracefully handle missing rst2pdf. +try: + import rst2pdf + extensions += ['rst2pdf.pdfbuilder'] +except ImportError: + pass + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = 'The Linux Kernel' +copyright = '2016, The kernel development community' +author = 'The kernel development community' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '4.5' +# The full version, including alpha/beta/rc tags. +release = '4.5' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['output'] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +#keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = False + +primary_domain = 'C' +highlight_language = 'C' + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. + +# The Read the Docs theme is available from +# - https://github.com/snide/sphinx_rtd_theme +# - https://pypi.python.org/pypi/sphinx_rtd_theme +# - python-sphinx-rtd-theme package (on Debian) +try: + import sphinx_rtd_theme + html_theme = 'sphinx_rtd_theme' + html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] +except ImportError: + sys.stderr.write('Warning: The Sphinx \'sphinx_rtd_theme\' HTML theme was not found. Make sure you have the theme installed to produce pretty HTML output. Falling back to the default theme.\n') + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +#html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +#html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +#html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +#html_static_path = ['_static'] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +#html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +#html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +#html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +#html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +#html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +#html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr' +#html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +#html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +#html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = 'TheLinuxKerneldoc' + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { +# The paper size ('letterpaper' or 'a4paper'). +#'papersize': 'letterpaper', + +# The font size ('10pt', '11pt' or '12pt'). +#'pointsize': '10pt', + +# Additional stuff for the LaTeX preamble. +#'preamble': '', + +# Latex figure (float) alignment +#'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation', + 'The kernel development community', 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'thelinuxkernel', 'The Linux Kernel Documentation', + [author], 1) +] + +# If true, show URL addresses after external links. +#man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'TheLinuxKernel', 'The Linux Kernel Documentation', + author, 'TheLinuxKernel', 'One line description of project.', + 'Miscellaneous'), +] + +# Documents to append as an appendix to all manuals. +#texinfo_appendices = [] + +# If false, no module index is generated. +#texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +#texinfo_no_detailmenu = False + + +# -- Options for Epub output ---------------------------------------------- + +# Bibliographic Dublin Core info. +epub_title = project +epub_author = author +epub_publisher = author +epub_copyright = copyright + +# The basename for the epub file. It defaults to the project name. +#epub_basename = project + +# The HTML theme for the epub output. Since the default themes are not +# optimized for small screen space, using the same theme for HTML and epub +# output is usually not wise. This defaults to 'epub', a theme designed to save +# visual space. +#epub_theme = 'epub' + +# The language of the text. It defaults to the language option +# or 'en' if the language is not set. +#epub_language = '' + +# The scheme of the identifier. Typical schemes are ISBN or URL. +#epub_scheme = '' + +# The unique identifier of the text. This can be a ISBN number +# or the project homepage. +#epub_identifier = '' + +# A unique identification for the text. +#epub_uid = '' + +# A tuple containing the cover image and cover page html template filenames. +#epub_cover = () + +# A sequence of (type, uri, title) tuples for the guide element of content.opf. +#epub_guide = () + +# HTML files that should be inserted before the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_pre_files = [] + +# HTML files that should be inserted after the pages created by sphinx. +# The format is a list of tuples containing the path and title. +#epub_post_files = [] + +# A list of files that should not be packed into the epub file. +epub_exclude_files = ['search.html'] + +# The depth of the table of contents in toc.ncx. +#epub_tocdepth = 3 + +# Allow duplicate toc entries. +#epub_tocdup = True + +# Choose between 'default' and 'includehidden'. +#epub_tocscope = 'default' + +# Fix unsupported image types using the Pillow. +#epub_fix_images = False + +# Scale large images. +#epub_max_image_width = 0 + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +#epub_show_urls = 'inline' + +# If false, no index is generated. +#epub_use_index = True + +#======= +# rst2pdf +# +# Grouping the document tree into PDF files. List of tuples +# (source start file, target name, title, author, options). +# +# See the Sphinx chapter of http://ralsina.me/static/manual.pdf +# +# FIXME: Do not add the index file here; the result will be too big. Adding +# multiple PDF files here actually tries to get the cross-referencing right +# *between* PDF files. +pdf_documents = [ + ('index', u'Kernel', u'Kernel', u'J. Random Bozo'), +] diff --git a/Documentation/index.rst b/Documentation/index.rst new file mode 100644 index 000000000000..71a276f34c7f --- /dev/null +++ b/Documentation/index.rst @@ -0,0 +1,23 @@ +.. The Linux Kernel documentation master file, created by + sphinx-quickstart on Fri Feb 12 13:51:46 2016. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Welcome to The Linux Kernel's documentation! +============================================ + +Nothing for you to see here *yet*. Please move along. + +Contents: + +.. toctree:: + :maxdepth: 2 + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` + -- cgit v1.2.3 From 81cd318102aedee64d3bd38d2e04ea333569abfd Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Thu, 19 May 2016 20:53:23 +0300 Subject: Documentation: add .gitignore The Sphinx output directory is generated. Signed-off-by: Jani Nikula --- Documentation/.gitignore | 1 + 1 file changed, 1 insertion(+) create mode 100644 Documentation/.gitignore (limited to 'Documentation') diff --git a/Documentation/.gitignore b/Documentation/.gitignore new file mode 100644 index 000000000000..53752db253e3 --- /dev/null +++ b/Documentation/.gitignore @@ -0,0 +1 @@ +output -- cgit v1.2.3 From c56de1db54dfbbdfc02a500388952af16e3a9368 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Wed, 18 May 2016 23:30:30 +0300 Subject: Documentation/sphinx: add Sphinx kernel-doc directive extension Add an extension to handle kernel-doc directives, to call kernel-doc according to the arguments and parameters given to the reStructuredText directive. The syntax for the kernel-doc directive is: .. kernel-doc:: FILENAME :export: :internal: :functions: FUNCTION [FUNCTION ...] :doc: SECTION TITLE Of the directive options export, internal, functions, and doc, currently only one option may be given at a time. The FILENAME is relative from the kernel source tree root. The extension notifies Sphinx about the document dependency on FILENAME, causing the document to be rebuilt when the file has been changed. Signed-off-by: Jani Nikula --- Documentation/sphinx/kernel-doc.py | 102 +++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 Documentation/sphinx/kernel-doc.py (limited to 'Documentation') diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py new file mode 100644 index 000000000000..87a1332fe934 --- /dev/null +++ b/Documentation/sphinx/kernel-doc.py @@ -0,0 +1,102 @@ +# coding=utf-8 +# +# Copyright © 2016 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. +# +# Authors: +# Jani Nikula + +import os +import subprocess +import sys + +from docutils import nodes, statemachine +from docutils.parsers.rst import directives +from sphinx.util.compat import Directive + +class KernelDocDirective(Directive): + """Extract kernel-doc comments from the specified file""" + required_argument = 1 + optional_arguments = 4 + option_spec = { + 'doc': directives.unchanged_required, + 'functions': directives.unchanged_required, + 'export': directives.flag, + 'internal': directives.flag, + } + has_content = False + + def run(self): + env = self.state.document.settings.env + cmd = [env.config.kerneldoc_bin, '-rst'] + + filename = env.config.kerneldoc_srctree + '/' + self.arguments[0] + + # Tell sphinx of the dependency + env.note_dependency(os.path.abspath(filename)) + + tab_width = self.options.get('tab-width', self.state.document.settings.tab_width) + source = self.state_machine.input_lines.source(self.lineno - self.state_machine.input_offset - 1) + + # FIXME: make this nicer and more robust against errors + if 'export' in self.options: + cmd += ['-export'] + elif 'internal' in self.options: + cmd += ['-internal'] + elif 'doc' in self.options: + cmd += ['-function', str(self.options.get('doc'))] + elif 'functions' in self.options: + for f in str(self.options.get('functions')).split(' '): + cmd += ['-function', f] + + cmd += [filename] + + try: + env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd))) + + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + out, err = p.communicate() + + # assume the kernel sources are utf-8 + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + + if p.returncode != 0: + sys.stderr.write(err) + + env.app.warn('kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode)) + return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] + elif env.config.kerneldoc_verbosity > 0: + sys.stderr.write(err) + + lines = statemachine.string2lines(out, tab_width, convert_whitespace=True) + self.state_machine.insert_input(lines, source) + return [] + except Exception as e: + env.app.warn('kernel-doc \'%s\' processing failed with: %s' % + (" ".join(cmd), str(e))) + return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] + +def setup(app): + app.add_config_value('kerneldoc_bin', None, 'env') + app.add_config_value('kerneldoc_srctree', None, 'env') + app.add_config_value('kerneldoc_verbosity', 1, 'env') + + app.add_directive('kernel-doc', KernelDocDirective) -- cgit v1.2.3 From 24dcdeb28b55afafa9cdf6acedd78cf1aa8f8428 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Fri, 20 May 2016 11:51:47 +0300 Subject: Documentation/sphinx: configure the kernel-doc extension Tell Sphinx where to find the extension, and pass on the kernel src tree and kernel-doc paths to the extension. With this, any .rst files under Documentation may contain the kernel-doc rst directive to include kernel-doc documentation from any source file. While building, it may be handy to pass kernel-doc extension configuration on the command line. For example, 'make SPHINXOPTS="-D kerneldoc_verbosity=0" htmldocs' silences all stderr output from kernel-doc when the kernel-doc exit code is 0. (The stderr will be logged unconditionally when the exit code is non-zero.) Signed-off-by: Jani Nikula --- Documentation/Makefile.sphinx | 4 +++- Documentation/conf.py | 10 ++++++++-- 2 files changed, 11 insertions(+), 3 deletions(-) (limited to 'Documentation') diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx index 25a7970bfe5c..8a662be9bd87 100644 --- a/Documentation/Makefile.sphinx +++ b/Documentation/Makefile.sphinx @@ -25,7 +25,9 @@ HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) +KERNELDOC = $(srctree)/scripts/kernel-doc +KERNELDOC_CONF = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC) +ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . diff --git a/Documentation/conf.py b/Documentation/conf.py index 0f0e20757bb1..8b72577f38f5 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -18,7 +18,7 @@ import os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -#sys.path.insert(0, os.path.abspath('.')) +sys.path.insert(0, os.path.abspath('sphinx')) # -- General configuration ------------------------------------------------ @@ -28,7 +28,7 @@ import os # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = [] +extensions = ['kernel-doc'] # Gracefully handle missing rst2pdf. try: @@ -385,3 +385,9 @@ epub_exclude_files = ['search.html'] pdf_documents = [ ('index', u'Kernel', u'Kernel', u'J. Random Bozo'), ] + +# kernel-doc extension configuration for running Sphinx directly (e.g. by Read +# the Docs). In a normal build, these are supplied from the Makefile via command +# line arguments. +kerneldoc_bin = '../scripts/kernel-doc' +kerneldoc_srctree = '..' -- cgit v1.2.3 From c13ce448c8532965eb4e1115e78cb5b8ff8261a3 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Sat, 28 May 2016 15:25:41 +0300 Subject: Documentation/sphinx: set version and release properly Read the version and release from the top level Makefile (for use when Sphinx is invoked directly, by e.g. Read the Docs), but override them via Sphinx command line arguments in a normal documentation build. Signed-off-by: Jani Nikula --- Documentation/Makefile.sphinx | 2 +- Documentation/conf.py | 29 +++++++++++++++++++++++++---- 2 files changed, 26 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx index 8a662be9bd87..addf32309bc3 100644 --- a/Documentation/Makefile.sphinx +++ b/Documentation/Makefile.sphinx @@ -27,7 +27,7 @@ PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter KERNELDOC = $(srctree)/scripts/kernel-doc KERNELDOC_CONF = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC) -ALLSPHINXOPTS = -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) +ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src) # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . diff --git a/Documentation/conf.py b/Documentation/conf.py index 8b72577f38f5..6cc41a0555a3 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -60,10 +60,31 @@ author = 'The kernel development community' # |version| and |release|, also used in various other places throughout the # built documents. # -# The short X.Y version. -version = '4.5' -# The full version, including alpha/beta/rc tags. -release = '4.5' +# In a normal build, version and release are are set to KERNELVERSION and +# KERNELRELEASE, respectively, from the Makefile via Sphinx command line +# arguments. +# +# The following code tries to extract the information by reading the Makefile, +# when Sphinx is run directly (e.g. by Read the Docs). +try: + makefile_version = None + makefile_patchlevel = None + for line in open('../Makefile'): + key, val = [x.strip() for x in line.split('=', 2)] + if key == 'VERSION': + makefile_version = val + elif key == 'PATCHLEVEL': + makefile_patchlevel = val + if makefile_version and makefile_patchlevel: + break +except: + pass +finally: + if makefile_version and makefile_patchlevel: + version = release = makefile_version + '.' + makefile_patchlevel + else: + sys.stderr.write('Warning: Could not extract kernel version\n') + version = release = "unknown version" # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. -- cgit v1.2.3 From 89a66d76109935aad387a5058f120aeb8246ae40 Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Sun, 6 Mar 2016 20:29:31 -0700 Subject: sphinx: cheesy script to convert .tmpl files This script uses pandoc to convert existing DocBook template files to RST templates. A couple of sed scripts are need to massage things both before and after the conversion, but the result is then usable with no hand editing. [Jani: Change usage to tmplcvt . Fix escaping for docproc directives. Add support the new kernel-doc extension.] Signed-off-by: Jonathan Corbet Signed-off-by: Jani Nikula --- Documentation/sphinx/convert_template.sed | 14 ++++++++++++++ Documentation/sphinx/post_convert.sed | 19 +++++++++++++++++++ Documentation/sphinx/tmplcvt | 19 +++++++++++++++++++ 3 files changed, 52 insertions(+) create mode 100644 Documentation/sphinx/convert_template.sed create mode 100644 Documentation/sphinx/post_convert.sed create mode 100755 Documentation/sphinx/tmplcvt (limited to 'Documentation') diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed new file mode 100644 index 000000000000..d53bb8220a26 --- /dev/null +++ b/Documentation/sphinx/convert_template.sed @@ -0,0 +1,14 @@ +# +# Pandoc doesn't grok or , so convert them +# ahead of time. +# +# Use "$bq" instead of "`" so that pandoc won't mess with it. +# +s%\([^<(]\+\)()%:c:func:$bq\1$bq%g +s%\([^<(]\+\)%:c:func:$bq\1$bq%g +s%struct *\([^<]\+\)%:ref:$bqstruct \1$bq%g +s%\([^<]\+\)%:ref:$bqstruct \1$bq%g +# +# Wrap docproc directives in para and code blocks. +# +s%^\(!.*\)$%DOCPROC: \1% diff --git a/Documentation/sphinx/post_convert.sed b/Documentation/sphinx/post_convert.sed new file mode 100644 index 000000000000..ce17049f6a8e --- /dev/null +++ b/Documentation/sphinx/post_convert.sed @@ -0,0 +1,19 @@ +# +# pandoc thinks that both "_" needs to be escaped. Remove the extra +# backslashes. Also put in proper backquotes now that pandoc won't quote +# them. +# +s/$bq/`/g +s/\\_/_/g +# +# Unwrap docproc directives. +# +s/^``DOCPROC: !E\(.*\)``$/.. kernel-doc:: \1\n :export:/ +s/^``DOCPROC: !I\(.*\)``$/.. kernel-doc:: \1\n :internal:/ +s/^``DOCPROC: !F\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :functions: \2/ +s/^``DOCPROC: !P\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :doc: \2/ +s/^``DOCPROC: \(!.*\)``$/.. WARNING: DOCPROC directive not supported: \1/ +# +# Trim trailing whitespace. +# +s/[[:space:]]*$// diff --git a/Documentation/sphinx/tmplcvt b/Documentation/sphinx/tmplcvt new file mode 100755 index 000000000000..909a73065e0a --- /dev/null +++ b/Documentation/sphinx/tmplcvt @@ -0,0 +1,19 @@ +#!/bin/bash +# +# Convert a template file into something like RST +# +# fix +# feed to pandoc +# fix \_ +# title line? +# + +in=$1 +rst=$2 +tmp=$rst.tmp + +cp $in $tmp +sed --in-place -f convert_template.sed $tmp +pandoc -s -S -f docbook -t rst -o $rst $tmp +sed --in-place -f post_convert.sed $rst +rm $tmp -- cgit v1.2.3 From 2e83ecb834ca923806fd58fab17d9f405767e1d9 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Thu, 26 May 2016 10:15:18 +0300 Subject: sphinx: update docbook->rst conversion script match C domain spec Function references should include the parens (), struct references should not include "struct". Signed-off-by: Jani Nikula --- Documentation/sphinx/convert_template.sed | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed index d53bb8220a26..0553eabeac79 100644 --- a/Documentation/sphinx/convert_template.sed +++ b/Documentation/sphinx/convert_template.sed @@ -4,10 +4,10 @@ # # Use "$bq" instead of "`" so that pandoc won't mess with it. # -s%\([^<(]\+\)()%:c:func:$bq\1$bq%g -s%\([^<(]\+\)%:c:func:$bq\1$bq%g -s%struct *\([^<]\+\)%:ref:$bqstruct \1$bq%g -s%\([^<]\+\)%:ref:$bqstruct \1$bq%g +s%\([^<(]\+\)()%:c:func:$bq\1()$bq%g +s%\([^<(]\+\)%:c:func:$bq\1()$bq%g +s%struct *\([^<]\+\)%:c:type:$bq\1$bq%g +s%\([^<]\+\)%:c:type:$bq\1$bq%g # # Wrap docproc directives in para and code blocks. # -- cgit v1.2.3 From 30ca7aaf279944da930b587b05d9b325ec7f821d Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Thu, 26 May 2016 10:40:17 +0300 Subject: Documentation/sphinx: nicer referencing of struct in docbook->rst conversion Add "struct" in the label of the reference. Signed-off-by: Jani Nikula --- Documentation/sphinx/convert_template.sed | 10 +++++++--- Documentation/sphinx/post_convert.sed | 10 +++++++--- 2 files changed, 14 insertions(+), 6 deletions(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/convert_template.sed b/Documentation/sphinx/convert_template.sed index 0553eabeac79..c1503fcca4ec 100644 --- a/Documentation/sphinx/convert_template.sed +++ b/Documentation/sphinx/convert_template.sed @@ -2,12 +2,16 @@ # Pandoc doesn't grok or , so convert them # ahead of time. # -# Use "$bq" instead of "`" so that pandoc won't mess with it. +# Use the following escapes to pass through pandoc: +# $bq = "`" +# $lt = "<" +# $gt = ">" # s%\([^<(]\+\)()%:c:func:$bq\1()$bq%g s%\([^<(]\+\)%:c:func:$bq\1()$bq%g -s%struct *\([^<]\+\)%:c:type:$bq\1$bq%g -s%\([^<]\+\)%:c:type:$bq\1$bq%g +s%struct *\([^<]\+\)%:c:type:$bqstruct \1 $lt\1$gt$bq%g +s%struct \([^<]\+\)%:c:type:$bqstruct \1 $lt\1$gt$bq%g +s%\([^<]\+\)%:c:type:$bqstruct \1 $lt\1$gt$bq%g # # Wrap docproc directives in para and code blocks. # diff --git a/Documentation/sphinx/post_convert.sed b/Documentation/sphinx/post_convert.sed index ce17049f6a8e..392770bac53b 100644 --- a/Documentation/sphinx/post_convert.sed +++ b/Documentation/sphinx/post_convert.sed @@ -1,9 +1,13 @@ # -# pandoc thinks that both "_" needs to be escaped. Remove the extra -# backslashes. Also put in proper backquotes now that pandoc won't quote -# them. +# Unescape. # s/$bq/`/g +s/$lt//g +# +# pandoc thinks that both "_" needs to be escaped. Remove the extra +# backslashes. +# s/\\_/_/g # # Unwrap docproc directives. -- cgit v1.2.3 From ba35018593547eac69998e2571fc95cf620f2220 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 31 May 2016 18:09:12 +0300 Subject: Documentation/sphinx: fix kernel-doc extension on python3 Reconcile differences between python2 and python3 on dealing with stdout, stderr from Popen. This fixes "name 'unicode' is not defined" errors on python3. We'll need to try to keep the extension working on both python-sphinx and python3-sphinx so we don't need two copies. Reported-and-tested-by: Marius Vlad Signed-off-by: Jani Nikula --- Documentation/sphinx/kernel-doc.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py index 87a1332fe934..a6e170872c8a 100644 --- a/Documentation/sphinx/kernel-doc.py +++ b/Documentation/sphinx/kernel-doc.py @@ -23,6 +23,9 @@ # # Authors: # Jani Nikula +# +# Please make sure this works on both python2 and python3. +# import os import subprocess @@ -75,8 +78,10 @@ class KernelDocDirective(Directive): p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) out, err = p.communicate() - # assume the kernel sources are utf-8 - out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + # python2 needs conversion to unicode. + # python3 with universal_newlines=True returns strings. + if sys.version_info.major < 3: + out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') if p.returncode != 0: sys.stderr.write(err) -- cgit v1.2.3 From 9cc1a44c03113cb2f3be73650d7c611c3e273ada Mon Sep 17 00:00:00 2001 From: Daniel Vetter Date: Wed, 1 Jun 2016 11:12:42 +0200 Subject: doc/sphinx: Pass right filename as source With this error output becomes almost readable. The line numbers are still totally bonghits, but that's a lot harder to pull out of kerneldoc. We'd essentially have to insert some special markers in the kernel-doc output, split the output along these markers and then insert each block separately using state_machine.insert_input(block, source, first_line) Cc: Jani Nikula Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet Signed-off-by: Daniel Vetter Signed-off-by: Jani Nikula --- Documentation/sphinx/kernel-doc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py index a6e170872c8a..9fc2c27916a9 100644 --- a/Documentation/sphinx/kernel-doc.py +++ b/Documentation/sphinx/kernel-doc.py @@ -57,7 +57,7 @@ class KernelDocDirective(Directive): env.note_dependency(os.path.abspath(filename)) tab_width = self.options.get('tab-width', self.state.document.settings.tab_width) - source = self.state_machine.input_lines.source(self.lineno - self.state_machine.input_offset - 1) + source = filename # FIXME: make this nicer and more robust against errors if 'export' in self.options: -- cgit v1.2.3 From 16e161c8c67ac3b27cfc096cf07af9bceb144707 Mon Sep 17 00:00:00 2001 From: Daniel Vetter Date: Thu, 2 Jun 2016 14:59:18 +0200 Subject: doc/sphinx: Stop touching state_machine internals Instead of just forcefully inserting our kernel-doc input and letting the state machine stumble over it the recommended way is to create ViewList, parse that and then return the list of parsed nodes. Suggested by Jani. Cc: Jani Nikula Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet Signed-off-by: Daniel Vetter Signed-off-by: Jani Nikula --- Documentation/sphinx/kernel-doc.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py index 9fc2c27916a9..bd422870101e 100644 --- a/Documentation/sphinx/kernel-doc.py +++ b/Documentation/sphinx/kernel-doc.py @@ -32,6 +32,7 @@ import subprocess import sys from docutils import nodes, statemachine +from docutils.statemachine import ViewList from docutils.parsers.rst import directives from sphinx.util.compat import Directive @@ -92,8 +93,14 @@ class KernelDocDirective(Directive): sys.stderr.write(err) lines = statemachine.string2lines(out, tab_width, convert_whitespace=True) - self.state_machine.insert_input(lines, source) - return [] + result = ViewList(lines, source) + + node = nodes.section() + node.document = self.state.document + self.state.nested_parse(result, self.content_offset, node) + + return node.children + except Exception as e: env.app.warn('kernel-doc \'%s\' processing failed with: %s' % (" ".join(cmd), str(e))) -- cgit v1.2.3 From d90368f2fa7ded7c56d214aef087e88bba5199e7 Mon Sep 17 00:00:00 2001 From: Daniel Vetter Date: Fri, 3 Jun 2016 22:21:35 +0200 Subject: doc/sphinx: Track line-number of starting blocks Design is pretty simple: kernel-doc inserts breadcrumbs with line numbers, and sphinx picks them up. At first I went with a sphinx comment, but inserting those at random places seriously upsets the parser, and must be filtered. Hence why this version now uses "#define LINEO " since one of these ever escape into output it's pretty clear there is a bug. It seems to work well, and at least the 2-3 errors where sphinx complained about something that was not correct in kernel-doc text the line numbers matched up perfectly. v2: Instead of noodling around in the parser state machine, create a ViewList and parse it ourselves. This seems to be the recommended way, per Jani's suggestion. v3: - Split out ViewList pach. Splitting the kernel-doc changes from the sphinx ones isn't possible, since emitting the LINENO lines wreaks havoc with the rst formatting. We must filter them. - Improve the regex per Jani's suggestions, and compile it just once for speed. - Now that LINENO lines are eaten, also add them to function parameter descriptions. Much less content and offset than for in-line struct member descriptions, but still nice to know which exact continuation line upsets sphinx. - Simplify/clarify the line +/-1 business a bit. v4: Split out the scripts/kernel-doc changes and make line-numbers opt-in, as suggested by Jani. Cc: Jani Nikula Cc: linux-doc@vger.kernel.org Cc: Jonathan Corbet Signed-off-by: Daniel Vetter Signed-off-by: Jani Nikula --- Documentation/sphinx/kernel-doc.py | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py index bd422870101e..4adfb0e91ecc 100644 --- a/Documentation/sphinx/kernel-doc.py +++ b/Documentation/sphinx/kernel-doc.py @@ -30,6 +30,7 @@ import os import subprocess import sys +import re from docutils import nodes, statemachine from docutils.statemachine import ViewList @@ -50,7 +51,7 @@ class KernelDocDirective(Directive): def run(self): env = self.state.document.settings.env - cmd = [env.config.kerneldoc_bin, '-rst'] + cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno'] filename = env.config.kerneldoc_srctree + '/' + self.arguments[0] @@ -93,7 +94,19 @@ class KernelDocDirective(Directive): sys.stderr.write(err) lines = statemachine.string2lines(out, tab_width, convert_whitespace=True) - result = ViewList(lines, source) + result = ViewList() + + lineoffset = 0; + line_regex = re.compile("^#define LINENO ([0-9]+)$") + for line in lines: + match = line_regex.search(line) + if match: + # sphinx counts lines from 0 + lineoffset = int(match.group(1)) - 1 + # we must eat our comments since the upset the markup + else: + result.append(line, source, lineoffset) + lineoffset += 1 node = nodes.section() node.document = self.state.document -- cgit v1.2.3