Re: [PATCH] docs: translations: add translations links when they exist

[Jani Nikula]

On Sat, 28 Oct 2023, Vegard Nossum <vegard.nossum@xxxxxxxxxx> wrote:
> Add a new Sphinx extension that knows about the translations of kernel
> documentation and can insert links to the translations at the top of
> the document.
>
> It basically works like this:
>
> 1. Register a new node type, LanguagesNode.
>
> 2. Register a new transform, TranslationsTransform, that inserts a new
>    LanguageNode at the top of every document. The LanguageNode contains
>    "pending references" to translations of the document. The key here
>    is that these are pending (i.e. unresolved) references that may or
>    may not actually exist.
>
> 3. Register a 'doctree-resolved' event that iterates over all the
>    LanguageNode nodes. Any unresolved references are filtered out; the
>    list of resolved references is passed to the 'translations.html'
>    template and rendered as an HTML node (if HTML output is selected).
>
> Testing: make htmldocs with v7.3.0.

I'm just observing the kernel documentation has a system of its own for
translations. Maybe it's fine this way, but it's certainly different
from what the Sphinx developers had in mind. (But I'm not an expert on
this field, don't ask me how this should be done!)

Regardless, the implication is that this builds on the translation
solution chosen for kernel documentation, for better and for worse, and
raises the bar for switching later. It's something to be aware of.

>
> Signed-off-by: Vegard Nossum <vegard.nossum@xxxxxxxxxx>
> ---
>  Documentation/conf.py                         |  2 +-
>  Documentation/sphinx-static/custom.css        |  8 ++
>  .../sphinx/templates/translations.html        | 12 +++
>  Documentation/sphinx/translations.py          | 96 +++++++++++++++++++
>  4 files changed, 117 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/sphinx/templates/translations.html
>  create mode 100644 Documentation/sphinx/translations.py
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index d4fdf6a3875a..64eab500b2cd 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -55,7 +55,7 @@ needs_sphinx = '1.7'
>  extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
>                'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
>                'maintainers_include', 'sphinx.ext.autosectionlabel',
> -              'kernel_abi', 'kernel_feat']
> +              'kernel_abi', 'kernel_feat', 'translations']
>  
>  if major >= 3:
>      if (major > 3) or (minor > 0 or patch >= 2):
> diff --git a/Documentation/sphinx-static/custom.css b/Documentation/sphinx-static/custom.css
> index 084a884f6fb7..33adee4a35d9 100644
> --- a/Documentation/sphinx-static/custom.css
> +++ b/Documentation/sphinx-static/custom.css
> @@ -73,3 +73,11 @@ input.kernel-toc-toggle { display: none; }
>      h3.kernel-toc-contents { display: inline; }
>      div.kerneltoc a { color: black; }
>  }
> +
> +/* Language selection bar */
> +div.language-selection {
> +    background: #eeeeee;
> +    border: 1px solid #cccccc;
> +    margin-bottom: 1em;
> +    padding: .5em;
> +}
> diff --git a/Documentation/sphinx/templates/translations.html b/Documentation/sphinx/templates/translations.html
> new file mode 100644
> index 000000000000..08afb595c203
> --- /dev/null
> +++ b/Documentation/sphinx/templates/translations.html
> @@ -0,0 +1,12 @@
> +<!-- SPDX-License-Identifier: GPL-2.0 -->
> +<!-- Copyright © 2023, Oracle and/or its affiliates. -->
> +
> +{# Create a language bar for translations #}
> +{% if languages|length > 0: %}
> +<div class="language-selection">
> +Languages:
> +{% for ref in languages: %}
> +<a href="{{ ref.refuri }}">{{ ref.astext() }}</a>{% if not loop.last %}, {% endif %}
> +{% endfor %}
> +</div>
> +{% endif %}

This could also be part of the menu on the left, I think it's fairly
easy to extend in alabaster. But then again it's already pretty crowded,
and it's all a matter of taste.

> diff --git a/Documentation/sphinx/translations.py b/Documentation/sphinx/translations.py
> new file mode 100644
> index 000000000000..e1da811bdaf0
> --- /dev/null
> +++ b/Documentation/sphinx/translations.py
> @@ -0,0 +1,96 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Copyright © 2023, Oracle and/or its affiliates.
> +# Author: Vegard Nossum <vegard.nossum@xxxxxxxxxx>
> +#
> +# Add translation links to the top of the document.
> +#
> +
> +import os
> +
> +from docutils import nodes
> +from docutils.transforms import Transform
> +
> +import sphinx
> +from sphinx import addnodes
> +from sphinx.errors import NoUri
> +
> +all_languages = {
> +    # English is always first
> +    None: 'English',
> +
> +    # Keep the rest sorted alphabetically
> +    'zh_CN': 'Chinese',
> +    'it_IT': 'Italian',
> +    'ja_JP': 'Japanese',
> +    'ko_KR': 'Korean',
> +    'sp_SP': 'Spanish',
> +    'zh_TW': 'Taiwanese',
> +}

Maybe the path to translations should be a configuration option for the
extension, and it could read the translations from the directory instead
of hard coding them here.

> +
> +class LanguagesNode(nodes.Element):
> +    pass
> +
> +class TranslationsTransform(Transform):
> +    default_priority = 900
> +
> +    def apply(self):
> +        app = self.document.settings.env.app
> +        if app.builder.format not in ['html']:
> +            return
> +
> +        docname = self.document.settings.env.docname
> +
> +        this_lang_code = None
> +        components = docname.split(os.sep)
> +        if components[0] == 'translations' and len(components) > 2:
> +            this_lang_code = components[1]
> +
> +            # normalize docname to be the untranslated one
> +            docname = os.path.join(*components[2:])

Need to rename these files, and keep the translated document names
up-to-date:

for f in $(find Documentation/translations/ -name "*.rst"); do if ! test -f Documentation/${f##Documentation/translations/??_??/}; then echo $f; fi; done

Maybe the extension should warn about documents under translations that
do not have a corresponding original.

BR,
Jani.

> +
> +        new_nodes = LanguagesNode()
> +
> +        for lang_code, lang_name in all_languages.items():
> +            if lang_code == this_lang_code:
> +                continue
> +
> +            if lang_code is None:
> +                target_name = docname
> +            else:
> +                target_name = os.path.join('translations', lang_code, docname)
> +
> +            pxref = addnodes.pending_xref('', refdomain='std',
> +                reftype='doc', reftarget='/' + target_name, modname=None,
> +                classname=None, refexplicit=True)
> +            pxref += nodes.Text(lang_name)
> +            new_nodes += pxref
> +
> +        self.document.insert(0, new_nodes)
> +
> +def process_languages(app, doctree, docname):
> +    for node in doctree.traverse(LanguagesNode):
> +        languages = []
> +
> +        # Iterate over the child nodes; any resolved links will have
> +        # the type 'nodes.reference', while unresolved links will be
> +        # type 'nodes.Text'.
> +        languages = list(filter(lambda xref:
> +            isinstance(xref, nodes.reference), node.children))
> +
> +        html_content = app.builder.templates.render('translations.html',
> +            context={
> +                'languages': languages,
> +            })
> +
> +        node.replace_self(nodes.raw('', html_content, format='html'))
> +
> +def setup(app):
> +    app.add_node(LanguagesNode)
> +    app.add_transform(TranslationsTransform)
> +    app.connect('doctree-resolved', process_languages)
> +
> +    return {
> +        'parallel_read_safe': True,
> +        'parallel_write_safe': True,
> +    }

-- 
Jani Nikula, Intel