Mixing code documentation with code by appending various notes to sections?

Question

Consider the following minimal example:

---

docs/index.rst

* :ref:`modindex`

docs/modules.rst

exampleproject
==============

.. toctree::
   :maxdepth: 4

   exampleproject

.. print_notes_attached_to_topic::
   :topic: notes_about_someproblem

docs/conf.py

extensions = ['sphinx.ext.autodoc']

docs/exampleproject.rst

exampleproject package
======================

Module contents
---------------

.. automodule:: exampleproject
   :members:
   :undoc-members:
   :show-inheritance:

exampleproject/__init__.py

def example_fun():
    """
    This is the docstring I'll be discussing.
    .. attach_to_topic::
       :topic: notes_about_someproblem
       
       Grass is green.
    """
    pass

def other_example_fun():
    """
    This is the second docstring I'll be discussing.
    .. attach_to_topic::
       :topic: notes_about_someproblem
       
       Sky is blue.
    """
    pass

---

I know that sphinx.ext.autodoc can extract docstrings and put them within docs/exampleproject.rst. My question is: is there a way to also automatically append some sections to specific documents? Ideally I'd like to put as much as possible of my documentation in the source code, then use the source files of the documentation to put that information in a linear order. Something like:

1. in example_fun docstring in exampleproject/__init__.py, write expression "add a note to a section/topic named building_methods saying 'before running the project, do xyz'"
2. in index.rst, "list all notes tied to a section/topic named building_methods".

---

Comparison to autodoc

I know that sphinx has sphinx.ext.autodoc that processes docstrings of applications. The problem is that autodoc forces a certain "code-first" / "API-first" structure of the documentation. What I'm looking for is a solution that will let me keep as much of the documentation in the code as possible and then turn this documentation into a linear document, with data about certain topics extracted from something like the example attach_to_topic directive.

---

Is this possible with sphinx?

Answer 1

I agree with mzjn that your example doesn't give much to go on, but I'll hazard a couple of guesses:

Appending to apidoc stub pages

The docs/exampleproject.rst and docs/modules.rst stub files get generated automatically by sphinx-apidoc. Then you manually append your own content (a .. print_notes_attached_to_topic:: directive) into those.

This will be problematic, because every time sphinx-apidoc extracts the Python docstrings to rebuild its stub pages, your chunk is overwritten/lost. I'd never try and mix my own content into pages owned/generated by an extension - they can sit side-by-side, but separate, e.g. put all your directives in a ./docs/all-my-print-notes.rst page.

That said, if you want to complicate your life, then post-process. Amend the Sphinx makefile to detect stub page changes and trigger a Bash or Python script that plops your custom content on the end. Cleanest if done indirectly -- a single include directive goes in the stub page so that your custom content can remain isolated elsewhere.

The Jinja2 templates for autodoc and autosummary (in ./site-packages/sphinx/templates/apidoc/ & ./site-packages/sphinx/ext/autosummary/templates/) offer limited customization, but really only good for layout and static content.

Your Sphinx extension

The functionally you want sounds a lot like sphinx.ext.todo. Your .. attach_to_topic:: directives are scattered throughout the Python module docstrings (equivalent to .. todo:: directives) and your .. print_notes_attached_to_topic:: directives collect up those chunks into "a linear order" (equivalent to .. todolist::).

Luckily for you, the docs about creating extensions gives that very example which you can adapt to your needs. Your extension will have one extra feature, because it provides a :topic: option to tag/classify each .. todo:: item, and also to limit the scope of each .. todolist:: collation.