genindex and modindex footer links don't work in readthedocs.io

Question

I have a Python project using Sphinx for docs. I am building the docs remotely on readthedocs.io service.

I used sphinx-quickstart and it generated an index.rst file with these links in the footer:

Indices and tables
~~~~~~~~~~~~~~~~~~

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

When I push changes to readthedocs.io and build the docs, my build succeeds. Docs that I manually linked via toctree directive all work fine.

The search link works fine.

But the genindex link goes to an empty page, titled "Index"

And the modindex page links to py-modindex.html, which is a 404.

Following this guide: https://samnicholls.net/2016/06/15/how-to-sphinx-readthedocs I had run sphinx-apidoc -o api-docs/ ../myproject to generate the autodoc .rst files.

I linked the resulting api-docs/modules.rst in the toctree section at the top of my index.rst... That link works and if I click through the api-docs have been generated correctly.

Also generated by sphinx-autodoc were files for each package in my project, they contain directives like:

myproject.whatever module
-------------------------

.. automodule:: myproject.whatever
   :members:
   :undoc-members:
   :show-inheritance:

If I browse directly to these pages they have content, but they don't appear in the index (only the tocs they are manually linked in).

I also have some manually authored pages, again linked via toc.

My docs/conf.py looks like:

import os
import sys

sys.path.insert(0, os.path.abspath("../"))

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.viewcode",
    "sphinx.ext.napoleon",
    "sphinx_autodoc_typehints",
]

templates_path = ["_templates"]

exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

html_theme = "alabaster"

html_static_path = ["_static"]

I believe the fact that html generated from the autodoc .rst stub files are filled with modules and classes extracted from the .py files in my project indicates that the sys path fix and autodoc are basically working.

So my question is:

• How to make :ref:`genindex` have some content?
• How to fix :ref:`modindex` points to py-modindex.html which does not exist?

Answer 1

As I eventually worked out in the comments, thanks to help from @bad_coder, my problem was specific to building the docs in readthedocs.io

Building the docs locally worked fine.

The reason came down to use of sphinx.ext.autodoc, perhaps in conjunction with sphinx_autodoc_typehints, which seems to need to import my actual python code. Checking the logs of my apparently successful readthedocs build showed actually there were warnings like:

WARNING: autodoc: failed to import module 'whatever' from module 'myproject'; the following exception was raised:
No module named 'somelib'

i.e the docs had only partially built, it had skipped the parts it couldn't do.

The build worked locally because I was already in a virtualenv with all my project's dependencies installed.

(IMHO this seems like a bad design of the sphinx.ext.autodoc and/or sphinx_autodoc_typehints ...good static-analysis tools exist for Python which can build an AST or CST and extract structure and docstrings without importing any code.)

Well anyway, this meant that I needed to tell readthedocs.io how to install all my project deps. This is slightly complicated by the fact I'm using Poetry, which is not explicitly supported. This means I don't have a requirements.txt file to point to (and I don't want to manually create one that is a duplicate of everything in my pyproject.toml).

Fortunately the pyproject.toml file is understandable by pip, so we're able to use the pip install method described here for readthedocs.io to install both my project deps, plus extra deps that are only needed for building docs: https://github.com/readthedocs/readthedocs.org/issues/4912#issuecomment-664002569

To summarise:

1. Deleted my docs/requirements.txt
2. Added:

   [tool.poetry.dependencies]
   ...
   sphinx = {version = "^3.1.1", optional = true}
   sphinx-autodoc-typehints ={version = "^1.11.1", optional = true}
   

   and:

   [tool.poetry.extras]
   docs = [
       "sphinx",
       "sphinx-autodoc-typehints"
   ]
   

   to my pyproject.toml
3. Updated my .readthedocs.yml to:

   version: 2
   
   sphinx:
     configuration: docs/conf.py
   
   python:
     version: 3.8
     install:
       - method: pip
         path: .
         extra_requirements:
           - docs
   

4. Pushed these changes to readthedocs.io ...voilà, now it works.