What is the correct workflow in using Sphinx for Python project documentation?

I want to use Sphinx to document a large project that is currently not well-documented. In particular I want to use sphinx-apidoc to produce the documentation from the code as I document it.

However I also want to have some pages of tutorial on how to use the project etc., but it seems when I call sphinx-apidoc it generates the whole document at once.

So my question is: What is the correct workflow here so I could write the tutorial pages that are to be written manually and at the same time update the documentation in the code? Note that if I update the manually written tutorial pages (e.g. included in the index.txt) and run sphinx-apidoc I will lose them as the whole document is generated at once.

In general are there any guidelines as how to proceed in building the documentation?

Note: The source of inconvenience is that the basic procedure assumes you have all the code documentation already in place and will not make any updates as you proceed in producing the documentation. At least this is what I need to resolve.

Solution

First you run sphinx-quickstart and accept the defaults to set up your basic structure this is only done once and you edit the table of contents section of index.rst to point to your tutorials, give an introduction, etc. - the you at least outline your tutorials in separate .rst files. You also edit config.py to add autodoc - from the website:

When documenting Python code, it is common to put a lot of documentation in the source files, in documentation strings. Sphinx supports the inclusion of docstrings from your modules with an extension (an extension is a Python module that provides additional features for Sphinx projects) called “autodoc”.

In order to use autodoc, you need to activate it in conf.py by putting the string 'sphinx.ext.autodoc' into the list assigned to the extensions config value. Then, you have a few additional directives at your disposal.

For example, to document the function io.open(), reading its signature and docstring from the source file, you’d write this:

.. autofunction:: io.open You can also document whole classes or even modules automatically, using member options for the auto directives, like

.. automodule:: io :members: autodoc needs to import your modules in order to extract the docstrings. Therefore, you must add the appropriate path to sys.path in your conf.py.

Add your code modules to the list as above and then call make html to buildyour documentation and use a web browser look at it.

Make some changes and then run make html again.

If you have to use the sphinx-apidoc then I would suggest:

putting your tutorials in a separate directory as .rst files and
referencing the documentation produced from the API doc from within them plus
referencing the tutorials from within your code comments at the points that they are intended to illustrate.

This should allow you to build your tutorials and API documentation separately depending on where you have made changes and still have linkage between them.

I would strongly recommend the following:

Use a version control system such as mercurial or git so that you can commit your changes before running sphinx,
Put your tutorial .rst files under the VCS for your project but not the generated documentation files.
Put all of the tutorial files under a separate directory with a clear name, e.g. tutorials.
If you are delivering documentation then use a separate repository for your generated documents that is used to store the deliveries.
Always generate documents to a location outside of you code tree.
Put your invocation of sphinx-apidoc into a batch or make file so that you are consistent with the settings that you use.