How to pre-process source files while a Sphinx run?

Question

I have set up a Sphinx documentation for my project and would like to extract doc strings for the source files and embed them into the final documentation. Unfortunately, the source file's language (VHDL) is not supported by Sphinx. There seems to be no Sphinx domain for VHDL.

So my ideas is as follows:

• Hook into the Sphinx run and execute some Python code before Sphinx
• The Python codes extracts text blocks from each source file (the top-most multi-line comment block) and assembles one reST file per source file, consisting of this comment block and some other reST markup.
• All source files are listed in an index.rst, to generate the apropriate .. toctree:: directive.
• The text extraction and transformation is done recursively per source code directory.

So the main question is: How to hook into Spinx?

Or should I just import and run my own configuration in conf.py?

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
from my_preprocessor import my_proc
proc = my_proc()
proc.run()
#
# Test documentation build configuration file, created by
# sphinx-quickstart on Tue May 24 11:28:20 2016.
# ....

I can't modify the build process files: Makefile and make.bat, because the real build process runs on ReadTheDocs.org. RTDs only executes conf.py.

Answer 1

As noted in my previous comments and mertyildiran's answer, the official way to hook into Sphinx for a language would be to create an extension to implement a new domain for VHDL.

This has already been done for many other languages - e.g. Erlang, PHP, CoffeeScript - and APIs - e.g. HTTP REST - just to name a few from sphinx-contrib. However, that is going to take a lot of time, which you don't have... You are therefore left with an option to do some quick parsing yourself and then hooking that into your Sphinx build somehow.

Since you are bypassing the official hooks, this question becomes "how do I run my own code inside a Sphinx build?" For which, I would recommend that you simply follow the guidance for a local extension - i.e. put it in a separate directory, add that to your path, then import and invoke it. As noted in the docs:

The configuration file is executed as Python code at build time (using execfile(), and with the current directory set to its containing directory), and therefore can execute arbitrarily complex code. Sphinx then reads simple names from the file’s namespace as its configuration.

As a final though, this opens up the options to use 3rd party packages like pyVhdl2Sch (with a nod again to mertyildiran's answer) to create some schematic and then maybe write your static rst files around it to explain the schematic.