Documenting Python Programs With Sphinx

Last Updated: April 27, 2023
Originally Published: July 17, 2020

Skill Level: Intermediate

Table Of Contents

• Introduction
• What Is Needed
• Background Information
• Installing Sphinx
• Creating A Sample Python Program
• Creating The Sphinx Configuration Files
• Editing The Configuration File
• Adding Markup Files For All Modules
• Editing The Main Page Markup File
• Running Sphinx
• Viewing The Generated Documentation
• Summary

Introduction

This tutorial will teach you how to use the Sphinx utility to generate program documentation for your Python based projects. A basic understanding of Python programming is expected.

The resources created for this tutorial are available on GitHub for your reference.

This tutorial is provided as a free service to our valued readers. Please help us continue this endeavor by considering a GitHub sponsorship or a PayPal donation.

What Is Needed

• Linux, macOS, Or Windows Based Computer

Background Information

Sphinx is a tool that generates project documentation from a combination of source code and reStructuredText markup files. Although it was originally developed to create documentation for the Python language itself, it can be used with other programming languages by using language specific domains and extensions. It is the predominant project documentation generator used by Python based authors.

Sphinx parses source code annotated with certain commenting styles and special annotations. It will document almost all of the elements and members defined in your program. Markup files are used to include additional information not found in the source code comments.

Sphinx can generate documentation in a variety of formats, e.g. HTML, LaTex, ePub, Texinfo, manual pages, etc. I will be focusing on HTML in this tutorial.

I am using a macOS based computer. If you are using a Linux or Windows computer, the vast majority of this tutorial should still apply, however, some minor changes may be necessary.

If you need assistance with your particular setup, post a question in the comments section below and I, or someone else, can try to help you.

Installing Sphinx

Please see the Sphinx installation page for general installation instructions for your computer platform.

I installed Sphinx from the command line on my Mac via the Homebrew package manager using the following command.

$ brew install sphinx-doc

This installed the sphinx-* executables as a keg only. This means they were not symlinked from /usr/local/Cellar/sphinx-doc/6.2.1/bin into the /usr/local/bin directory (or /opt/homebrew instead of /usr/local on an Apple Silicon based Mac). You could run the executables directly from their Cellar location, but I chose to force the symlinks so that I would not have to prepend the location each time I wanted to execute a Sphinx command.

$ brew link sphinx-doc --force

Test that it is installed correctly by executing the following command within a terminal or command window that will simply print its version number.

$ sphinx-quickstart --version

My version shows the following.

sphinx-quickstart 6.2.1

Creating A Sample Python Program

In order to generate source code based documentation using Sphinx, we first need to have source code for it to use. We will create a main program, named sphinx_example.py, and a module, named sensors.py, that will be used by the program. This program, along with the associated module, are not meant to actually do anything useful. They merely provide an example of how to comment your source code so that it can be properly parsed by the Sphinx utility. It contains various types of elements (e.g. constants, variables, functions, classes, modules, etc.) that are common in Python programs.

Create a project directory named MySphinxExample and go into that directory. Create a src directory under the project directory and go into that directory as well. This is where we will place our source code. Create and save a Python program named sphinx_example.py within this src directory with the code shown below.

#!/usr/bin/env python3

"""An example Python program with Sphinx style comments.

Description
-----------

An example Python program that demonstrates how to use Sphinx (reStructuredText)
style comments.

Libraries/Modules
-----------------

- *time* Standard Library (https://docs.python.org/3/library/time.html)
    - Provides access to the *sleep* function.
- *sensors* Module (local)
    - Provides access to the *Sensor* and *TempSensor* classes.

Notes
-----

- Comments are Sphinx (reStructuredText) compatible.

TODO
----

- None.

Author(s)
---------

- Created by John Woolsey on 05/27/2020.
- Modified by John Woolsey on 04/26/2023.

Copyright (c) 2020 Woolsey Workshop.  All rights reserved.

Members
-------
"""


# Imports
from time import sleep
import sensors


# Global Constants
DEBUG: bool = True
"""The mode of operation; `False` = normal, `True` = debug."""

MIN_BASE: int = 1
"""The minimum number to map."""

MAX_BASE: int = 10
"""The maximum number to map."""

MIN_MAPPED: int = 0
"""The minimum mapped value."""

MAX_MAPPED: int = 255
"""The maximum mapped value."""


# Functions
def map_range(number: float, in_min: float, in_max: float, out_min: float, out_max: float, constrained: bool = True) -> float:
    """Maps a value from one range to another.

    This function takes a value within an input range and maps it to the
    equivalent value within an output range, maintaining the relative position
    of the value within the range.

    :param number:      The value to be mapped.
    :type number:       float
    :param in_min:      The minimum value of the input range.
    :type in_min:       float
    :param in_max:      The maximum value of the input range.
    :type in_max:       float
    :param out_min:     The minimum value of the output range.
    :type out_min:      float
    :param out_max:     The maximum value of the output range.
    :type out_max:      float
    :param constrained: If `True`, the mapped value is constrained to the output
        range; default is `True`.
    :type constrained:  bool

    :return: The mapped value.
    :rtype:  float
    """

    mapped = out_min
    if in_max - in_min != 0:
        mapped = (number - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
    if out_min <= out_max:
        mapped = max(min(mapped, out_max), out_min)
    else:
        mapped = min(max(mapped, out_max), out_min)
    return mapped


def main() -> None:
    """The main program entry."""

    if DEBUG:
        print("Running in DEBUG mode.  Turn off for normal operation.")

    # Map numbers
    for i in range(MIN_BASE, MAX_BASE + 1):
        print(
            f"Base: {i:2d}, Mapped: "
            f"{round(map_range(i, MIN_BASE, MAX_BASE, MIN_MAPPED, MAX_MAPPED)):3d}"
        )
        sleep(0.25)  # wait 250 milliseconds

    # Sensors
    sensor: int = sensors.Sensor("MySensor")
    print(sensor)
    temp_in: int = sensors.TempSensor("Inside")
    print(temp_in)
    temp_out: int = sensors.TempSensor("Outside", "C")
    print(temp_out)


if __name__ == "__main__":  # required for generating Sphinx documentation
    main()

Likewise, create and save a Python module named sensors.py with its code shown below.

"""Defines the sensor classes.

Description
-----------

Defines the base and end user classes for various sensors.

- *Sensor* - The base sensor class.
- *TempSensor* - The temperature sensor class.

Libraries/Modules
-----------------

- *random* Standard Library (https://docs.python.org/3/library/random.html)
    - Provides access to the *randint* function.

Notes
-----

- Comments are Sphinx (reStructuredText) compatible.

TODO
----

- None.

Author(s)
---------

- Created by John Woolsey on 05/27/2020.
- Modified by John Woolsey on 04/21/2023.

Copyright (c) 2020 Woolsey Workshop.  All rights reserved.

Members
-------
"""


import random


class Sensor:
    """The sensor base class.

    Defines the base class utilized by all sensors.
    """

    def __init__(self, name: str) -> None:
        """The Sensor base class initializer.

        :param name: The name of the sensor.
        :type name:  str
        """

        self.name: str = name
        """The name of the sensor."""
        self.value: int = random.randint(0, 50)
        """The value of the sensor."""

    def __str__(self) -> str:
        """Retrieves the sensor's description.

        :return: A description of the sensor.
        :rtype:  str
        """

        return f"The {self.name} sensor has a value of {self.value}."


class TempSensor(Sensor):
    """The temperature sensor class.

    Provides access to the connected temperature sensor.

    Supported units are `"F"` (Fahrenheit), `"C"` (Celsius), and `"K"` (Kelvin).
    """

    def __init__(self, name, unit="F") -> None:
        """The TempSensor class initializer.

        :param name: The name of the temperature sensor.
        :type name:  str
        :param unit: The unit of the temperature sensor with values of
            `"F"`, `"C"`, or `"K"`; defaults to `"F"`.
        :type unit:  str
        """

        super().__init__(name)
        self.unit: str = unit
        """The temperature unit."""

    def __str__(self) -> str:
        """Retrieves the temperature sensor's description.

        :return: A description of the temperature sensor.
        """

        return (
            f"The {self.name} temperature sensor has a value of "
            f"{self.value} degrees {self.unit}."
        )

Sphinx parses the standard Python docstring comments and uses them as summary descriptions within the generated documentation. If a docstring contains reStructuredText based comments, such as that used at the beginning of each source file, or special annotations, such as those used for listing function parameters, those comments will have additional formatting applied in the generated documentation.

I realize the last couple of lines of the first docstring

Members
-------

within the source files look a bit kludgy, but it does make the final generated documentation look a lot nicer by providing an extra separation and title for the automated documentation of the module’s members.

Now run the program to make sure we did not accidentally introduce any errors. The first line of the main program, sphinx_example.py, contains a shebang (#!) statement allowing us to run the program as a command line script. To do so, open a terminal or command window and make the program an executable with

$ chmod a+x sphinx_example.py

and then execute it.

$ ./sphinx_example.py

Alternatively, you could just run it with the Python interpreter.

$ python3 sphinx_example.py

The program output should look similar to the following.

Running in DEBUG mode.  Turn off for normal operation.
Base:  1, Mapped:   0
Base:  2, Mapped:  28
Base:  3, Mapped:  57
Base:  4, Mapped:  85
Base:  5, Mapped: 113
Base:  6, Mapped: 142
Base:  7, Mapped: 170
Base:  8, Mapped: 198
Base:  9, Mapped: 227
Base: 10, Mapped: 255
The MySensor sensor has a value of 49.
The Inside temperature sensor has a value of 22 degrees F.
The Outside temperature sensor has a value of 47 degrees C.

Again, this is just an example program. Don’t pay too much attention to what it is actually doing, just how the comments are formatted for the various types of programming elements or pages.

Creating The Sphinx Configuration Files

Now let’s create a documentation directory where our Sphinx based configuration and generated documentation will be located. Create a directory named sphinx parallel to the src directory. Alternatively, you could name the documentation directory as docs, as many people prefer, but I choose to name it based on the documentation generator in case I choose to use additional generators as well.

In order to effectively parse the source code and generate our project documentation, we first need to configure Sphinx for our project. A sphinx-quickstart command is provided to help us begin that task. We can also tell Sphinx to automatically include documentation from standard Python docstrings, by adding the ––ext-autodoc option to this command. This adds the autodoc Sphinx extension to the configuration file. Other extensions are available that are listed within the Sphinx documentation. Go into the sphinx documentation directory and run the following command.

$ sphinx-quickstart --ext-autodoc

Upon running the above command, Sphinx will ask a few questions to configure your project.

> Separate source and build directories (y/n) [n]:

This is the documentation based source and build directories and is not related to your project’s Python source code itself. Hit Enter to accept the default answer of no.

> Project name:

This will be the title of the project within our generated documentation. Specify a name that makes sense for your project. I chose My Sphinx Example Project.

> Author name(s):

I entered John Woolsey for my name.

> Project release []:

Enter your project’s version number or just hit Enter for none. I specified 1.0 for mine.

> Project language [en]:

Enter your native language here. A list of supported languages is listed in the Sphinx documentation. I just hit Enter to accept the default of English.

Once the questions have been answered, Sphinx will create your documentation directory structure and populate it with various configuration and markup files. Of particular interest are the conf.py and index.rst files. These are your configuration and top level documentation files respectively.

Editing The Configuration File

The conf.py file is a Python based configuration file that Sphinx uses to configure your project’s documentation generation. Some of the questions we answered when running the sphinx-quickstart command above were added as settings to this file upon creation. We will make further changes in order to generate the resulting documentation to our liking.

Make the highlighted additions and modifications to the conf.py file as shown below.

# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

import os
import sys
sys.path.insert(0, os.path.abspath('../src'))

# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information

project = 'My Sphinx Example Project'
copyright = '2020, Woolsey Workshop'
author = 'John Woolsey'
release = '1.0'

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
    'sphinx.ext.autodoc',
]

templates_path = ['_templates']
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# Disable prepending module names
add_module_names = False

# Sort members by type
autodoc_member_order = 'groupwise'

# Document __init__, __repr__, and __str__ methods
def skip(app, what, name, obj, would_skip, options):
    if name in ("__init__", "__repr__", "__str__"):
        return False
    return would_skip

def setup(app):
    app.connect("autodoc-skip-member", skip)

# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = 'alabaster'
html_static_path = ['_static']

Lines 6-8 specify the path to our project’s source code.

You probably don’t need to, but I changed the copyright setting to properly reflect my organization in line 14.

By default, Sphinx prepends module names to members within the generated documentation and sorts all of those members alphabetically. Lines 28-29 remove the prepended module names. Lines 31-32 groups members by type so that all functions will be listed together, and likewise, all variables will be grouped together. In my opinion, this makes the documentation look a bit cleaner, but it is not necessary.

The skip() and setup() functions shown in lines 34-41 tell Sphinx to include documentation for the __init__(), __repr__(), and __str__() special methods, that are skipped by default. We are currently only using __init__() and __str__() in this example, but I added the last one for completeness.

If you are generating documentation for a program where some of the libraries used by the program are not available, you will also need to include the autodoc_mock_imports option in the General configuration section. This option mocks library modules used by your program that Sphinx was not able to import successfully. This is very handy for CircuitPython programs where the core library modules are often not available on your computer but you still want to generate the documentation. Below is an example used for a CircuitPython program.

# Mock unavailable library modules
autodoc_mock_imports = ["board", "analogio", "digitalio"]

Save your updated conf.py configuration file when you are finished making changes.

Adding Markup Files For All Modules

In addition to the sphinx-quickstart command we used previously to create our configuration setup, Sphinx also provides a sphinx-apidoc helper command that automatically creates markup files for all of our modules. Run the following command

$ sphinx-apidoc -f -o . ../src

to generate the sensors.rst and sphinx_example.rst module specific files along with the general modules.rst file that provides a listing for all of the modules found in our project. The -f command line option forces regeneration of the files if they already exist. The -o option specifies where to place the files; set here to the current directory. The last command line argument specifies where the project’s source files are located.

These generated files (*.rst) are formatted as reStructuredText and also include Sphinx specific instructions for how and what to include in the automatic module documentation. Additional custom documentation can be added to these files, just above or below the Sphinx instructions, that will be displayed on the page for the module. I included all of the module specific information within the source code comments of the modules themselves, so I am not adding any custom documentation here.

Editing The Main Page Markup File

The index.rst file is the markup file, in reStructuredText format along with some Sphinx instructions, representing the main page of the project. It will be the basis for the index.html file in our generated HTML documentation. General project information, not associated with any specific module, should be included in this file.

Make the highlighted additions and modifications to the index.rst file as shown below.

.. My Sphinx Example Project documentation master file, created by
   sphinx-quickstart on Thu Apr 27 11:02:30 2023.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to My Sphinx Example Project's documentation!
=====================================================

Description
-----------

An example Python program demonstrating how to use Sphinx style
(reStructuredText) comments for generating source code documentation with
Sphinx.

Notes
-----

- Add special project notes here that you want to communicate to the user.

Modules
-------

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules


Indices and tables
------------------

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Lines 9-22 provide the general project description and other information.

Line 28 adds a link to the modules.rst file that was generated in the last section.

Line 32 is just a cosmetic change that demotes the Indices and tables title from a main title, like the Welcome message at the top of the file, to be a sub-title consistent with the rest of the sub-sections in the file, like Description.

Save the updated index.rst markup file when you are done making changes.

Running Sphinx

Now that all of the Sphinx configuration and markup files have been created and updated, we can generate the HTML documentation for our Python based project. In the same directory as the Sphinx conf.py file, run the following command.

$ make html

This will utilize the Makefile located in the same directory to generate the documentation. Sphinx will print to the screen the various tasks it is performing while running. It will also print any warnings or errors that occurred during execution. Once complete, the generated HTML documentation will be located in the _build/html directory.

Viewing The Generated Documentation

Load the index.html file located within the _build/html directory into your browser. This is the main project page and displays all of the information, separated by sections, that we specified within the index.rst file. Links to the module specific documentation for the various modules used within our project, i.e. sensors and sphinx_example, are listed in the Modules section. The bottom of the page provides additional index and search capabilities to make it easer to find items within the generated documentation. There is even a Quick search in the navigation area on the left side of the page.

Click on the sphinx_example module link to view the documentation for the main module. At the top of the page, you will see the information we included in the top level docstring of the sphinx_example.py file.

At the bottom of the page is the generated documentation that Sphinx created, from the associated docstrings, for the various members of that module.

Click on the My Sphinx Example Project link at the top left corner of the page to take you back to the main page. Now click the sensors module link and view the documentation generated for the sensors module.

Don’t forget to try out the index and search features on the main page to see how they work.

Summary

In this tutorial, we learned how to generate project documentation from source code and reStructuredText markup files using the Sphinx utility for a Python based project. Generously commenting your code and generating the project documentation is a great way to provide both a high level architectural overview and the low level implementation details of a project. Not only does it provide others the means to more easily understand your code, it can also help the original programmer who hasn’t worked on that code in a while.

We barely touched the surface of all the things you can do with Sphinx. If you are interested in learning more, please see the Sphinx documentation.

The Python project and Sphinx configuration used for this tutorial are available on GitHub.

Thank you for joining me on this journey and I hope you enjoyed the experience. Please feel free to share your thoughts or questions in the comments section below.