Software Development

Documenting Python Programs With Sphinx

Sphinx Graphic
Written by John Woolsey

Skill Level: Intermediate

Table Of Contents

Introduction

This tutorial will teach you how to use the Sphinx tool to generate program documentation for your Python based project. A basic understanding of Python programming is expected.

The resources created for this tutorial are available on GitHub for your reference.

What Is Needed

  • Linux, macOS, Or Windows Based Computer

Background Information

Sphinx is a tool that generates project documentation from a combination of source code and reStructuredText markup files. Although it was originally developed to create documentation for the Python language itself, it can be used with other programming languages by using language specific domains and extensions. It is the predominant project documentation generator used by Python based authors.

Sphinx parses source code annotated with certain commenting styles and special annotations. It will document almost all of the elements and members defined in your program. Markup files are used to include additional information not found in the source code comments.

Sphinx can generate documentation in a variety of formats, e.g. HTML, LaTex, ePub, Texinfo, manual pages, etc. I will be focusing on HTML in this tutorial.

I am using a macOS based computer. If you are using a Linux or Windows computer, the vast majority of this tutorial should still apply, however, some minor changes may be necessary.

If you need assistance with your particular setup, post a question in the comments section below and I, or someone else, can try to help you.

Installing Sphinx

Please see the Sphinx installation page for general installation instructions for your computer platform.

I installed Sphinx from the command line on my Mac via the Homebrew package manager using the following command.

$ brew install sphinx-doc

This installed the sphinx-* executables as a keg only. This means they were not symlinked from /usr/local/Cellar/sphinx-doc/3.1.1/bin into the /usr/local/bin directory. You could run the executables directly from their Cellar location, but I chose to force the symlinks so that I would not have to prepend the location each time I wanted to execute a Sphinx command.

$ brew link sphinx-doc --force

Test that it is installed correctly by executing the following command within a terminal or command window that will simply print its version number.

$ sphinx-quickstart --version

My version shows the following.

sphinx-quickstart 3.1.1

Creating A Sample Python Program

In order to generate source code based documentation using Sphinx, we first need to have source code for it to use. We will create a main program, named sphinx_example.py, and a module, named sensors.py, that will be used by the program. This program, along with the associated module, are not meant to actually do anything useful. They merely provide an example of how to comment your source code so that it can be properly parsed by the Sphinx utility. It contains various types of elements (e.g. constants, variables, functions, classes, modules, etc.) that are common in Python programs.

Create a project directory named MySphinxExample and go into that directory. Create a src directory under the project directory and go into that directory as well. This is where we will place our source code. Create and save a Python program named sphinx_example.py within this src directory with the code shown below.

#!/usr/bin/env python3

"""Example Python program with Sphinx style comments.

Description
-----------

Example Python program with Sphinx style (reStructuredText) comments.

Libraries/Modules
-----------------

- time standard library (https://docs.python.org/3/library/time.html)
    - Access to sleep function.
- sensors module (local)
    - Access to Sensor and TempSensor classes.

Notes
-----

- Comments are Sphinx (reStructuredText) compatible.

TODO
----

- None.

Author(s)
---------

- Created by John Woolsey on 05/27/2020.
- Modified by John Woolsey on 07/02/2020.

Copyright (c) 2020 Woolsey Workshop.  All rights reserved.

Members
-------
"""


# Imports
from time import sleep
import sensors


# Global Constants
DEBUG = 1
"""The mode of operation; 0 = normal, 1 = debug."""
MIN_BASE = 1
"""The minimum number to map."""
MAX_BASE = 10
"""The maximum number to map."""
MIN_MAPPED = 0
"""The minimum mapped value."""
MAX_MAPPED = 255
"""The maximum mapped value."""


# Functions
def init():
    """Initializes the program."""

    if DEBUG:
        print("Initializing program.")


def map_range(number, in_min, in_max, out_min, out_max):
    """Maps a number from one range to another.

    :param number:  The input number to map.
    :param in_min:  The minimum value of an input number.
    :param in_max:  The maximum value of an input number.
    :param out_min: The minimum value of an output number.
    :param out_max: The maximum value of an output number.

    :return: The mapped number.
    """

    mapped = (number - in_min) * (out_max - out_min) / (in_max - in_min) + out_min
    if out_min <= out_max:
        return max(min(mapped, out_max), out_min)
    return min(max(mapped, out_max), out_min)


def main():
    """Main program entry."""

    init()  # program initialization

    # Map numbers
    for i in range(MIN_BASE, MAX_BASE + 1):
        print(
            f"Base: {i:2d}, Mapped: "
            f"{map_range(i, MIN_BASE, MAX_BASE, MIN_MAPPED, MAX_MAPPED):5.1f}"
        )
        sleep(0.25)  # wait 250 milliseconds

    # Sensors
    sensor = sensors.Sensor("MySensor")
    print(sensor)
    temp_in = sensors.TempSensor("Inside")
    print(temp_in)
    temp_out = sensors.TempSensor("Outside", "C")
    print(temp_out)
    temp_out.set_unit("K")
    print(temp_out)


if __name__ == "__main__":
    main()

Likewise, create and save a Python module named sensors.py with its code shown below.

"""Defines the sensor classes.

Description
-----------

Defines the base and end user classes for various sensors.

- Sensor (base class)
- TempSensor

Libraries/Modules
-----------------

- random standard library (https://docs.python.org/3/library/random.html)
    - Access to randint function.

Notes
-----

- Comments are Sphinx (reStructuredText) compatible.

TODO
----

- None.

Author(s)
---------

- Created by John Woolsey on 05/27/2020.
- Modified by John Woolsey on 07/02/2020.

Copyright (c) 2020 Woolsey Workshop.  All rights reserved.

Members
-------
"""


import random


class Sensor:
    """The sensor base class.

    Defines the base class utilized by all sensors.
    """

    def __init__(self, name):
        """The Sensor base class initializer.

        :param name: The name of the sensor.

        :return: An instance of the Sensor class initialized with the specified name.
        """

        self.name = name
        """The name of the sensor."""
        self.value = random.randint(0, 50)
        """The value of the sensor."""

    def __str__(self):
        """Retrieves the sensor's description.

        :return: A description of the sensor.
        """

        return f"The {self.name} sensor has a value of {self.value}."


class TempSensor(Sensor):
    """The temperature sensor class.

    Provides access to the connected temperature sensor.

    Supported units are "F" (Fahrenheit), "C" (Celsius), and "K" (Kelvin)
    with "F" being the default unit.
    """

    def __init__(self, name, unit="F"):
        """The TempSensor class initializer.

        :param name: The name of the temperature sensor.
        :param unit: The unit of the temperature sensor, defaults to "F".

        :return: An instance of the TempSensor class initialized with the specified name and unit.
        """

        super().__init__(name)
        self.unit = unit
        """The temperature unit."""

    def __str__(self):
        """Retrieves the temperature sensor's description.

        :return: A description of the temperature sensor.
        """

        return (
            f"The {self.name} temperature sensor has a value of "
            f"{self.value} degrees {self.unit}."
        )

    def set_unit(self, unit):
        """Sets the temperature unit.

        :param unit: The temperature unit ("F", "C", or "K"),
            defaults to "F" if a valid unit is not provided.
        """

        if unit in ("C", "K"):
            self.unit = unit
        else:
            self.unit = "F"

Sphinx parses the standard Python docstring comments and uses them as summary descriptions within the generated documentation. If a docstring contains reStructuredText based comments, such as that used at the beginning of each source file, or special annotations, such as those used for listing function parameters, those comments will have additional formatting applied in the generated documentation.

I realize the last couple of lines of the first docstring

Members
-------

within the source files look a bit kludgy, but it does make the final generated documentation look a lot nicer by providing an extra separation and title for the automated documentation of the module’s members.

Now run the program to make sure we did not accidentally introduce any errors. The first line of the main program, sphinx_example.py, contains a shebang (#!) statement allowing us to run the program as a command line script. To do so, open a terminal or command window and make the program an executable with

$ chmod a+x sphinx_example.py

and then execute it.

$ ./sphinx_example.py

Alternatively, you could just run it with the Python interpreter.

$ python3 sphinx_example.py

The program output should look similar to the following.

Initializing program.
Base:  1, Mapped:   0.0
Base:  2, Mapped:  28.3
Base:  3, Mapped:  56.7
Base:  4, Mapped:  85.0
Base:  5, Mapped: 113.3
Base:  6, Mapped: 141.7
Base:  7, Mapped: 170.0
Base:  8, Mapped: 198.3
Base:  9, Mapped: 226.7
Base: 10, Mapped: 255.0
The MySensor sensor has a value of 25.
The Inside temperature sensor has a value of 29 degrees F.
The Outside temperature sensor has a value of 48 degrees C.
The Outside temperature sensor has a value of 48 degrees K.

Again, this is just an example program. Don’t pay too much attention to what it is actually doing, just how the comments are formatted for the various types of programming elements or pages.

Creating The Sphinx Configuration Files

Now let’s create a documentation directory where our Sphinx based configuration and generated documentation will be located. Create a directory named sphinx parallel to the src directory. Alternatively, you could name the documentation directory as docs, as many people prefer, but I choose to name it based on the documentation generator in case I choose to use additional generators as well.

In order to effectively parse the source code and generate our project documentation, we first need to configure Sphinx for our project. A sphinx-quickstart command is provided to help us begin that task. We can also tell Sphinx to automatically include documentation from standard Python docstrings, by adding the –ext-autodoc option to this command. This adds the autodoc Sphinx extension to the configuration file. Other extensions are available that are listed within the Sphinx documentation. Go into the sphinx documentation directory and run the following command.

$ sphinx-quickstart --ext-autodoc

Upon running the above command, Sphinx will ask a few questions to configure your project.

> Separate source and build directories (y/n) [n]:

This is the documentation based source and build directories and is not related to your project’s Python source code itself. Hit Enter to accept the default answer of no.

> Project name:

This will be the title of the project within our generated documentation. Specify a name that makes sense for your project. I chose My Sphinx Example Project.

> Author name(s):

I entered John Woolsey for my name.

> Project release []:

Enter your project’s version number or just hit Enter for none. I specified 1.0 for mine.

> Project language [en]:

Enter your native language here. A list of supported languages is listed in the Sphinx documentation. I just hit Enter to accept the default of English.

Once the questions have been entered, Sphinx will create your documentation directory structure and populate it with various configuration and markup files. Of particular interest are the conf.py and index.rst files. These are your configuration and top level documentation files respectively.

Editing The Configuration File

The conf.py file is a Python based configuration file that Sphinx uses to configure your project’s documentation generation. Some of the questions we answered when running the sphinx-quickstart command above were added as settings to this file upon creation. We will make further changes in order to generate the resulting documentation to our liking.

Make the highlighted additions and modifications to the conf.py file as shown below.

# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
sys.path.insert(0, os.path.abspath('../src'))


# -- Project information -----------------------------------------------------

project = 'My Sphinx Example Project'
copyright = '2020, Woolsey Workshop'
author = 'John Woolsey'

# The full version, including alpha/beta/rc tags
release = '1.0'


# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
    'sphinx.ext.autodoc',
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# Turn off prepending module names
add_module_names = False

# Sort members by type
autodoc_member_order = 'groupwise'

# Document __init__, __repr__, and __str__ methods
def skip(app, what, name, obj, would_skip, options):
    if name in ("__init__", "__repr__", "__str__"):
        return False
    return would_skip

def setup(app):
    app.connect("autodoc-skip-member", skip)


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']


# -- Extension configuration -------------------------------------------------

Lines 13-15 specify the path to our project’s source code.

You probably don’t need to, but I changed the copyright setting to properly reflect my organization in line 21.

By default, Sphinx prepends module names to members within the generated documentation and sorts all of those members alphabetically. Lines 45-46 remove the prepended module names. Lines 48-49 groups members by type so that all functions will be listed together, and likewise, all variables will be grouped together. In my opinion, this makes the documentation look a bit cleaner, but it is not necessary.

The skip() and setup() functions shown in lines 51-58 tell Sphinx to include documentation for the __init__(), __repr__(), and __str__() special methods, that are skipped by default. We are currently only using __init__() in this example, but I added the others for completeness.

Save your updated conf.py configuration file when you are finished making changes.

Adding Markup Files For All Modules

In addition to the sphinx-quickstart command we used previously to create our configuration setup, Sphinx also provides a sphinx-apidoc helper command that automatically creates markup files for all of our modules. Run the following command

$ sphinx-apidoc -f -o . ../src

to generate the sensors.rst and sphinx_example.rst module specific files along with the general modules.rst file that provides a listing for all of the modules found in our project. The -f command line option forces regeneration of the files if they already exist. The -o option specifies where to place the files; set here to the current directory. The last command line argument specifies where the project’s source files are located.

These generated files (*.rst) are formatted as reStructuredText and also include Sphinx specific instructions for how and what to include in the automatic module documentation. Additional custom documentation can be added to these files, just above or below the Sphinx instructions, that will be displayed on the page for the module. I included all of the module specific information within the source code comments of the modules themselves, so I am not adding any custom documentation here.

Editing The Main Page Markup File

The index.rst file is the markup file, in reStructuredText format along with some Sphinx instructions, representing the main page of the project. It will be the basis for the index.html file in our generated HTML documentation. General project information, not associated with any specific module, should be included in this file.

Make the highlighted additions and modifications to the index.rst file as shown below.

.. My Sphinx Example Project documentation master file, created by
   sphinx-quickstart on Thu Jun 11 16:43:50 2020.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to My Sphinx Example Project's documentation!
=====================================================

Description
-----------

An example Python program demonstrating how to use Sphinx style
(reStructuredText) comments for generating source code documentation with
Sphinx.

Notes
-----

- Add special project notes here that you want to communicate to the user.

Modules
-------

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules

Indices and tables
------------------

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Lines 9-22 provide the general project description and other information.

Line 28 adds a link to the modules.rst file that was generated in the last section.

Line 31 is just a cosmetic change that demotes the Indices and tables title from a main title, like the Welcome message at the top of the file, to be a sub-title consistent with the rest of the sub-sections in the file, like Description.

Save the updated index.rst markup file when you are done making changes.

Running Sphinx

Now that all of the Sphinx configuration and markup files have been created and updated, we can generate the HTML documentation for our Python based project. In the same directory as the Sphinx conf.py file, run the following command.

$ make html

This will utilize the Makefile located in the same directory to generate the documentation. Sphinx will print to the screen the various tasks it is performing while running. It will also print any warnings or errors that occurred during execution. Once complete, the generated HTML documentation will be located in the _build/html directory.

Viewing The Generated Documentation

Load the index.html file located within the _build/html directory into your browser. This is the main project page and displays all of the information, separated by sections, that we specified within the index.rst file. Links to the module specific documentation for the various modules used within our project, i.e. sensors and sphinx_example, are listed in the Modules section. The bottom of the page provides additional index and search capabilities to make it easer to find items within the generated documentation. There is even a Quick search in the navigation area on the left side of the page.

Sphinx Main Page
Sphinx Main Page

Click on the sphinx_example module link to view the documentation for the main module. At the top of the page, you will see the information we included in the top level docstring of the sphinx_example.py file.

Top Of Sphinx Main Module (sphinx_example) Page
Top Of Sphinx Main Module (sphinx_example) Page

At the bottom of the page is the generated documentation that Sphinx created, from the associated docstrings, for the various members of that module.

Bottom Of Sphinx Main Module (sphinx_example) Page
Bottom Of Sphinx Main Module (sphinx_example) Page

Click on the My Sphinx Example Project link at the top left corner of the page to take you back to the main page. Now click the sensors module link and view the documentation generated for the sensors module.

Don’t forget to try out the index and search features on the main page to see how they work.

Summary

In this tutorial, we learned how to generate project documentation from source code and reStructuredText markup files using the Sphinx utility for a Python based project. Generously commenting your code and generating the project documentation is a great way to provide both a high level architectural overview and the low level implementation details of a project. Not only does it provide others the means to more easily understand your code, it can also help the original programmer who hasn’t worked on that code in a while.

We barely touched the surface of all the things you can do with Sphinx. If you are interested in learning more, please see the Sphinx documentation.

The Python project and Sphinx configuration used for this tutorial are available on GitHub.

Thank you for joining me in this journey and I hope you enjoyed the experience. Please feel free to share your thoughts in the comments section below.

About the author

John Woolsey

John is an electrical engineer who loves science, math, and technology and teaching it to others even more.
 
He knew he wanted to work with electronics from an early age, building his first robot when he was in 8th grade. His first computer was a Timex/Sinclair 2068 followed by the Tandy 1000 TL (aka really old stuff).
 
He put himself through college (The University of Texas at Austin) by working at Motorola where he worked for many years afterward in the Semiconductor Products Sector in Research and Development.
 
John started developing mobile app software in 2010 for himself and for other companies. He has also taught programming to kids for summer school and enjoyed years of judging kids science projects at the Austin Energy Regional Science Festival.
 
Electronics, software, and teaching all culminate in his new venture to learn, make, and teach others via the Woolsey Workshop website.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.