Take 35% off Publishing Python Packages by entering fcchillard2 into the discount code box at checkout at manning.com.

Python package builds are the product of coordination between a few different tools driven by a standardized process. One of the biggest choices you have as a package author is which set of tools to use. It can be difficult to assess the nuances of each, especially if you’re new to packaging. Fortunately, tools are standardizing around the same core workflow, so once you learn it you’ve got the agility to switch between tools with minimal effort.

This article covers what you first need to learn about the pieces of the Python build system itself.

To follow along with this article, you will need to install and use build. If you haven’t done this before, see the steps below.

Installing build

build (github.com/pypa/build) is a tool provided by the Python Packaging Authority (PyPA) for building Python packages.

Install build using the following command:

 
 python -m pip install build
  

To verify your configuration, run the following command:

 
 $ python -m build --version
  

In the root directory for your package, start by running build using the following command:

 
 $ python -m build
  

Because your package has no content yet, you should see an error like the following:

 
 ERROR Source /Users/<you>/code/first-python-package does not appear to be a Python project: no pyproject.toml or setup.py
  

The output makes two file suggestions. pyproject.toml is the newer standard file for configuring Python packaging introduced in PEP 518 (https://www.python.org/dev/peps/pep-0518/), and should be preferred unless a third-party tool you want to use is only compatible with setup.py. The file uses TOML (https://toml.io/en/), an INI-like language, to split configuration into relevant sections. Create the pyproject.toml file using the following command to correct the error:

 
 $ touch pyproject.toml
  

Run the build command again. This time the build should run successfully, and you should see a large amount of output with a few notable lines as shown in listing 1. What’s happening here? At a high level, the build command consumes your source code and the metadata you supply, along with some files it generates, to create:

1. A source distribution package. A Python source distribution, or sdist, is a compressed archive file of the source code with a .tgz extension.
2. A binary distribution package. A Python built distribution package is a binary file. The current standard for built distributions is what’s known as a wheel or bdist_wheel, a file with a .whl extension.

Listing 1. The result of building an empty Python package

 
 ...
 Successfully installed setuptools-57.0.0 wheel-0.36.2                       ❶
 ...
 running sdist                                                               ❷
 ...
 warning: sdist: standard file not found:                                    ❸
 should have one of README, README.rst, README.txt, README.md
  
 running check
 warning: check: missing required meta-data: name, url                       ❹
  
 warning: check: missing meta-data: either (author and author_email)         ❺
 or (maintainer and maintainer_email) should be supplied
  
 creating UNKNOWN-0.0.0                                                      ❻
 ...
 Creating tar archive                                                        ❼
 ...
 Successfully installed setuptools-57.0.0 wheel-0.36.2
 ...
 running bdist_wheel                                                         ❽
 ...
 creating '/Users/<you>/code/first-python-package/dist/tmpgdfzly_7/          ❾
 UNKNOWN-0.0.0-py3-none-any.whl' and adding
 'build/bdist.macosx-11.2-x86_64/wheel' to it
  

❶ Setuptools and the wheel package are used for the build backend

❷ The source distribution package is built by the build_sdist hook

❸ The build process expects a README file in one of a few formats

❹ The build process expects a name and a URL for the package

❺ The build process expects an author or maintainer of the package

❻ The package is called UNKNOWN because no name was specified

❼ The source distribution is a compressed archive file

❽ The binary wheel distribution package is built by the build_wheel hook

❾ The binary wheel distribution is a .whl file

Because you haven’t supplied any metadata yet, the build process alerts you to the fact that it’s missing some important information like a README file, the author, and so on. Adding this information is covered later in this chapter.

Notice that the build process installs the setuptools and wheel packages. Setuptools (https://setuptools.readthedocs.io) is a library that was, for a long time, one of the only ways to create Python packages. Now, Setuptools is one of a variety of available build backends for Python package builds.

A build backend does the logistical work of creating package artifacts during the build process, namely through the build_sdist and build_wheel hooks. Setuptools uses the wheel package to build the wheel during the build_wheel step. The build tool uses Setuptools as a build backend by default when you don’t specify one.

The presence of build backends may leave you wondering if there may be build frontends as well. As it turns out, you’ve been using a build frontend already. The build tool is a build frontend!

To recap, you use a build frontend tool like build to trigger a build backend like Setuptools to create package artifacts from your source code and metadata (figure 1).

Figure 1. The Python build system consists of a frontend user interface that integrates with a backend to build package artifacts.

Because the build process creates package artifacts, you can now check the effect of running the build. List the contents of the root directory for your package now. You should see the following:

 
 $ ls -a1 $HOME/code/first-python-package/
 .
 ..
 .venv/
 UNKNOWN.egg-info/
 build/
 dist/
 pyproject.toml
  

The UKNOWN.egg-info/ and build/ directories are intermediate artifacts. List the contents of the dist/ directory, where you should see the source and binary wheel package files:

 
 $ ls -a1 $HOME/code/first-python-package/dist/
 UNKNOWN-0.0.0-py3-none-any.whl
 UNKNOWN-0.0.0.tar.gz
  

Other build system tools

As I mentioned earlier, other options exist for both build frontends and backends. Some packages provide both a frontend and a backend, but I like build and Setuptools.

If you want to explore some alternative build tools, check out poetry (https://python-poetry.org/) and flit (https://flit.readthedocs.io). Each build system makes different trade-offs between ease of configuration, capability, and user interface. As an example, flit and poetry are geared toward pure-Python packages whereas Setuptools can support extensions in other languages.

You can switch to another build system in a couple of steps:

1. Install the new build frontend package
2. Update pyproject.toml to specify the new build backend and its requirements
3. Move the metadata about the package to the location expected by the new build backend

Recall that build used Setuptools as the fallback build backend because you didn’t specify one. You can specify Setuptools as the build backend for your package by adding the lines in listing 2 to pyproject.toml. These lines specify the following:

1. build-system — This section describes the package build system.
2. requires — These are a list of dependencies, as strings, which must be installed for the build system to work. A Setuptools build system needs setuptools and wheel as you saw earlier in this chapter.
3. build-backend — This identifies the entrypoint to the build backend object, using the dotted path as a string. The Setuptools build backend object is available at setuptools.build_meta.

These represent the complete configuration you need to specify the build backend.

Listing 2. A build system backend specification to use Setuptools

 
 [build-system]                                         ❶
 requires = ["setuptools", "wheel"]                     ❷
 build-backend = "setuptools.build_meta"                ❸
  

❶ Opens a new TOML section

❷ List of package names as strings

❸ Dotted path to the object as a string

Once you’ve added the build system information, run the build again. Nothing should change in the output: you’ve just locked in Setuptools as the explicit backend instead of letting build fall back to it as a default. Now that you’ve got a handle on the Python package build system, you need to add some metadata about your package. You can read all about it in the book here.