Environment and Utilities#

asdf#

We develop against one baseline version of the CPython reference implementation; this is the first version listed in the .tool-versions file, located at the top level of the project repository. However, we also test against successive minor versions beyond this baseline version, as well as equivalent versions for alternative implementations. Those versions are listed after the baseline version in the same file. This file is used by the excellent asdf version manager to determine which versions of Python are considered active for the project. We strongly recommend the use of asdf to manage multiple versions of Python, if you use an operating system that is supported by asdf.

To build the CPython implementations that we support, you may need to install some packages with your OS package manager first:

sudo apt update && sudo apt install libbz2-dev libffi-dev libsqlite3-dev libssl-dev libreadline-dev zlib1g-dev

If you have installed asdf, then you can execute the following commands to ensure that the necessary Python environments are available:

. $HOME/.asdf/asdf.sh  # or equivalent for non-standard installation
asdf plugin add python
asdf install python

† Contemporary Bourne shells include ash, bash, and zsh.

Contrasts to Alternatives#

pyenv#

When one considers all of the other language-specific version managers out there, such as jEnv, rbenv, and tfenv, it becomes clear that having a single version management interface reduces cognitive load on developers, as only one command-line interface needs to be remembered to manage versions. A generalized version manager, such as asdf, can be reused consistently across all projects, regardless of their implementation language, whereas pyenv cannot.

invoke#

Many common development tasks, such as running tests, pushing new commit tags, uploading a new release, etc… can be automated. We use invoke to run such automations. It has various nice features, such as the ability to tee standard output streams, run commands in a pseudo-TTY, manage dependencies between tasks … to name a few. To get started, run:

python3 develop.py bootstrap

This will automatically install invoke to a local cache in the project directory and setup a standard set of virtual environments that can be used for testing.

While you can write python3 develop.py for each task invocation, there exists a shortcut for popular shells:

eval "$(python3 develop.py ease --with-completions)"

By default, this will create a shell function, called devshim, which you can use as a command instead of python3 develop.py.

To use certain development tools that we support, you may need to install some packages with your OS package manager first:

sudo apt update && sudo apt install gpg rustc

Contrasts to Alternatives#

make#

  • The Makefile language, while generally elegant and well-suited to deterministic workflows, such as software maintenance automation, is an additional language which must be remembered.

  • While GNU Make is essentially ubiquitous in the Unix world, it is not available by default on Windows. Moreover, Microsoft Nmake does not have entirely compatible syntax. Dual maintenance of DOS/Windows batch files or Powershell scripts is also undesirable.

  • As Python is a prerequisite for this project and we have the infrastructure to guarantee a particular software environment, we can ensure a specific version of invoke is available. We would have no similar assurance with a system-provided make and cannot provide this command via the Python package ecosystem.

  • We can avoid the use of commands, such as find, which have platform-specific variations, and instead use equivalent standardized functions. An additional benefit is that function invocations are within the same Python interpreter session, whereas command invocations have fork-exec overhead.

  • Separate options can be passed for each of multiple targets to invoke, whereas make only consumes global options and variables.

  • A summary of all available targets/subcommands along with brief descriptions can be listed by invoke, whereas make does not provide such a facility.

EditorConfig#

Most modern code editors support per-file type configuration via EditorConfig. This ensures that project standards for things, such as maximum line length, trailing whitespace, and indentation are enforced without the need for lots of editor-specific configurations to be distributed with the project. We recommend that you install an EditorConfig plugin for your editor of choice, if necessary. We provide an .editorconfig file at the top level of the project repository; this file has configurations relevant to the project.

pre-commit#

As part of the development environment that we provide via Pipenv, there is the pre-commit command. Among other things, this allows you to install Git pre-commit hooks which will perform additional checks, such as TOML and YAML linting, before recording a new commit. These hooks will be installed if you ran python3 develop.py bootstrap. To update them at a later time you can run:

devshim freshen-git-hooks install-git-hooks