Contribution

Contribution to this project is welcome! Here are some ways that you can help:

  • Filing issues for bug reports, feature requests, and other relevant discussion.

  • Helping out with the development of the source code and documentation.

Please do note that all interactions about this project on media under its control are subject to its Code of Conduct.

Code of Conduct

Please take a look at:

as well as note the following bullet points from the Rust Code of Conduct:

• Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.

• Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.

• Likewise any spamming, trolling, flaming, baiting or other attention-stealing behavior is not welcome.

Note

Please do not contact the authors of the above documents for anything related to this project.

Please act in the spirit of respect for others, whether you are asking questions, providing feedback, or answering questions. Please remember that volunteerism powers much of the open source available for you to use.

Also, no need to create noise with “+1” or “me too” comments in issue tracker threads. Please only add substantive comments that will further advance the understanding of an issue. If you wish to upvote a comment, then please use an appropriate reacji on that comment.

Development

Environment

asdf

We develop against one baseline version of the CPython reference implementation; this is the first version listed in the .tool-versions file, located at the top level of the project repository. However, we also test against successive minor versions beyond this baseline version, as well as equivalent versions for alternative implementations. Those versions are listed after the baseline version in the same file. This file is used by the excellent asdf version manager to determine which versions of Python are considered active for the project. We strongly recommend the use of asdf to manage multiple versions of Python.

To build the CPython implementations that we support, you may need to install some packages with your OS package manager first:

sudo apt update && sudo apt install libbz2-dev libffi-dev libsqlite3-dev libssl-dev libreadline-dev zlib1g-dev

If you have installed asdf, then you can execute the following commands to ensure that the necessary Python environments are available:

. $HOME/.asdf/asdf.sh  # or equivalent for non-standard installation
asdf plugin add python
asdf install python

† Contemporary Bourne shells include ash, bash, and zsh.

Why not use pyenv ?

When one considers all of the other language-specific version managers out there, such as jEnv, rbenv, and tfenv, it becomes clear that having a single version management interface reduces cognitive load on developers, as they only need to remember one command-line interface and one file format for managed versions. And, while this project may only be Python-specific, the practice of using a more generalized version manager may cross to other projects.

pipenv

If you have done any significant amount of Python development or have tried using Python on a machine where you do not have administrative privileges, then you have probably encountered the concept of virtual environments to provide sandboxes into which you can install packages. Traditionally, you would have to create such virtual environments, “activate” them, and then install packages into them. The creation of such an environment may involve a tool, like virtualenv, or the venv module that comes with all recent versions of Python 3. Activation of such an environment typically involves using a shell script. And, installing into an activated environment is commonly done by running pip from within that environment. All of that is a lot of work if you simply want to play in a sandbox - and you do not even get reproducible installations for your hassle.

The pipenv tool is a very respectable solution to the above problems. It creates a virtual environment and can transparently install packages into it. Installed development dependencies and runtime dependencies are both tracked as separate categories within a Pipfile. This grants portability and reproducibility of development environments and removes the need to maintain various requirements.txt files. Furthermore, the pipenv shell and pipenv run commands are very useful for iterating on dependency changes during development. For these reasons, we strongly recommend installing and using this tool.

We provide a Pipfile in the top level of the project repository. After you clone the project and have installed pipenv, just run:

pipenv sync --dev

to prepare a virtual environment for development.

To use certain development tools that we support, you may need to install some packages with your OS package manager first:

sudo apt update && sudo apt install gpg

EditorConfig

Most modern code editors support per-file type configuration via EditorConfig. This ensures that project standards for things, such as maximum line length, trailing whitespace, and indentation are enforced without the need for lots of editor-specific configurations to be distributed with the project. We recommend that you install an EditorConfig plugin for your editor of choice, if necessary. We provide an .editorconfig file at the top level of the project repository; this file has configurations relevant to the project.

Commit Signatures

All commits to the project must be signed with a valid GPG/PGP or S/MIME secret key. You can use GNU Privacy Guard or a similar tool to generate a signing key if you do not already have one. And, you can likewise use such a tool to sign your commits. Github has a good guide on the following topics:

If you do not wish to expose a personal email address in association with a signing key, you can use the no-reply email address associated with your Github account instead.

In addition to registering the signature verification public key, which corresponds to your secret signing key, with Github, you may also publish the signature verification public key to well-known public key servers, such as:

Common Tasks

During development, some tasks are repeated again and again. And some tasks are dependent upon other tasks. Traditionally, the make command or a script or batch file would be used to automate such tasks. However, in to provide maximum portability and to reduce the number of languages that a developer needs to remember, we use a Python-native solution: invoke. The invoke command is available as part of the virtual environment maintained by Pipenv for this project.

Instead of defining tasks in a Makefile, we instead define them as Python code in tasks. To see a summary of the available tasks, you can execute:

pipenv run invoke --list

We recommend the use of invoke rather than running tools directly, since it performs environment sanitization and other frequently-overlooked tasks automatically, freeing you from fighting weird cache effects or various oversights, such as validating URLs in documentation or linting tests.

Documentation

Documentation artifacts are built by Sphinx using an extended form of the reStructuredText (rst) plaintext markup language. The sources for the documentation are under sources/sphinx; the artifacts can be produced by:

pipenv run invoke make-html

Some useful references for editing the sources are:

Why not use a flavor of Markdown instead?

While Markdown is almost certainly better known than reStructuredText, Sphinx provides first-class support for the latter; Sphinx is a very powerful documentation generator and it is also the de facto standard within the Python community. Furthermore, reStructuredText is very well-suited to technical documentation, whereas Markdown is more general purpose. Although Markdown may have a somewhat simpler syntax than reStructuredText, multiple, competing flavors of the language exist, which partially negates the simplicity. In light of these factors, we have standardized on 100% reStructuredText for all project documentation.

Why not use Markdown for some things at least?

Context-switching between multiple languages adds cognitive load and increases the chance of misapplied syntax.

Linting

Our primary Python linter is Pylint. Note that we use this as an actual static analysis tool and not merely a style checker. You can lint the sources by:

pipenv run invoke lint

Why not use Flake8 ?

Flake8 does not catch nearly as many issues as Pylint does and tends to focus on code style rather than finding actual bugs and smells. Because of Pylint’s greater sensitivity, it may have a reputation as “not working” out of the box, which may be why some people prefer Flake8. However, we prefer the fact that Pylint finds more bugs and smells and have already taken the time to tune it for the project’s needs.

Testing

Our primary Python tester is Pytest. We use this to run unit tests and doctests. In general, we prefer to write test functions rather than heavyweight unittest classes. Also, we make extensive use of parametrized tests and property-based testing. To run the test suite in the current virtual environment maintained by pipenv, you can execute:

pipenv run invoke test

To ensure that the test suite passes on all Python implementations, which are supported by the project, you can execute:

pipenv run invoke test-all-versions

This may take a longer time to complete. Under the covers, it is using tox, which runs the test suite in an array of isolated virtual environments. The virtual environments are instantiated via the tox-asdf plugin, thus integrating with the Python implementations already available via asdf.

Internal API Documentation

Package foundation. Mostly internal classes and functions.

class lockup.base.Class(name, bases, namespace)

Bases: type

Produces classes which have immutable attributes.

Non-public attributes of each class are concealed from dir().

Note

Only class attributes are immutable. Instances of immutable classes will have mutable attributes without additional intervention beyond the scope of this package.

mro()

Return a type’s method resolution order.

class lockup.base.NamespaceClass(name, bases, namespace)

Bases: lockup.Class

Produces namespace classes which have immutable attributes.

Each produced namespace is a unique class, which cannot be instantiated.

Non-public attributes of each class are concealed from dir().

Warning

Although most descriptor attributes will be inert on a class, types.DynamicClassAttribute() and similar, may trigger attribute errors when accessed. However, these are a fairly rare case and are probably not needed on namespaces, in general.

mro()

Return a type’s method resolution order.

lockup.base.create_namespace(**nomargs)

Creates immutable namespaces from nominative arguments.

lockup.base.DictionaryProxy

alias of mappingproxy

exception lockup.base.Exception0(*things, tags=None, **sundry)

Bases: BaseException

Base for all exceptions in the package.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.FugitiveException(supplement=None)

Bases: lockup.exceptions.InvalidState, RuntimeError

Alert about fugitive exception intercepted at API boundary.

An fugitive exception is one which is not intended to be reported across the package API boundary. Fugitive exceptions include Python built-ins, such as IndexError.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.ImpermissibleAttributeOperation(*things, tags=None, **sundry)

Bases: lockup.exceptions.ImpermissibleOperation, AttributeError

Complaint about impermissible attribute operation.

Cannot use ImpermissibleOperation because some packages, such as Sphinx Autodoc, expect an AttributeError.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.ImpermissibleOperation(*things, tags=None, **sundry)

Bases: lockup.exceptions.InvalidOperation, TypeError

Complaint about impermissible operation.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.InaccessibleAttribute(*things, tags=None, **sundry)

Bases: lockup.exceptions.InaccessibleEntity, AttributeError

Complaint about attempt to retrieve inaccessible attribute.

Cannot use InaccessibleEntity because some Python internals expect an AttributeError.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.InaccessibleEntity(*things, tags=None, **sundry)

Bases: lockup.exceptions.InvalidOperation

Complaint about attempt to retrieve inaccessible entity.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.IncorrectData(*things, tags=None, **sundry)

Bases: lockup.exceptions.InvalidOperation, TypeError, ValueError

Complaint about incorrect data for invocation or operation.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.InvalidOperation(*things, tags=None, **sundry)

Bases: lockup.exceptions.Exception0, Exception

Complaint about invalid operation.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception lockup.base.InvalidState(supplement=None)

Bases: lockup.exceptions.Exception0, Exception

Alert about invalid internal state in the package.

Owner of problem: maintainers of this package.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

lockup.base.calculate_class_label(classes, attribute_label=None)

Produces human-comprehensible label for class or tuple of classes.

Each provided class may be a class object or namespace dictionary that is present during class creation.

lockup.base.calculate_instance_label(object_, attribute_label=None)

Produces human-comprehensible label for instance of class.

lockup.base.calculate_invocable_label(invocable)

Produces human-comprehensible label for invocable object.

An invocable object may be a function, bound method, class, or invocable instance of a class.

lockup.base.calculate_label(object_, attribute_label=None)

Produces human-comprehensible label, based on classification.

lockup.base.calculate_module_label(module, attribute_label=None)

Produces human-comprehensible label for module.

lockup.base.create_argument_validation_exception(name, invocation, expectation_label)

Creates error with context about invalid argument.

lockup.base.create_attribute_immutability_exception(name, context, action='assign')

Creates error with context about immutable attribute.

lockup.base.create_attribute_indelibility_exception(name, context)

Creates error with context about indelible attribute.

lockup.base.create_attribute_nonexistence_exception(name, context)

Creates error with context about nonexistent attribute.

lockup.base.create_class_attribute_rejection_exception(name, class_)

Creates error with context about class attribute rejection.

lockup.base.create_impermissible_instantiation_exception(class_)

Creates error with context about impermissible instantiation.

lockup.base.create_invocation_validation_exception(invocation, cause)

Creates error with context about invalid invocation.

lockup.base.intercept(invocation)

Decorator to intercept fugitive exceptions.

Fugitive exceptions are ones which are not expected to cross an API boundary.

lockup.base.is_operational_name(name)

Returns True if name is operational.

An operational name begins and ends with a double underscore (__).

lockup.base.is_public_name(name)

Returns True if name is user-public.

A user-public name does not begin with an underscore (_).

lockup.base.is_public_or_operational_name(name)

Returns True if name is user-public or operational.

See is_public_name() and is_operational_name() for details.

lockup.base.is_python_identifier(name)

Is object a legal Python identifier? Excludes Python keywords.

lockup.base.module_qualify_class_name(class_)

Concatenates module name and qualified name of class.

Also supports class namespace dictionaries.

lockup.base.select_public_attributes(class_, object_, *, includes=(), excludes=())

Selects all attributes with user-public names on object.

Can optionally include specific attributes that would not be selected under normal operation and can exclude specific attributes that would selected under normal operation.

lockup.base.validate_argument_invocability(argument, name, invocation)

Validates argument as an invocable object, such as a function.

lockup.base.validate_attribute_existence(name, context)

Validates attribute exists on context object.

lockup.base.validate_attribute_name(name, context)

Validates attribute name as Python identifier.

Principles and Advices

Assertions and Exceptions

  • Do not use assert statements except in unit tests. Assertions are not triggered in optimized compilations. Raise a InvalidState exception to report invalid internal state.

  • Do not report incorrect invocations or invalid argument data as invalid internal state. I.e., raise an exception in the InvalidOperation family rather than an exception in the InvalidState family.

  • Never raise a non-package exception from inside the package. Part of the API contract is that only exceptions in the InvalidOperation and InvalidState families are permitted to cross the API boundary.

  • Always consider whether an invocation can leak Python built-in exceptions across the API boundary and explicitly manage those which can.

Code Style and Legibility

Code style can be a surprisingly contentious issue. This project does not run any style checker or code reformatter, such as autopep8 or black. However, it is expected that patches will follow a style similar to what is already present. Some basic guidance is enforced via EditorConfig, but mostly it is up to the patch submitter to try to match the existing style. Though it is not perfect and good judgment will still need to be exercised, you can execute the following on an uncommitted patch to receive additional hints:

pipenv run invoke check-code-style

Unwitting “offenses” in new code can be readily forgiven, but patches which contain reformatting noise against existing code will almost certainly be rejected. Please digest this Stack Exchange discussion on reformatting.

Some tips:

  • Horizontal space between identifiers and literals improves legibility. Prior to the introduction of spaces between words around fourteen centuries ago, readers had to either parse words from scriptio continua or with the use of a word divider symbol, such as the interpunct. Reading identifiers from code, which are separated only by delimiters, is equivalent to reading words separated by interpuncts, except that the cognitive load is even higher due to the variety of delimiters (parentheses, commas, colons, etc…). Using horizontal space, an innovation from nearly 1.5 millennia ago, can reduce this cognitive load.

  • Try to keep function bodies under twenty (20) lines, no more than thirty (30) lines maximum. Long functions are probably doing too much and they are also difficult for the reader to comprehend. Bugs in them will be more likely to occur and possibly harder to track down.

  • Related to the previous bullets is the principle of maintaining a vertically compact visual presentation for more related code on screen at the same time. To this end, single line conditional statements are not only accepted but encouraged. Examples include if <condition>: break, if <condition>: continue, if <condition>: return <data>, else: return <data>, except <exception-list> as exc: pass, etc….

  • Please avoid excessive indentation. For example, using if not <condition>: continue inside a loop body is generally preferable to using if <condition>: followed by indented lines inside that body. Excessive indentation tends to cause code to become smushed against the right margin, which reduces legibility.

  • Please do not use lower camel case (camelCase) for anything. In addition to increasing the chance for typographic errors from not pressing or releasing the shift key on time, it suffers from legibility issues. Upper camel case (aka., Pascal case) (CamelCase) can be used for the names of types, but not for anything else. Aside from that, please use an underscore (_) to separate words within an identifier. Since Python does not support hyphens in identifiers, underscores can serve as reasonable visual separators instead, although they can also suffer typographic errors from incorrect shift key timing. I.e., prefer is_function to isfunction and starts_with to startswith, in spite of seeing the unseparated words convention in use on standard Python types and in the Python standard library. The convenience of legibility outweighs the inconvenience of mistyping underscores or letters around them, as code should be read more than it is written.