The popular Python web framework Pyramid recently had a bug. Installing a released version of Pyramid, such as 1.8 or 1.9, somehow resulted in installing a pre-release version of one of its most important dependencies, WebOb. This was surprising and potentially disruptive in a production environment. Having been surprised by a similar bug in a different project, I followed this with interest.

What Was Happening

I'll quote from the bug description here:

Since WebOb 1.8.0rc1 is released, Pyramid 1.8 and 1.9 versions install a the rc version of WebOb, instead of the latest stable 1.7.4.

1.7

$ pip install 'pyramid < 1.8'
Collecting WebOb>=1.3.1 (from pyramid<1.8)
    Using cached WebOb-1.7.4-py2.py3-none-any.whl

1.8

$ pip install 'pyramid < 1.9'
Collecting WebOb>=1.7.0rc2 (from pyramid<1.9)
    Using cached WebOb-1.8.0rc1-py2.py3-none-any.whl

1.9

$ pip install 'pyramid'
Collecting WebOb>=1.7.0rc2 (from pyramid)
    Using cached WebOb-1.8.0rc1-py2.py3-none-any.whl

A simple pip install webob installs the stable 1.7.4.

What Was So Surprising

Python Version Numbers

PEP 440 is the standard that specifies how version numbers are to be parsed and interpreted in Python packaging. Unlike some other packaging environments (npm), the Python standard was designed to be extremely flexible, taking into account existing version numbers and permitting both Semantic Versioning and Calendar Versioning.

The PEP defines how to order version numbers, and what constitutes a release versus a pre-release. Under this scheme, 1.8 is a released version (equivalent to 1.8.0) that succeeds (is greater than) 1.7.5. 1.9a1 is an alpha pre-release which can be succeeded by 1.9a2 for a second alpha release, which can in turn be succeeded by 1.9b1 and 1.9rc1 for the first beta and release candidate, respectively. 1.9 is a release that succeeds all of those versions.

In other words, the released version of WebOb was 1.7.4, and it was expected that installing a released version of Pyramid would install released versions of its dependencies. But instead, installing the released version of Pyramid installed the pre-release version of WebOb.

This was surprising because, beginning with pip version 1.4, pip does not install pre-release versions by default. If you want a pre-release version, you have to opt-in with the special --pre command line argument. Release 1.4 of pip was all the way back in 2013, an eternity in terms of the Python packaging ecosystem.

What Was Going On

During development of Pyramid 1.8, Pyramid had needed access to a feature only available in newer versions of WebOb (1.7). WebOb 1.7 wasn't released yet, but there was a pre-release available, 1.7.0.rc2. Thus 1.7.0rc2 was listed as the dependency version for Pyramid (WebOb >= 1.7.0rc2), and Pyramid 1.8 and 1.9 were released with that dependency version. The expectation was that the rule about pre-releases not being installed by default would continue to hold: when there was a 1.7.0 final released, that would be installed, and because 1.8.0rc1 was a pre-release, but 1.7.x was not, and was also greater than 1.7.0rc1, the 1.7.x release would be installed.

As detailed in a pip bug report, it turns out that's an incorrect expectation. The authors of of PEP 440 and pip weighed in and explained that the intent was that once you opt in to pre-releases, whether implicitly with the --pre command line argument, or explicitly in your dependency versions, you opt in to all pre-releases. 1.8.0rc1 is greater than 1.7.4, and meets the requirement (WebOb >= 1.7.0rc2), so that's what you get.

What The Fix Was

The fix in Pyramid was to remove the rc2 pre-release qualifier from their dependency list, thus opting out of pre-releases. Because 1.7.0 final had been released, this had the desired effect of having access to the needed feature, while also not opening up end-users to unexpected pre-release versions.

To me, the take away for deployments is to be explicit about your expected dependencies, even transitive ones. This can be in a pip requirements.txt or a buildout versions.cfg.