Pipenv and Poetry: Benchmarks & Ergonomics

Pipenv and Poetry are maturing next-generation Python dependency management tools. Each simplifies the process of creating a virtual environment and organizing dependencies. They also help guarantee that builds of your package are reproducible by locking dependencies to specific versions.

After using both, I decided to do a command-to-command comparison of each tool's dependency management capabilities to determine which one I could see myself with long-term.

It's like a season of Bachelor where the suitors are pip alternatives and the romance level is float('inf').

This post has two parts. First, I do a brief feature comparison. Second, I perform common dependency management tasks in both Pipenv and Poetry, highlighting notable differences in usability.

Feature Comparison

Both Pipenv and Poetry provide modern Python package management workflows.

pip-based dependency organization typically involves separate files for production and development dependencies:

requirements.txt
requirements-dev.txt

Pipenv and Poetry replace separate dependency files by specifying core and development dependencies in separate sections of the same file. This is similar to how npm organizes dependencies and devDependencies in a package.json file.

Poetry takes more inspiration from package.json, too. Its pyproject.toml, described in PEP 518, can specify publishing information and configure installed tools, like flake8. Pipenv's Pipfile is narrower in scope and doesn't handle publishing or configuration.

Here's a quick look at how the tools compare:

Benchmarks & Ergonomics

These benchmarks are meant to compare the relative speed of common commands used during development. Each of the commands is benchmarked using time, running on my AMD Ryzen 5 1600.

These benchmarks, unless otherwise noted, use the following package packages:

I benchmarked these actions:

• Pipenv and Poetry: Benchmarks & Ergonomics
  • Feature Comparison
  • Benchmarks & Ergonomics
    • Installing from scratch
      • Pipenv
      • Poetry
    • Adding a dependency
      • Pipenv
      • Poetry
    • Locking dependencies
      • Pipenv
      • Poetry
    • Uninstalling a dependency
      • Pipenv
      • Poetry
  • Wrap-up

Installing from scratch

By default, poetry installs dependencies and dev dependencies, while pipenv installs only non-dev dependencies. For a fair comparison, pipenv is run with the --dev flag below.

poetry install does something unexpected: it installs the current project's root package as its own dependency. At the time of writing, there's an active pull request to add a flag that disables this behaviour, which should make it into Poetry v1.0.

This extra step could explain why this is the only command I benchmarked where Pipenv is faster than Poetry.

Pipenv

$ time -f "Total time: %es" pipenv install --dev
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (19a88e)!
Installing dependencies from Pipfile.lock (19a88e)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 23/23 — 00:00:01
Total time: 9.23s

Poetry

$ time -f "Total time: %es" poetry install
Updating dependencies
Resolving dependencies... (1.2s)


Package operations: 25 installs, 0 updates, 0 removals

Writing lock file

  - Installing decorator (4.3.2)
  - Installing ipython-genutils (0.2.0)
  - Installing parso (0.3.4)
  - Installing ptyprocess (0.6.0)
  - Installing six (1.12.0)
  - Installing wcwidth (0.1.7)
  - Installing backcall (0.1.0)
  - Installing entrypoints (0.3)
  - Installing jedi (0.13.2)
  - Installing mccabe (0.6.1)
  - Installing pexpect (4.6.0)
  - Installing pickleshare (0.7.5)
  - Installing prompt-toolkit (2.0.9)
  - Installing pycodestyle (2.5.0)
  - Installing pyflakes (2.1.0)
  - Installing pygments (2.3.1)
  - Installing pytz (2018.9)
  - Installing tornado (5.1.1)
  - Installing traitlets (4.3.2)
  - Installing django (2.1.7)
  - Installing djangorestframework (3.9.1)
  - Installing flake8 (3.7.6)
  - Installing ipython (7.3.0)
  - Installing livereload (2.6.0)
  - Installing yapf (0.26.0)
Total time: 19.37s

Adding a dependency

Pipenv's install command is dual-purpose. If a package is specified after pipenv install, Pipenv will install only that package. If no package is specified, pipenv installs all (non-dev) dependencies.

Poetry, like Yarn, has separate commands for adding a new dependency and installing existing ones.

Poetry's output also provides more information about Pipenv, including all sub-dependencies and their versions.

Pipenv

$ time -f "Total time: %es" pipenv install --dev pytest
Installing pytest…
Adding pytest to Pipfile's [dev-packages]…
✔ Installation Succeeded
Pipfile.lock (05d2a3) out of date, updating to (19a88e)…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (05d2a3)!
Installing dependencies from Pipfile.lock (05d2a3)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 29/29 — 00:00:01
Total time: 9.66s

Poetry

$ time -f "Total time: %es" poetry add --dev pytest
Using version ^4.3 for pytest

Updating dependencies
Resolving dependencies... (0.4s)


Package operations: 6 installs, 0 updates, 0 removals

Writing lock file

  - Installing atomicwrites (1.3.0)
  - Installing attrs (18.2.0)
  - Installing more-itertools (6.0.0)
  - Installing pluggy (0.8.1)
  - Installing py (1.8.0)
  - Installing pytest (4.3.0)
Total time: 5.43s

Locking dependencies

poetry performs its locking significantly faster than pipenv. Because both tools perform file locking after nearly every command, poetry feels snappier in everyday use.

I'm far from the first to notice:

A maintainer in that thread explains that Pipenv "re-hash[es] every file for security" each time it performs file locking. Pipenv locks on every package add & remove, which means that the more dependncies your project has, the slower Pipenv operations will be.

Poetry isn't immune from speed complaints, either, and its owner explains that it will likely never be as fast as pip:

Poetry orders the packages to install so that the deepest packages in the dependency graph are installed first to avoid errors at installation time. This requires sequential installation of the packages which takes longer but is more "secure".

Also, Poetry checks hashes of installed packages for security reasons and due to the way pip works, Poetry has to generate a temporary requirements.txt file to make pip check hashes. This adds an overhead which explains the difference between the two tools.

Pipenv

$ time -f "Total time: %es" pipenv lock
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (05d2a3)!
Total time: 6.11s

Poetry

$ time -f "Total time: %es" poetry lock
Updating dependencies
Resolving dependencies... (0.3s)


Total time: 1.67s

Uninstalling a dependency

When removing a dependency, Poetry also uninstalls its sub-dependencies if no other package uses them. Pipenv does not. In this benchmark, for instance, Pipenv orphans 5 dependencies.

This is an important difference in usability.

Let's say your code depends on a sub-dependency that gets installed with a top-level dependency. If you remove that dependency, your local code will continue to work because that sub-dependency is still installed.

You might not realize that your dependencies are out of whack until you've pushed a broken build. This is one of the reasons Poetry was developed.

Pipenv

$ time -f "Total time: %es" pipenv uninstall pytest
Uninstalling pytest…
Uninstalling pytest-4.3.0:
  Successfully uninstalled pytest-4.3.0

Removing pytest from Pipfile…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (19a88e)!
Total time: 6.21s

Poetry

$ time -f "Total time: %es" poetry remove --dev pytest
Updating dependencies
Resolving dependencies... (0.2s)


Package operations: 0 installs, 0 updates, 6 removals

Writing lock file

  - Removing atomicwrites (1.3.0)
  - Removing attrs (18.2.0)
  - Removing more-itertools (6.0.0)
  - Removing pluggy (0.8.1)
  - Removing py (1.8.0)
  - Removing pytest (4.3.0)
Total time: 3.50s

Wrap-up

For Python dependency management, Pipenv and Poetry share an admirable goal: simplified dependency management that creates reproducible builds.

Pipenv has broad support. It is an official project of the Python Packaging Authority, alongside pip. It's also supported by the Heroku Python buildpack, which is useful for anyone with Heroku or Dokku-based deployment strategies (like me).

Poetry is a one-stop shop for dependency management and package management. It simplifies creating a package, managing its dependencies, and publishing it. Compared to Pipenv, Poetry's separate add and install commands are more explicit, and it's faster for everything except for a full dependency install.

Poetry's version 1.0 is set to address the two areas where I preferred Pipenv: being able to export a requirements.txt file, and being able to avoid installing a package as a dependency of itself.

If Poetry keeps it up, once version 1.0 is released, I can see myself reaching for it in every new project I start.

Until that point, Pipenv could speed up, remove orphaned dependencies on uninstall, and win me back.

Wouldn't that be romantic?