JF's Dev Blog

Django, Vue, and other things, too

Create and Publish a Python Package with Poetry

Poetry Pyton Package Index

Note

"Produce and Publish a Python Package with Poetry" would've been a better title if search engines ranked by literary device usage. That's a 5-P alliteration. You could fill a pod with all those Ps.

If you've ever published a Python package using a setup.py script, you might've found that writing the script to publish your package was harder than writing the package itself.

Python developers recognized that, and there are tools that employ a more modern way of building packages. Poetry and Flit are two popular tools for building Python packages.

Because I've used Poetry as a Python dependency management tool, I decided to take it for a spin for its package management functionality.

To get some hands-on experience publishing a package using Poetry, I recently released Flake8 Markdown, a tool that uses Flake8 to lint Python code in Markdown files.

This post walks through the changes I made to Flake8 Markdown's pyproject.toml and codebase to get it ready to publish on PyPI, the Python Package Index.

Info

Check out the Flake8 Markdown repository for the published version.

Creating a package

To create a package with Poetry, it helps if Poetry is installed. To do that, follow the Poetry installation instructions.

Now, to create a package with Poetry, we'll run poetry new along with the name of the directory that will house the package:

$ poetry new flake8-markdown
Created package flake8-markdown in flake8-markdown

We've got mail! Well, we've got a package, anyway. Let's open it up:

flake8-markdown/
├── flake8_markdown/
│   └── __init__.py
├── tests/
│   ├── __init__.py
│   └── test_flake8_markdown.py
├── pyproject.toml
└── README.rst

Out-of-the-box, Poetry gives us a simple package structure and a pyproject.toml file with:

  • A package version of 0.1.0
  • A minimum Python version (^3.7, in my case)
  • pytest support and one unit test

Info

For infomation and background on pyproject.toml, see PEP 518.

This package wouldn't look like much if we published it on the Python Package Index (PyPI) as-is, so let's make it look like something worth installing.

Customizing the package

Here's the initial pyproject.toml file:

[tool.poetry]
name = "flake8-markdown"
version = "0.1.0"
description = ""
authors = ["John Franey <johnfraney@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^3.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Here's the finished pyproject.toml:

[tool.poetry]
name = "flake8-markdown"
version = "0.1.1"
description = "Lints Python code blocks in Markdown files using flake8"
authors = ["John Franey <johnfraney@gmail.com>"]
# New attributes
license = "MIT"
readme = "README.md"
homepage = "https://github.com/johnfraney/flake8-markdown"
repository = "https://github.com/johnfraney/flake8-markdown"
keywords = ["flake8", "markdown", "lint"]
classifiers = [
    "Environment :: Console",
    "Framework :: Flake8",
    "Operating System :: OS Independent",
    "Topic :: Software Development :: Documentation",
    "Topic :: Software Development :: Libraries :: Python Modules",
    "Topic :: Software Development :: Quality Assurance",
]
include = [
    "LICENSE",
]

[tool.poetry.dependencies]
# Updated Python version
python = "^3.6"
# New dependency
flake8 = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^3.0"

# New scripts
[tool.poetry.scripts]
flake8-markdown = 'flake8_markdown:main'

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Info

For a full list of available sections to customize a Poetry package, see Poetry's pyproject.toml documentation.

Let's step through the changes from top to bottom.

version

The version specifies the current version of a package. I try to follow Semantic Versioning so it's obvious when an update is backwards-incompatible.

Note

Once a version of a package exists on PyPI, it's impossible to upload different code using that same version number. Be sure to use test changes using TestPyPI before releasing a version on PyPI.

description

This short desription appears in pip search and PyPI search results.

license

The license appears in the "Meta" section of the package's PyPI page.

I used GitHub's choosealicense.com to find an appropriate license and settled on MIT. Poetry lists other common licenses and their recommended notation, too.

readme

Poetry creates a README.rst file, but I prefer to write documentation in Markdown.

The README content appears as the project description on PyPI, so it's good to explain why and how to use your package.

My README includes:

  • Shields (because they look fancy)
  • Introduction
  • Installation
  • Usage
  • Code of Conduct
  • History (roughly following keep a changelog)

homepage and repository

These appear in the Project links section of the package's PyPI page:

PyPI project links

There's also a documentation attribute if your package has a separate documentation website.

keywords

These appear in the Meta section of the PyPI page:

PyPI keywords

classifiers

"Trove classifiers" act as categories for PyPI packages, and they can be used to filter packages on PyPI. They appear in the "Classifiers" section of the PyPI page:

PyPI classifiers

See the PyPI classifiers page for a complete list.

[tool.poetry.dependencies]

The dependencies section showcases one of the best features of Poetry.

When managing packages with vanilla pip or Pipenv, it's common to specify packages in two places: requirements.txt or Pipfile for dev and deployment dependencies, and setup.py for runtime/install dependencies.

Poetry uses pyproject.toml for all dependencies, which simplifies dependency management.

This section includes two changes:

  1. Update the minimum Python version from 3.7 to 3.6 for wider compatibility
  2. Add flake8 as a dependency

[tool.poetry.scripts]

Scripts are "the scripts or executable that will be installed when installing the package". In other words, this is where developers can create CLI commands from functions.

Scripts take the form:

script_name = '{package_name}:{function_name}'

Flake8 Markdown—contain your surprise—has a CLI command called flake8-markdown:

flake8-markdown = 'flake8_markdown:main'

After installing the flake8-markdown package, running flake8-markdown will call the main() function from flake8_markdown/__init__.py.

Info

For a package to be a runnable module, like python -m flake8-markdown, it needs a __main__.py module. In Flake8 Markdown, the __main__.py file imports and runs the same main() function as the above script.

Publishing the package

TestPyPI

TestPyPI is "a separate instance of the Python Package Index that allows you to try distribution tools and processes without affecting the real index". Uploading packages to TestPyPI and installing from there can help package maintainers avoid shipping broken versions of their packages.

Let's see how to upload a package to TestPyPI.

Note

You'll need to register for a TestPyPI account before uploading packages to the test package index.

First, build the package:

$ poetry build

Next, add Test PyPI as an alternate package repository:

$ poetry config repositories.testpypi https://test.pypi.org/simple

Now, publish the package to Test PyPI:

$ poetry publish -r testpypi

Publishing flake8-markdown (0.1.1) to testpypi
Username:
Password:

 - Uploading flake8-markdown-0.1.1.tar.gz 100%
 - Uploading flake8_markdown-0.1.1-py3-none-any.whl 100%

Finally, verify that the package looks and works as intended by viewing it on testpypi.pypi.org and installing the test version in a separate virtual environment:

pip install --index-url https://test.pypi.org/simple/ flake8-markdown

PyPI

If the package looks great on Test PyPI and works to boot, publishing to PyPI is as easy as:

poetry publish

Note

You'll need to register for a PyPI account before uploading packages to the package index. This account is separate from any account on TestPyPI.

Wrap-up

PEP 517 opened the door for tools like Poetry to provide a developer-friendly way to build Python packages. As a result, creating and publishing a package with Poetry is a straightforward, gotcha-free experience. Building a package is as easy as writing the code and adding sections to a pyproject.toml file.

It was so pleasant, I decided to write a poem about it:

Floating Python code

With Poetry assembled

Journeys to the cloud

Poetry poetry. How about that?