Over the last few years the process of creating a Python package has evolved. However, for historical reasons a lot of Python packages are still using old packaging mechanisms, and the laziness of developers, which are just copying and adapting existing packages, delays the convergence to recent approaches. However, no matter of the used approach the least common denominator is that all Python packages should be easily installable using PyPI, the Python Package Index.
Instead of sketching the history of possible old approaches and mistakes, in the following, a simple up-to-date approach is presented to create a Python package and publish it on PyPI.
Starting from Python 2.7.9/ 3.4
pip should already be installed. If it is not installed yet, install it following PIP's documentation.
Make sure to upgrade to the latest version:
pip install -U pip setuptools
pip install twine
Each python package has a certain structure and must include several files. Here an overview of the required package files (here: package example):
example/ example/ __init.__.py example.py ... CHANGELOG.rst LICENSE.txt MANIFEST.in README.rst setup.cfg setup.py
Details on required files
In the following the required files and their meanings are discussed:
setup.py is the most important file at the root level of the project. It serves two main functions:
- Set the basic configuration for the package via the
- Provide a CLI for package related tasks. Available commands can be listed by calling:
python setup.py --help-commands
from setuptools import setup, find_packages import codecs import os # get current directory here = os.path.abspath(os.path.dirname(__file__)) def get_long_description(): """ get long description from README.rst file """ with codecs.open(os.path.join(here, "README.rst"), "r", "utf-8") as f: return f.read() setup( name='example', version='0.0.1', description='An example project', long_description=get_long_description(), url='https://myurl.net/example', author='John Doe', firstname.lastname@example.org', license='MIT', classifiers=[ 'Development Status :: 3 - Alpha', 'Intended Audience :: Developers', 'Topic :: Software Development :: Build Tools', 'License :: OSI Approved :: MIT License', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', ], keywords='python packaging', packages=find_packages( exclude=['contrib', 'docs', 'tests'] ), install_requires=, )
The most important parameters are:
name: name of the project as it can be found on PyPI. Make sure that it follows the naming conventions as stated in PEP 426 and that the name is not used by another project yet.
version: the version of the project (each new version will be listed on PyPI). Make sure to use a valid version scheme as stated in PEP 440. It is recommended to store the version only at one place in the project. See the advance technique of single sourcing the version for ways to implement this.
long_descriptionwill be shown on the PyPI webpage on publishing
url: URL to the project's webpage
author_email: author's contact information
license: every project MUST have a license, otherwise people can not legally use it! Make sure that it corresponds to the licence file
classifiers: a list of classifiers that categorize the project on PyPI. See the Classifiers List for available categories. Since packages with unknown categories are rejected, it is possible to add a category like
Private :: Do Not Uploadto avoid the upload of a package. Nevertheless, it is recommended to state at least the following categories for a package:
- development status (alpha, beta, production)
- intended audience and topic
- supported programming language versions
keywords: list of keywords describing the project
packages: list of packages that should be included in the project.
find_packages()can be used to automatically find these packages.
install_requires: dependencies that are required to run the project. Will be automatically installed when a package is installed via
Note that there are a couple of other parameters that are not listed here.
setup.cfg is an ini file that contains default options for
setup.py commands. For example, the
universal flag can be set that states that the package works with Python 2 and 3.
Contains the license of the project. This is very important because otherwise the project cannot be legally used by others. NEVER made up some custom license, but rather rely on one of the Open Source Licenses.
The MIT License (MIT) Copyright (c) <year> <copyright holders> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
MANIFEST.in file contains a list of all additional files that should be included in the package, but that are not covered by the
A detailed explanation and a template can be found in the diskutils help.
# include additional documents such as readme files include *.rst *.txt
The check-manifest package can be helpful to check its content and probably missing files.
Though, it is not a strict requirement, it is recommended to add a
CHANGELOG.rst file to highlight the changes between the versions.
The package itself (example/)
The source code of the package is normally put into a separate directory that has the same name as the package (here:
There are a couple of other things that may be interesting in the context of package creation:
For a real package it would make sense to create some automated tests. For example, there are a couple of approaches that use Continuous Integration. In this context tox provides a standardize way of testing Python code. It can be used in combination with Travis-CI a distributed CI server which builds tests for open source projects for free and which is well-integrated with GitHub.
Packaging the projects
Finally, a package, also referred to as distribution, can be created. There are different ways of creating a package:
The simplest way to create a package is to create a source distribution:
python setup.py sdist
The drawback of this approach is that it is a little bit slow on installation time, because the build step must be conducted each time the package is installed via
A wheel is a built package i.e. it can be installed without the build step on installation. As a result, the installation is much faster for the user.
python setup.py bdist_wheel
There are different types of wheels:
- Universal wheels: can be created for pure python packages that support Python 2 and 3. For this, the
--universalflag must be attached to the command or must be set in
- Pure python wheels: can be created from pure python packages that cannot run on both Python 2 and 3 without modification. However, often Python 2 code can be converted with 2to3 and then build again for Python 3.
- Plattform wheels: are created when the package can only be built for a certain plattform due to the involvement of non-Python binary extensions. The platform dependency will automatically be detected so that the package name changes correspondingly.
Uploading the project to PyPI
After building the python package, it can now be uploaded to PyPI.
Create an account
First, create an account on the PyPI webpage. You will obtain a mail with a confirmation link that must be visited.
After the successful registration, create the
~/.pypirc file with the following content:
[distutils] index-servers=pypi [pypi] repository = https://pypi.python.org/pypi username = <username> password = <password>
Register the project at PyPI
twine register dist/*
The project's metadata are used to register the project at the webpage.
Upload the project to PyPI
twine upload dist/*
Now, your python package should be findable on PyPI.
Building a python package is still a little bit tricky, but by following the described steps it should be possible. The described steps do only cover the basics -- for a detailed description of possible options and parameters check the Python Packaging User Guide.