Deploying Python Packages to PyPI using Flit
The traditional way of package deployment in Python is using a setup.py
script located in the root of your project and then running python setup.py sdist upload
to release a new version.
It works by using distutils, setuptools or distribute and there is also twine which is a command line application to manage uploads.
History
distutils is the standard way of Python package distribution included in standard library since Python 2.x then setuptools was created to overcome distutils limitations and also introduces a command line application called easy_install
(currently we use its sucessor called pip
) and also setuptools introduced a very handy feature called pkg_resources
. One of the characteristics of setuptools is that it uses Monkey Patching over the standard distutils to fix existing problems.
Other forks of setuptools has been created to fix that issues and add common developers preferences so well known forks like distribute and distutils2 and distlib has been merged back to the original setuptools
Lots of other packaging tools has been created to try to fix the distribution problems, some maintained by PyPA (Python Package Authority) and some maintained by community.
How it works in standard way
using one of the above you should create a file called setup.py
in the root of your project, e.g:
from <my_favorite_dist_tool> import setup
# Example taken from Django's repository
setup(
name='Django',
version=version,
url='https://www.djangoproject.com/',
author='Django Software Foundation',
author_email='foundation@djangoproject.com',
description=('A high-level Python Web framework that encourages '
'rapid development and clean, pragmatic design.'),
license='BSD',
packages=find_packages(exclude=EXCLUDE_FROM_PACKAGES),
include_package_data=True,
scripts=['django/bin/django-admin.py'],
entry_points={'console_scripts': [
'django-admin = django.core.management:execute_from_command_line',
]},
install_requires=['pytz'],
extras_require={
"bcrypt": ["bcrypt"],
"argon2": ["argon2-cffi >= 16.1.0"],
},
zip_safe=False,
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Environment :: Web Environment',
'Framework :: Django',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Topic :: Internet :: WWW/HTTP',
'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
'Topic :: Internet :: WWW/HTTP :: WSGI',
'Topic :: Software Development :: Libraries :: Application Frameworks',
'Topic :: Software Development :: Libraries :: Python Modules',
],
)
As you can see it is very confusing to decide which of the distribute tools to addopt and how the setup.py
should be writen because there are different examples over the github most famous Python repositories.
Making things easier with Flit
NOTE: Forget all about all that history and
setup.py
you read above and consider using only Flit.
Flit is a simple way to Package and deploy Python projects on PyPI, Flit makes it easier by using a simple flit.ini
file and assumes common defaults to save your time and typing.
I knew about Flit when I was taking a look at Mariatta Wijaya game called Tic Tac Taco Pizza and noticed that she used flit
to deploy the game, so we also asked her the reason for using this on the podcast we recorded so I decided to try porting my projects to Flit.
How it works?
Instead of a complex setup.py
you put a simple flit.ini
in the root of your project and it looks like:
[metadata]
module = markdocs
author = Bruno Rocha
author-email = rochacbruno@gmail.com
maintainer = rochacbruno
maintainer-email = rochacbruno@gmail.com
home-page = https://github.com/rochacbruno/markdocs
requires = mistune
click
description-file = README.md
classifiers = Programming Language :: Python :: 3.6
Intended Audience :: Developers
License :: OSI Approved :: MIT License
Topic :: Documentation
Topic :: Software Development :: Documentation
Topic :: Software Development :: Quality Assurance
requires-python = >=3.6
[scripts]
markdocs = markdocs:main
Now you only need to have flit
installed in your local machine pip3 install flit
(unsing pip3 as flit works only in Python3+) and optionally is recommended to have pandoc
and pypandoc
installed because Flit can convert your README.md
into the .rst
, the format still used by PyPI (note that Markdown support is coming to PyPi.org soon).
The advantages are:
- No more complicated
setup.py
- If you omit some fields it will assume common used defaults
- It is easier to read and write
- Will convert your README.md
- Will take the
__version__
included in yourprogram/__init__.py
- Can deploy to TestPyPI
- Avoids over engineering on
setup.py
Development installation
To install your package during development use
flit install --symlink --python path/to/virtualenv/bin/python
the
--python
is optional, by default will take the currentwhich python
Registering and deploying
It is easy and will register the new project for you if doesn't exist on PyPI
flit publish
Flit packages a single importable module or package at a time, using the import name as the name on PyPI. All subpackages and data files within a package are included automatically.
Important!
- Flit will use the data from
~/.pypirc
to authenticate and to find the server deployment addresses - You can also set
FLIT_USERNAME
andFLIT_PASSWORD
andFLIT_INDEX_URL
as environment variables which makes flit good for CI deployment (e.g: TravisCI)
What is missing?
NOTE: Flit is open-source, so some of this things are already under consideration and there are PRs opened.
- Flit will not bump your project version automatically, you can still use tools like bumpversion but this feature would be better if builtin
- Flit will not parse a
requirements.txt
file and would be nice to have it as tools likepyup.io
can track those files but notflit.ini
yet - Flit does not create a
.lock
frozen file with the version used on specific release and it is interesting just likepipenv
does
Conclusion
Flit is the easier way to deploy packaged to PyPI following 3 simple steps.
- Install flit
- Describe your flit.ini
- Run
flit publish
then your library is released to PyPI.
However
Python still needs better standards because you still need separated tools to make common tasks and using a single tool to that tasks (pack, install, deploy, create) would be better (just like what cargo does for Rust), instead in Python you have:
- Flit to deploy packages to PyPI for distribution
- bumpversion to bump your semver number
- pip to install and update packages from PyPI (or pipenv/WIP to do the same with more powers)
- Cookiecutter to create a new Python Package (from strucured templates)
- safety to check dependencies security
- flake or pylint to static and styling checks
- venv, pyenv or virtualenvwrapper to manage isolated environments
- pytest to run tests
- pyup.io to watch for depdendency updates
- tox for testing on multiple environments
- sphinx to create documentation
- lots of other options
Having so many tools brings a lot of confusion and makes it hard to choose, why not having a single tool, based on different plugins sharing the same API?
Just an idea
python -m manage [install packagename] # <-- calls pip or pipenv
[publish --options --bump=version] # <-- calls `flit` and `bumpversion`
[new packagename templatename] # <-- calls cookiecutter
[safecheck] # <-- calls safety
[checkupdates] # <-- checks in the same way as Pyup.io does
[test path] # <-- calls pytest, nose or unittest
[lint path] # <-- calls flake, pylint
[venv options] # <-- calls the existing venv module
[docs --options] # <-- calls sphinx, pydoc, markdocs or other
All above configurable via config file or env vars and each of that endpoints
would be provided by many plugins sharing the same API, so you could choose between flit
or twine
as your publish manager
etc..
So maybe I can implement that features in manage
Please share in comments if you know some other Python management tool
comments powered by Disqus Go Top