py2deb: Python to Debian package converter

Welcome to the documentation of py2deb version 5.0! The following sections are available:

User documentation

The readme is the best place to start reading, it’s targeted at all users and documents the command line interface:

py2deb: Python to Debian package converter

https://travis-ci.org/paylogic/py2deb.svg?branch=master https://coveralls.io/repos/paylogic/py2deb/badge.svg?branch=master

The Python package py2deb converts Python source distributions to Debian binary packages (the ones used for installation). It uses pip-accel (based on pip) to download, unpack and compile Python packages. Because of this py2deb is compatible with the command line interface of the pip install command. For example you can specify packages to convert as command line arguments but you can also use requirement files if you want.

During the conversion process dependencies are automatically taken into account and converted as well so you don’t actually have to use requirement files including transitive dependencies. In fact you might prefer not explicitly listing your transitive dependencies in requirement files because py2deb will translate the version constraints of Python packages into Debian package relationships.

The py2deb package is currently tested on CPython 2.7, 3.5, 3.6, 3.7 and PyPy 2 and 3. Unfortunately Python 3.8+ is not yet supported (see below). For usage instructions please refer to the documentation hosted on Read The Docs.

Installation

The py2deb package is available on PyPI, so installation is very simple:

$ pip install py2deb

There are some system dependencies which you have to install as well:

$ sudo apt-get install dpkg-dev fakeroot

Optionally you can also install Lintian (which is not a hard dependency but more of a “nice to have”):

$ sudo apt-get install lintian

When Lintian is installed it will be run automatically to sanity check converted packages. This slows down the conversion process somewhat but can be very useful, especially when working on py2deb itself. Currently py2deb doesn’t fail when Lintian reports errors, this is due to the unorthodox ways in which py2deb can be used. This may change in the future as py2deb becomes more mature.

Usage

There are two ways to use the py2deb package: As the command line program py2deb and as a Python API. For details about the Python API please refer to the API documentation hosted on Read the Docs. The command line interface is described below.

Command line

Usage: py2deb [OPTIONS] …

Convert Python packages to Debian packages according to the given command line options (see below). The command line arguments are the same as accepted by the “pip install” command because py2deb invokes pip during the conversion process. This means you can name the package(s) to convert on the command line but you can also use “requirement files” if you prefer.

If you want to pass command line options to pip (e.g. because you want to use a custom index URL or a requirements file) then you will need to tell py2deb where the options for py2deb stop and the options for pip begin. In such cases you can use the following syntax:

$ py2deb -r /tmp -- -r requirements.txt

So the “–” marker separates the py2deb options from the pip options.

Supported options:

Option Description
-c, --config=FILENAME

Load a configuration file. Because the command line arguments are processed in the given order, you have the choice and responsibility to decide if command line options override configuration file options or vice versa. Refer to the documentation for details on the configuration file format.

The default configuration files /etc/py2deb.ini and ~/.py2deb.ini are automatically loaded if they exist. This happens before environment variables and command line options are processed.

Can also be set using the environment variable $PY2DEB_CONFIG.
-r, --repository=DIRECTORY

Change the directory where *.deb archives are stored. Defaults to the system wide temporary directory (which is usually /tmp). If this directory doesn’t exist py2deb refuses to run.

Can also be set using the environment variable $PY2DEB_REPOSITORY.
--use-system-package=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME Exclude a Python package (the name before the comma) from conversion and replace references to the Python package with a specific Debian package name. This allows you to use system packages for specific Python requirements.
--name-prefix=PREFIX

Set the name prefix used during the name conversion from Python to Debian packages. Defaults to “python”. The name prefix and package names are always delimited by a dash.

Can also be set using the environment variable $PY2DEB_NAME_PREFIX.
--no-name-prefix=PYTHON_PACKAGE_NAME Exclude a Python package from having the name prefix applied during the package name conversion. This is useful to avoid awkward repetitions.
--rename=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME Override the package name conversion algorithm for the given pair of package names. Useful if you don’t agree with the algorithm :-)
--install-prefix=DIRECTORY

Override the default system wide installation prefix. By setting this to anything other than “/usr” or “/usr/local” you change the way py2deb works. It will build packages with a file system layout similar to a Python virtual environment, except there will not be a Python executable: The packages are meant to be loaded by modifying Python’s module search path. Refer to the documentation for details.

Can also be set using the environment variable $PY2DEB_INSTALL_PREFIX.
--install-alternative=LINK,PATH Use Debian’s “update-alternatives” system to add an executable that’s installed in a custom installation prefix (see above) to the system wide executable search path. Refer to the documentation for details.
--python-callback=EXPRESSION

Set a Python callback to be called during the conversion process. Refer to the documentation for details about the use of this feature and the syntax of EXPRESSION.

Can also be set using the environment variable $PY2DEB_CALLBACK.
--report-dependencies=FILENAME Add the Debian relationships needed to depend on the converted package(s) to the given control file. If the control file already contains relationships the additional relationships will be added to the control file; they won’t overwrite existing relationships.
-y, --yes

Instruct pip-accel to automatically install build time dependencies where possible. Refer to the pip-accel documentation for details.

Can also be set using the environment variable $PY2DEB_AUTO_INSTALL.
-v, --verbose Make more noise :-).
-h, --help Show this message and exit.

Future improvements

The following sections list possible improvements to the project:

Python 3.8+ compatibility

The py2deb project builds on top of pip-accel, which was developed between 2013 and 2015 on top of pip >= 7.0, < 7.2. Since that time pip has grown enormously: At the time of writing (in August 2020) we’re now at pip 20!

None of the improvements made between pip 7-20 are available in pip-accel and py2deb and this has become somewhat of a glaring issue that plenty of users have run into (see #17, #18, #27 and #31).

Known issues being caused by this include:

  • The old pip version prevents Python 3.8+ compatibility.
  • The old pip version doesn’t know about python_requires metadata provided by PyPI and this forces users to maintain constraints files themselves, even though this shouldn’t be necessary.
  • While pip-accel supports installation from wheels, it was never exposed via the Python API and so py2deb lacks support for converting wheels (it currently needs source distributions).

The current state of affairs is best summarized in this comment. I’m hoping to complete the upgrade to newer pip and pip-accel releases in the coming weeks (as of this writing in August 2020) but can’t commit to a date.

Installation of system wide files

Find a way to facilitate (explicit / opt-in) installation of system wide files (not related to Python per se) based on a Python distribution? This could significantly reduce the need for “wrapper packages” that basically just pull in packages converted by py2deb and drop a few configuration files into place.

Related issues:See issue #7 for a related discussion.
Conversion of binary wheels

Investigate the feasability of supporting conversion of binary wheels. Slowly but surely the Python community seems to be gravitating towards (binary) wheels and once gravity has shifted we don’t want to be left in the dust! 😉

Full PEP-440 compatibility

Dive into PEP-440 and see if it can be fully supported? Then this question on Reddit can finally get a satisfying answer 🙂.

Similar projects

There are several projects out there that share similarities with py2deb, for example I know of stdeb, dh-virtualenv and fpm. The documentation includes a fairly detailed comparison with each of these projects.

Contact

The latest version of py2deb is available on PyPI and GitHub. The documentation is hosted on Read the Docs and includes a changelog. For questions, bug reports, suggestions, etc. please create an issue on GitHub.

License

This software is licensed under the MIT license.

© 2020 Peter Odding, Arjan Verwer and Paylogic International.

The comparison between py2deb and similar tools can help to determine if py2deb can be useful for you.

Where does py2deb fit in?

There are several projects out there that share similarities with py2deb, for example I know of stdeb, dh-virtualenv and fpm. For those who know these other projects already and are curious about where py2deb fits in, I would classify py2deb as a sort of pragmatic compromise between dh-virtualenv and stdeb (without the disadvantages that I see in both of these projects).

Below I will attempt to provide a fair comparison between these projects. Please note that it is not my intention to discourage the use of any of these projects or to list just the down sides: They all have their place! Of course I do think py2deb has something to add to the mix, otherwise I wouldn’t have created it :-).

If you feel that another project should be discussed here or that an existing comparison is inaccurate then feel free to mention this on the py2deb issue tracker.

The short comparison

In my research into py2deb, stdeb and dh-virtualenv I’ve come to a sort of realization about all of these projects that makes it fairly easy to differentiate them for those who have a passing familiarity with one or more of these projects: The projects can be placed on a spectrum ranging from very pragmatic (and dumb, to a certain extent :-) to very perfectionistic (and idealistic and fragile, to a certain extent). Based on my observations:

  • dh-virtualenv is a pragmatic solution to a concrete problem. It solves this single problem and seems to do so quite well.
  • stdeb is somewhat pragmatic in the sense that it tries to make the contents of the Python Package Index available to Debian based systems, but it is quite perfectionistic (idealistic) in how it goes about accomplishing this. When it works it results in fairly high quality conversions.
  • py2deb sits somewhere between dh-virtualenv and stdeb:
    • It allows complete requirement sets to be converted (similar to dh-virtualenv).
    • It converts requirement sets by generating individual binary packages (similar to stdeb).
    • It can convert requirement sets using a custom name and installation prefix to allow the same kind of isolation that dh-virtualenv provides.
    • It uses dpkg-shlibdeps to automatically track dependencies on system packages (inspired by stdeb).

Comparison to stdeb

The stdeb program converts Python source distributions to Debian source packages which can then be compiled to Debian binary packages (optionally in a single call).

Shared goals with stdeb

The stdeb and py2deb projects share very similar goals, in fact py2deb started out being based on stdeb but eventually reimplemented the required functionality on top of pip-accel and deb-pkg-tools. The following goals are still shared between stdeb and py2deb:

  • Combine the power and ease of deployment of Debian packaging with the rich ecosystem of Python packages available on the Python Package Index.
  • Provide users with a very easy way to take a Python package and convert it into a Debian binary package that is ready to install, without having to know the intricate details of the Debian packaging ecosystem.
Notable differences with stdeb

Although py2deb started out being a wrapper for stdeb the goals of the two projects have diverged quite a bit since then. Some notable differences:

  • stdeb starts by generating Debian source packages while py2deb generates Debian binary packages without intermediate Debian source packages:
    • stdeb works by converting a Python package to a Debian source package that uses the existing Debian Python packaging mechanisms. The Debian source package can then be compiled into a Debian binary package. These two actions can optionally be combined into a single invocation. stdeb is intended to generate Python Debian packages that comply to the Debian packaging policies as much as possible (this is my interpretation).
      • For example Python modules are installed in the pyshared directory so that multiple Python versions can use the modules. The advantages of this are clear, but the main disadvantage is that stdeb is sensitive to changes in Debian packaging infrastructure. For example it doesn’t run on older versions of Ubuntu Linux (at one point this was a requirement for me). py2deb on the other hand is kind of dumb but works almost everywhere.
    • py2deb never generates Debian source packages, instead it generates Debian binary packages directly. This means py2deb doesn’t use or integrate with the Debian Python packaging mechanisms. This was a conscious choice that I’ll elaborate on further in the following point.
  • The main use case of stdeb is to convert individual Python packages to Debian packages that are installed system wide under the python-* name prefix. On the other hand py2deb always converts complete dependency sets (in fact py2deb started out as a wrapper for stdeb that just added the “please convert a complete dependency set for me” aspect). Some consequences of this:
    • stdeb works fine when converting a couple of Python packages individually but if you want to convert a large dependency set it quickly becomes hairy and fragile due to scripting of stdeb, conflicts with existing system packages and other reasons. If you want this process to run automatically and reliably without supervision then I personally wouldn’t recommend stdeb - it has given me quite a few headaches because I was pushing stdeb way beyond its intended use case (my fault entirely, I’m not blaming the tool).
    • The larger the dependency set given to py2deb, the larger the risk that conflicts will occur between Python packages from the official repositories versus the packages converted by py2deb. This is why py2deb eventually stopped being based on stdeb: In order to add the ability to install converted packages under a custom name prefix and installation prefix. When used in this mode py2deb is something of a pragmatic compromise between stdeb and dh-virtualenv.

Comparison to dh-virtualenv

The dh-virtualenv tool provides helpers to easily create a Debian source package that takes a pip requirements file and builds a Python virtual environment that is then packaged as a Debian binary package.

Shared goals with dh-virtualenv

The following goals are shared between dh-virtualenv and py2deb:

  • Combine the power and ease of deployment of Debian packaging with the rich ecosystem of Python packages available on the Python Package Index.
  • Easily deploy Python based applications with complex dependency sets which may conflict with system wide Python packages (dh-virtualenv always provides this isolation while py2deb provides the option but doesn’t enforce it).
Notable differences with dh-virtualenv

The following notable differences can be observed:

  • dh-virtualenv requires creating a Debian source package in order to generate a Debian binary package while py2deb focuses exclusively on generating Debian binary packages. Both approaches are valid and have advantages and disadvantages:
    • The use of dh-virtualenv requires a certain amount of knowledge about how to create, manage and build Debian source packages.
    • The use of py2deb requires fairly little knowledge about Debian packaging and it specifically doesn’t require any knowledge about Debian source packages.
  • dh-virtualenv includes a complete requirement set in a single binary package while py2deb converts each requirement individually (whether configured to use an isolated name space or not):
    • An advantage of dh-virtualenv here is that the generated *.deb is completely self contained. The disadvantage of this is that when you update only a few requirements in a large requirement set you get to rebuild, redownload and reinstall the complete requirement set anyway.
    • For py2deb the situation is the inverse: Generated binary packages are not self contained (each requirement gets a separate *.deb archive). This means that when only a few requirements in a large requirement set are updated only those requirements are rebuilt, redownloaded and reinstalled.

Comparison to fpm

The fpm program is a generic package converter that supports multiple input formats (Python packages, Ruby packages, etc.) and multiple output formats (Debian binary packages, Red Hat binary packages, etc.).

Shared goals with fpm

The fpm and py2deb projects in the end have very different goals but there is at least one shared goal:

  • Provide users with a very easy way to take a Python package and convert it into a Debian binary package that is ready to install, without having to know the intricate details of the Debian packaging ecosystem.
Notable differences with fpm

Here are some notable differences between fpm and py2deb:

  • fpm is a generic package converter while py2deb specializes in conversion of Python to Debian packages. This makes fpm more like a Swiss Army knife while py2deb has a very specialized use case for which it is actually specialized (py2deb is smarter about Python to Debian package conversion).
  • With py2deb it is very easy to convert packages using a custom name and installation prefix, allowing conversion of large/complex requirement sets that would inevitably conflict with Debian packages from official repositories (e.g. because of older or newer versions).
  • py2deb recognizes dependencies on system packages (libraries) and embeds them in the dependencies of the generated Debian packages. This is not so important when py2deb is used on the system where the converted packages will be installed (the dependencies will already have been installed, otherwise the package couldn’t have been built and converted) but it’s essential when the converted packages will be deployed to other systems.

API documentation

The following API documentation is automatically generated from the source code:

API documentation

The following documentation is based on the source code of version 5.0 of the py2deb package. The following modules are available:

py2deb

The top level py2deb module contains only a version number.

py2deb.__version__

The version number of the pydeb package (a string).

py2deb.cli

Usage: py2deb [OPTIONS] …

Convert Python packages to Debian packages according to the given command line options (see below). The command line arguments are the same as accepted by the “pip install” command because py2deb invokes pip during the conversion process. This means you can name the package(s) to convert on the command line but you can also use “requirement files” if you prefer.

If you want to pass command line options to pip (e.g. because you want to use a custom index URL or a requirements file) then you will need to tell py2deb where the options for py2deb stop and the options for pip begin. In such cases you can use the following syntax:

$ py2deb -r /tmp -- -r requirements.txt

So the “–” marker separates the py2deb options from the pip options.

Supported options:

Option Description
-c, --config=FILENAME

Load a configuration file. Because the command line arguments are processed in the given order, you have the choice and responsibility to decide if command line options override configuration file options or vice versa. Refer to the documentation for details on the configuration file format.

The default configuration files /etc/py2deb.ini and ~/.py2deb.ini are automatically loaded if they exist. This happens before environment variables and command line options are processed.

Can also be set using the environment variable $PY2DEB_CONFIG.
-r, --repository=DIRECTORY

Change the directory where *.deb archives are stored. Defaults to the system wide temporary directory (which is usually /tmp). If this directory doesn’t exist py2deb refuses to run.

Can also be set using the environment variable $PY2DEB_REPOSITORY.
--use-system-package=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME Exclude a Python package (the name before the comma) from conversion and replace references to the Python package with a specific Debian package name. This allows you to use system packages for specific Python requirements.
--name-prefix=PREFIX

Set the name prefix used during the name conversion from Python to Debian packages. Defaults to “python”. The name prefix and package names are always delimited by a dash.

Can also be set using the environment variable $PY2DEB_NAME_PREFIX.
--no-name-prefix=PYTHON_PACKAGE_NAME Exclude a Python package from having the name prefix applied during the package name conversion. This is useful to avoid awkward repetitions.
--rename=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME Override the package name conversion algorithm for the given pair of package names. Useful if you don’t agree with the algorithm :-)
--install-prefix=DIRECTORY

Override the default system wide installation prefix. By setting this to anything other than “/usr” or “/usr/local” you change the way py2deb works. It will build packages with a file system layout similar to a Python virtual environment, except there will not be a Python executable: The packages are meant to be loaded by modifying Python’s module search path. Refer to the documentation for details.

Can also be set using the environment variable $PY2DEB_INSTALL_PREFIX.
--install-alternative=LINK,PATH Use Debian’s “update-alternatives” system to add an executable that’s installed in a custom installation prefix (see above) to the system wide executable search path. Refer to the documentation for details.
--python-callback=EXPRESSION

Set a Python callback to be called during the conversion process. Refer to the documentation for details about the use of this feature and the syntax of EXPRESSION.

Can also be set using the environment variable $PY2DEB_CALLBACK.
--report-dependencies=FILENAME Add the Debian relationships needed to depend on the converted package(s) to the given control file. If the control file already contains relationships the additional relationships will be added to the control file; they won’t overwrite existing relationships.
-y, --yes

Instruct pip-accel to automatically install build time dependencies where possible. Refer to the pip-accel documentation for details.

Can also be set using the environment variable $PY2DEB_AUTO_INSTALL.
-v, --verbose Make more noise :-).
-h, --help Show this message and exit.
py2deb.cli.main()[source]

Command line interface for the py2deb program.

py2deb.converter

The py2deb.converter module contains the high level conversion logic.

This module defines the PackageConverter class which provides the intended way for external Python code to interface with py2deb. The separation between the PackageConverter and PackageToConvert classes is somewhat crude (because neither class can work without the other) but the idea is to separate the high level conversion logic from the low level conversion logic.

py2deb.converter.MACHINE_ARCHITECTURE_MAPPING = {'armv6l': 'armhf', 'i686': 'i386', 'x86_64': 'amd64'}

Mapping of supported machine architectures (a dictionary).

The keys are the names reported by os.uname() and the values are machine architecture labels used in the Debian packaging system.

class py2deb.converter.PackageConverter(load_configuration_files=True, load_environment_variables=True, **options)[source]

The external interface of py2deb, the Python to Debian package converter.

Here’s an overview of the PackageConverter class:

Superclass: PropertyManager
Special methods: __init__()
Public methods: convert(), get_source_distributions(), install_alternative(), load_configuration_file(), load_default_configuration_files(), load_environment_variables(), rename_package(), set_auto_install(), set_conversion_command(), set_install_prefix(), set_lintian_enabled(), set_name_prefix(), set_python_callback(), set_repository(), transform_name(), transform_version() and use_system_package()
Properties: alternatives, debian_architecture, install_prefix, lintian_enabled, lintian_ignore, name_mapping, name_prefix, prerelease_workaround, python_callback, repository, scripts and system_packages

You can set the values of the install_prefix, lintian_enabled, name_prefix, prerelease_workaround, python_callback and repository properties by passing keyword arguments to the class initializer.

__init__(load_configuration_files=True, load_environment_variables=True, **options)[source]

Initialize a Python to Debian package converter.

Parameters:
alternatives[source]

The update-alternatives configuration (a set of tuples).

The value of this property is a set of set of tuples with two strings each (the strings passed to install_alternative()). It’s used by create_alternatives() and cleanup_alternatives() during installation and removal of the generated package.

Note

The alternatives property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

debian_architecture[source]

The Debian architecture of the current environment (a string).

This logic was originally implemented in py2deb but has since been moved to deb_pkg_tools.utils.find_debian_architecture(). This property remains as a convenient shortcut.

Note

The debian_architecture property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

install_prefix[source]

The installation prefix for converted packages (a string, defaults to /usr).

To generate system wide packages one of the installation prefixes /usr or /usr/local should be used. Setting this property to any other value will create packages using a “custom installation prefix” that’s not included in sys.path by default.

The installation prefix directory doesn’t have to exist on the system where the package is converted and will be automatically created on the system where the package is installed.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the setter set_install_prefix() was the only documented interface. The use of this setter is no longer required but still allowed.

Note

The install_prefix property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

lintian_enabled[source]

True to enable Lintian, False to disable it (defaults to True).

If this is True then Lintian will automatically be run after each package is converted to sanity check the result. Any problems found by Lintian are information intended for the operator, that is to say they don’t cause py2deb to fail.

Note

The lintian_enabled property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

lintian_ignore[source]

A list of strings with Lintian tags to ignore.

Note

The lintian_ignore property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

name_mapping[source]

Mapping of Python package names to Debian package names (a dictionary).

The name_mapping property enables renaming of packages during the conversion process. The keys as well as the values of the dictionary are expected to be lowercased strings.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the rename_package() method was the only documented interface. The use of this setter is no longer required but still allowed.

Note

The name_mapping property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

name_prefix[source]

The name prefix for converted packages (a string).

The default value of name_prefix depends on the Python interpreter that’s used to run py2deb:

  • On Python 2 the default name prefix is python.
  • On Python 3 the default name prefix is python3.
  • On PyPy 2 the default name prefix is pypy.
  • On PyPy 3 the default name prefix is pypy3.

When one of these default name prefixes is used, converted packages may conflict with system wide packages provided by Debian / Ubuntu. If this starts to bite then consider changing the name and installation prefix.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the setter set_name_prefix() was the only documented interface. The use of this setter is no longer required but still allowed.

Release 2.0 introduced the alternative default name prefixes pypy and python3. Before that release the default name prefix python was (erroneously) used for all interpreters.

Note

The name_prefix property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

prerelease_workaround[source]

Whether to enable the pre-release workaround in normalize_package_version() (a boolean).

By setting this to False converted version numbers will match those generated by py2deb 0.25 and earlier. Release 1.0 introduced the pre-release workaround and release 2.1 added the option to control backwards compatibility in this respect.

Note

The prerelease_workaround property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

python_callback[source]

An optional Python callback to be called during the conversion process (defaults to None).

You can set the value of python_callback to one of the following:

  1. A callable object (to be provided by Python API callers).
  2. A string containing the pathname of a Python script and the name of a callable, separated by a colon. The Python script will be loaded using exec.
  3. A string containing the “dotted path” of a Python module and the name of a callable, separated by a colon. The Python module will be loaded using importlib.import_module().
  4. Any value that evaluates to False will clear an existing callback (if any).

The callback will be called at the very last step before the binary package’s metadata and contents are packaged as a *.deb archive. This allows arbitrary manipulation of resulting binary packages, e.g. changing package metadata or files to be packaged. An example use case:

  • Consider a dependency set (group of related packages) that has previously been converted and deployed.
  • A new version of the dependency set switches from Python package A to Python package B, where the two Python packages contain conflicting files (installed in the same location). This could happen when switching to a project’s fork.
  • A deployment of the new dependency set will conflict with existing installations due to “unrelated” packages (in the eyes of apt and dpkg) installing the same files.
  • By injecting a custom Python callback the user can mark package B as “replacing” and “breaking” package A. Refer to section 7.6 of the Debian policy manual for details about the required binary control fields (hint: Replaces: and Breaks:).

Warning

The callback is responsible for not making changes that would break the installation of the converted dependency set!

Raises:

The following exceptions can be raised when you set this property:

  • ValueError when you set this to something that’s not callable and cannot be converted to a callable.
  • ImportError when the expression contains a dotted path that cannot be imported.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the setter set_python_callback() was the only documented way to configure the callback. The use of this setter is no longer required but still allowed.

Note

The python_callback property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

repository[source]

The directory where py2deb stores generated *.deb archives (a PackageRepository object).

By default the system wide temporary files directory is used as the repository directory (usually this is /tmp) but it’s expected that most callers will want to change this.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the set_repository() method was the only documented interface. The use of this method is no longer required but still allowed.

Note

The repository property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

scripts[source]

Mapping of Python package names to shell commands (a dictionary).

The keys of this dictionary are expected to be lowercased strings.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the set_conversion_command() method was the only documented interface. The use of this method is no longer required but still allowed.

Note

The scripts property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

system_packages[source]

Mapping of Python package names to Debian package names (a dictionary).

The system_packages property enables Python packages in a requirement set to be excluded from the package conversion process. Any references to excluded packages are replaced with a reference to the corresponding system package. The keys as well as the values of the dictionary are expected to be lowercased strings.

New in version 2.0: Before his property became part of the documented and public API in release 2.0 the use_system_package() method was the only documented interface. The use of this method is no longer required but still allowed.

Note

The system_packages property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

install_alternative(link, path)[source]

Install system wide link for program installed in custom installation prefix.

Use Debian’s update-alternatives system to add an executable that’s installed in a custom installation prefix to the system wide executable search path using a symbolic link.

Parameters:
  • link – The generic name for the master link (a string). This is the first argument passed to update-alternatives --install.
  • path – The alternative being introduced for the master link (a string). This is the third argument passed to update-alternatives --install.
Raises:

ValueError when one of the paths is not provided (e.g. an empty string).

If this is a bit vague, consider the following example:

$ py2deb --name-prefix=py2deb \
         --no-name-prefix=py2deb \
         --install-prefix=/usr/lib/py2deb \
         --install-alternative=/usr/bin/py2deb,/usr/lib/py2deb/bin/py2deb \
         py2deb==0.1

This example will convert py2deb and its dependencies using a custom name prefix and a custom installation prefix which means the py2deb program is not available on the default executable search path. This is why update-alternatives is used to create a symbolic link /usr/bin/py2deb which points to the program inside the custom installation prefix.

rename_package(python_package_name, debian_package_name)[source]

Override the package name conversion algorithm for the given pair of names.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • debian_package_name – The name of the converted Debian package (a string).
Raises:

ValueError when a package name is not provided (e.g. an empty string).
set_auto_install(enabled)[source]

Enable or disable automatic installation of build time dependencies.

Parameters:enabled – Any value, evaluated using coerce_boolean().
set_conversion_command(python_package_name, command)[source]

Set shell command to be executed during conversion process.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • command – The shell command to execute (a string).
Raises:

ValueError when the package name or command is not provided (e.g. an empty string).

The shell command is executed in the directory containing the Python module(s) that are to be installed by the converted package.

Warning

This functionality allows arbitrary manipulation of the Python modules to be installed by the converted package. It should clearly be considered a last resort, only for for fixing things like packaging issues with Python packages that you can’t otherwise change.

For example old versions of Fabric bundle a copy of Paramiko. Most people will never notice this because Python package managers don’t complain about this, they just blindly overwrite the files… Debian’s packaging system is much more strict and will consider the converted Fabric and Paramiko packages as conflicting and thus broken. In this case you have two options:

  1. Switch to a newer version of Fabric that no longer bundles Paramiko;
  2. Use the conversion command rm -rf paramiko to convert Fabric (yes this is somewhat brute force :-).
set_install_prefix(directory)[source]

Set installation prefix to use during package conversion.

The installation directory doesn’t have to exist on the system where the package is converted.

Parameters:directory – The pathname of the directory where the converted packages should be installed (a string).
Raises:ValueError when no installation prefix is provided (e.g. an empty string).
set_lintian_enabled(enabled)[source]

Enable or disable automatic Lintian checks after package building.

Parameters:enabled – Any value, evaluated using coerce_boolean().
set_name_prefix(prefix)[source]

Set package name prefix to use during package conversion.

Parameters:prefix – The name prefix to use (a string).
Raises:ValueError when no name prefix is provided (e.g. an empty string).
set_python_callback(expression)[source]

Set the value of python_callback.

set_repository(directory)[source]

Set pathname of directory where py2deb stores converted packages.

Parameters:directory – The pathname of a directory (a string).
Raises:ValueError when the directory doesn’t exist.
use_system_package(python_package_name, debian_package_name)[source]

Exclude a Python package from conversion.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • debian_package_name – The name of the Debian package that should be used to fulfill the dependency (a string).
Raises:

ValueError when a package name is not provided (e.g. an empty string).

References to the Python package are replaced with a specific Debian package name. This allows you to use system packages for specific Python requirements.

load_environment_variables()[source]

Load configuration defaults from environment variables.

The following environment variables are currently supported:

  • $PY2DEB_CONFIG
  • $PY2DEB_REPOSITORY
  • $PY2DEB_NAME_PREFIX
  • $PY2DEB_INSTALL_PREFIX
  • $PY2DEB_AUTO_INSTALL
  • $PY2DEB_LINTIAN
load_configuration_file(configuration_file)[source]

Load configuration defaults from a configuration file.

Parameters:configuration_file – The pathname of a configuration file (a string).
Raises:Exception when the configuration file cannot be loaded.

Below is an example of the available options, I assume that the mapping between the configuration options and the setters of PackageConverter is fairly obvious (it should be :-).

# The `py2deb' section contains global options.
[py2deb]
repository = /tmp
name-prefix = py2deb
install-prefix = /usr/lib/py2deb
auto-install = on
lintian = on

# The `alternatives' section contains instructions
# for Debian's `update-alternatives' system.
[alternatives]
/usr/bin/py2deb = /usr/lib/py2deb/bin/py2deb

# Sections starting with `package:' contain conversion options
# specific to a package.
[package:py2deb]
no-name-prefix = true

Note that the configuration options shown here are just examples, they are not the configuration defaults (they are what I use to convert py2deb itself). Package specific sections support the following options:

no-name-prefix:
A boolean indicating whether the configured name prefix should be applied or not. Understands true and false (false is the default and you only need this option to change the default).
rename:
Gives an override for the package name conversion algorithm (refer to rename_package() for details).
script:
Set a shell command to be executed during the conversion process (refer to set_conversion_command() for details).
load_default_configuration_files()[source]

Load configuration options from default configuration files.

The following default configuration file locations are checked:

  • /etc/py2deb.ini
  • ~/.py2deb.ini
Raises:Exception when a configuration file exists but cannot be loaded.
convert(pip_install_arguments)[source]

Convert one or more Python packages to Debian packages.

Parameters:pip_install_arguments – The command line arguments to the pip install command.
Returns:A tuple with two lists:
  1. A list of strings containing the pathname(s) of the generated Debian package package archive(s).
  2. A list of strings containing the Debian package relationship(s) required to depend on the converted package(s).
Raises:DuplicateFilesFound if two converted package archives contain the same files (certainly not what you want within a set of dependencies).

Here’s an example of what’s returned:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> archives, relationships = converter.convert(['py2deb'])
>>> print(archives)
['/tmp/python-py2deb_0.18_all.deb']
>>> print(relationships)
['python-py2deb (=0.18)']
get_source_distributions(pip_install_arguments)[source]

Use pip-accel to download and unpack Python source distributions.

Retries several times if a download fails (so it doesn’t fail immediately when a package index server returns a transient error).

Parameters:pip_install_arguments – The command line arguments to the pip install command (an iterable of strings).
Returns:A generator of PackageToConvert objects.
Raises:When downloading fails even after several retries this function raises pip.exceptions.DistributionNotFound. This function can also raise other exceptions raised by pip because it uses pip-accel to call pip (as a Python API).
transform_name(python_package_name, *extras)[source]

Transform Python package name to Debian package name.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • extras – Any extras requested to be included (a tuple of strings).
Returns:

The transformed name (a string).

Examples:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> converter.transform_name('example')
'python-example'
>>> converter.set_name_prefix('my-custom-prefix')
>>> converter.transform_name('example')
'my-custom-prefix-example'
>>> converter.set_name_prefix('some-web-app')
>>> converter.transform_name('raven', 'flask')
'some-web-app-raven-flask'
transform_version(package_to_convert, python_requirement_name, python_requirement_version)[source]

Transform a Python requirement version to a Debian version number.

Parameters:
  • package_to_convert – The PackageToConvert whose requirement is being transformed.
  • python_requirement_name – The name of a Python package as found on PyPI (a string).
  • python_requirement_version – The required version of the Python package (a string).
Returns:

The transformed version (a string).

This method is a wrapper for normalize_package_version() that takes care of one additional quirk to ensure compatibility with pip. Explaining this quirk requires a bit of context:

  • When package A requires package B (via install_requires) and package A absolutely pins the required version of package B using one or more trailing zeros (e.g. B==1.0.0) but the actual version number of package B (embedded in the metadata of package B) contains less trailing zeros (e.g. 1.0) then pip will not complain but silently fetch version 1.0 of package B to satisfy the requirement.
  • However this doesn’t change the absolutely pinned version in the install_requires metadata of package A.
  • When py2deb converts the resulting requirement set, the dependency of package A is converted as B (= 1.0.0). The resulting packages will not be installable because apt considers 1.0 to be different from 1.0.0.

This method analyzes the requirement set to identify occurrences of this quirk and strip trailing zeros in install_requires metadata that would otherwise result in converted packages that cannot be installed.

py2deb.hooks

The py2deb.hooks module contains post-installation and pre-removal hooks.

This module is a bit special in the sense that it is a part of the py2deb code base but it is embedded in the Debian binary packages generated by py2deb as a post-installation and pre-removal hook.

Because this module is embedded in generated packages it can’t use the external dependencies of the py2deb project, it needs to restrict itself to Python’s standard library.

My reasons for including this Python script as a “proper module” inside the py2deb project:

  • It encourages proper documentation of the functionality in this module, which enables users to read through the documentation without having to dive into py2deb’s source code.
  • It makes it easier to unit test the individual functions in this script without jumping through too many hoops (I greatly value test suite coverage).

The generate_maintainer_script() method is responsible for converting this module into a post-installation or pre-removal script. It does so by reading this module’s source code and appending a call to post_installation_hook() or pre_removal_hook() at the bottom.

py2deb.hooks.post_installation_hook(package_name, alternatives, modules_directory, namespaces, namespace_style)[source]

Generic post-installation hook for packages generated by py2deb.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).
  • namespace_style – The style of namespaces being used (one of the strings returned by namespace_style).

Uses the following functions to implement everything py2deb needs from the post-installation maintainer script:

py2deb.hooks.pre_removal_hook(package_name, alternatives, modules_directory, namespaces)[source]

Generic pre-removal hook for packages generated by py2deb.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by py2deb.package.PackageToConvert.namespaces).

Uses the following functions to implement everything py2deb needs from the pre-removal maintainer script:

py2deb.hooks.initialize_logging()[source]

Initialize logging to the terminal and apt log files.

py2deb.hooks.find_installed_files(package_name)[source]

Find the files installed by a Debian system package.

Parameters:package_name – The name of the system package (a string).
Returns:A list of absolute filenames (strings).

Uses the dpkg -L command.

py2deb.hooks.generate_bytecode_files(package_name, installed_files)[source]

Generate Python byte code files for the *.py files installed by a package.

Parameters:
  • package_name – The name of the system package (a string).
  • installed_files – A list of strings with the absolute pathnames of installed files.

Uses py_compile.compile() to generate bytecode files.

py2deb.hooks.cleanup_bytecode_files(package_name, installed_files)[source]

Cleanup Python byte code files generated when a package was installed.

Parameters:
  • package_name – The name of the system package (a string).
  • installed_files – A list of strings with the absolute pathnames of installed files.
py2deb.hooks.cleanup_bytecode_helper(filenames)[source]

Cleanup Python byte code files.

Parameters:filenames – A list of strings with the absolute pathnames of installed files.
Returns:The number of files that were removed (an integer).
py2deb.hooks.remove_empty_directory(directory)[source]

Remove a directory if it is empty.

Parameters:directory – The pathname of the directory (a string).
py2deb.hooks.find_bytecode_files(python_file)[source]

Find the byte code file(s) generated from a Python file.

Parameters:python_file – The pathname of a *.py file (a string).
Returns:A generator of pathnames (strings).

Starting from Python 3.2 byte code files are written according to PEP 3147 which also defines imp.cache_from_source() to locate (optimized) byte code files. When this function is available it is used, when it’s not available the corresponding *.pyc and/or *.pyo files are located manually by find_bytecode_files().

py2deb.hooks.create_alternatives(package_name, alternatives)[source]

Use update-alternatives to install a global symbolic link to a program.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.

Install a program available inside the custom installation prefix in the system wide executable search path using the Debian alternatives system.

py2deb.hooks.cleanup_alternatives(package_name, alternatives)[source]

Cleanup the alternatives that were previously installed by create_alternatives().

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
py2deb.hooks.initialize_namespaces(package_name, modules_directory, namespaces, namespace_style)[source]

Initialize Python namespace packages so they can be imported in the normal way.

Both pkgutil-style and pkg_resources-style namespace packages are supported (although support for the former was added in 2020 whereas support for the latter has existed since 2015).

Parameters:
  • package_name – The name of the system package (a string).
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).
  • namespace_style – The style of namespaces being used (one of the strings returned by namespace_style).
py2deb.hooks.cleanup_namespaces(package_name, modules_directory, namespaces)[source]

Clean up Python namespace packages previously initialized using initialize_namespaces().

Parameters:
  • package_name – The name of the system package (a string).
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).
class py2deb.hooks.NameSpaceReferenceCount(modules_directory)[source]

Persistent reference counting for initialization of namespace packages.

__init__(modules_directory)[source]

Initialize a NameSpaceReferenceCount object.

Parameters:modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
__enter__()[source]

Load the persistent data file (if it exists).

__exit__(exc_type=None, exc_value=None, traceback=None)[source]

Save the persistent data file.

__getitem__(key)[source]

Get the reference count of a namespace (defaults to zero).

__setitem__(key, value)[source]

Set the reference count of a namespace.

py2deb.namespaces

Python package namespace auto detection.

This module is used by py2deb to detect pkgutil-style namespace packages to enable special handling of the __init__.py files involved, because these would otherwise cause dpkg file conflicts.

Note

The __init__.py files that define pkgutil-style namespace packages can contain arbitrary Python code (including comments and with room for minor differences in coding style) which makes reliable identification harder than it should be. We use ast.parse() to look for hints and only when we find enough hints do we consider a module to be part of a pkgutil-style namespace package.

py2deb.namespaces.find_pkgutil_namespaces(directory)[source]

Find the pkgutil-style namespace packages in an unpacked Python distribution archive.

Parameters:directory – The pathname of a directory containing an unpacked Python distribution archive (a string).
Returns:A generator of dictionaries similar to those returned by find_python_modules().

This function combines find_python_modules() and find_pkgutil_ns_hints() to make it easy for callers to identify the namespace packages defined by an unpacked Python distribution archive.

py2deb.namespaces.find_pkgutil_ns_hints(tree)[source]

Analyze an AST for hints that we’re dealing with a Python module that defines a pkgutil-style namespace package.

Parameters:tree – The result of ast.parse() when run on a Python module (which is assumed to be an __init__.py file).
Returns:A set of strings where each string represents a hint (an indication) that we’re dealing with a pkgutil-style namespace module. No single hint can definitely tell us, but a couple of unique hints taken together should provide a reasonable amount of confidence (at least this is the idea, how well this works in practice remains to be seen).
py2deb.namespaces.find_python_modules(directory)[source]

Find the Python modules in an unpacked Python distribution archive.

Parameters:directory – The pathname of a directory containing an unpacked Python distribution archive (a string).
Returns:A list of dictionaries with the following key/value pairs:
  • abspath gives the absolute pathname of a Python module (a string).
  • relpath gives the pathname of a Python module (a string) relative to the intended installation directory.
  • name gives the dotted name of a Python module (a string).

This function works as follows:

  1. Use os.walk() to recursively search for __init__.py files in the directory given by the caller and collect the relative pathnames of the directories containing the __init__.py files.
  2. Use os.path.commonprefix() to determine the common prefix of the resulting directory pathnames.
  3. Use os.path.split() to partition the common prefix into an insignificant part (all but the final pathname component) and the significant part (the final pathname component).
  4. Strip the insignificant part of the common prefix from the directory pathnames we collected in step 1.
  5. Replace os.sep occurrences with dots to convert (what remains of) the directory pathnames to “dotted paths”.

py2deb.package

The py2deb.package module contains the low level conversion logic.

This module defines the PackageToConvert class which implements the low level logic of converting a single Python package to a Debian package. The separation between the PackageConverter and PackageToConvert classes is somewhat crude (because neither class can work without the other) but the idea is to separate the high level conversion logic from the low level conversion logic.

class py2deb.package.PackageToConvert(converter, requirement)[source]

Abstraction for Python packages to be converted to Debian packages.

Contains a pip_accel.req.Requirement object, has a back reference to the PackageConverter and provides all of the Debian package metadata implied by the Python package metadata.

Here’s an overview of the PackageToConvert class:

Superclass: PropertyManager
Special methods: __init__() and __str__()
Public methods: convert(), determine_package_architecture(), find_egg_info_file(), generate_maintainer_script(), load_control_field_overrides(), transform_binary_dist() and update_shebang()
Properties: debian_dependencies, debian_description, debian_maintainer, debian_name, debian_provides, debian_version, existing_archive, has_custom_install_prefix, metadata, namespace_packages, namespace_style, namespaces, pkgutil_namespaces, python_name, python_requirements, python_requirements_fallback, python_version, setuptools_namespaces and vcs_revision
__init__(converter, requirement)[source]

Initialize a package to convert.

Parameters:
debian_dependencies[source]

Find Debian dependencies of Python package.

Converts Python version specifiers to Debian package relationships.

Returns:A list with Debian package relationships (strings) in the format of the Depends: line of a Debian package control file. Based on python_requirements.

Note

The debian_dependencies property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debian_description[source]

Get a minimal description for the converted Debian package.

Includes the name of the Python package and the date at which the package was converted.

Note

The debian_description property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debian_maintainer[source]

Get the package maintainer name and e-mail address.

The name and e-mail address are combined into a single string that can be embedded in a Debian package (in the format name <email>). The metadata is retrieved as follows:

  1. If the environment variable $DEBFULLNAME is defined then its value is taken to be the name of the maintainer (this logic was added in #25). If $DEBEMAIL is set as well that will be incorporated into the result.
  2. The Python package maintainer name and email address are looked up in the package metadata and if found these are used.
  3. The Python package author name and email address are looked up in the package metadata and if found these are used.
  4. Finally if all else fails the text “Unknown” is returned.

Note

The debian_maintainer property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debian_name[source]

The name of the converted Debian package (a string).

Note

The debian_name property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debian_provides[source]

A symbolic name for the role the package provides (a string).

When a Python package provides “extras” those extras are encoded into the name of the generated Debian package, to represent the additional dependencies versus the package without extras.

However the package including extras definitely also satisfies a dependency on the package without extras, so a Provides: ... control field is added to the Debian package that contains the converted package name without extras.

Note

The debian_provides property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debian_version[source]

The version of the Debian package (a string).

Reformats python_version using normalize_package_version().

Note

The debian_version property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

existing_archive[source]

Find *.deb archive for current package name and version.

Returns:The pathname of the found archive (a string) or None if no existing archive is found.

Note

The existing_archive property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

has_custom_install_prefix[source]

Check whether package is being installed under custom installation prefix.

Returns:True if the package is being installed under a custom installation prefix, False otherwise.

A custom installation prefix is an installation prefix whose bin directory is (likely) not available on the default executable search path (the environment variable $PATH).

Note

The has_custom_install_prefix property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

metadata[source]

Get the Python package metadata.

The metadata is loaded from the PKG-INFO file generated by pip when it unpacked the source distribution archive. Results in a pkginfo.UnpackedSDist object.

Note

The metadata property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

namespace_packages[source]

Get the Python namespace packages defined by the Python package.

Returns:A list of dotted names (strings).

When setuptools_namespaces is available that will be used, otherwise we fall back to pkgutil_namespaces. This order of preference may be switched in the future, but not until pkgutil_namespaces has seen more thorough testing:

  • Support for setuptools_namespaces was added to py2deb in release 0.22 (2015) so this is fairly mature code that has seen thousands of executions between 2015-2020.
  • Support for pkgutil_namespaces was added in August 2020 so this is new (and complicated) code that hasn’t seen a lot of use yet. Out of conservativeness on my part this is nested in the ‘else’ branch (to reduce the scope of potential regressions).

Additionally computing setuptools_namespaces is very cheap (all it has to do is search for and read one text file) compared to pkgutil_namespaces (which needs to recursively search a directory tree for __init__.py files and parse each file it finds to determine whether it’s relevant).

Note

The namespace_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

namespace_style[source]

Get the style of Python namespace packages in use by this package.

Returns:One of the strings pkgutil, setuptools or none.

Note

The namespace_style property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

namespaces[source]

Get the Python namespace packages defined by the Python package.

Returns:A list of unique tuples of strings. The tuples are sorted by increasing length (the number of strings in each tuple) so that e.g. zope is guaranteed to sort before zope.app.

This property processes the result of namespace_packages into a more easily usable format. Here’s an example of the difference between namespace_packages and namespaces:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> package = next(converter.get_source_distributions(['zope.app.cache']))
>>> package.namespace_packages
['zope', 'zope.app']
>>> package.namespaces
[('zope',), ('zope', 'app')]

The value of this property is used by initialize_namespaces() and cleanup_namespaces() during installation and removal of the generated package.

Note

The namespaces property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

pkgutil_namespaces[source]

Namespace packages declared through pkgutil.

Returns:A list of dictionaries similar to those returned by find_pkgutil_namespaces().

For details about this type of namespace packages please refer to <https://packaging.python.org/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages>.

The implementation of this property lives in a separate module (refer to find_pkgutil_namespaces()) in order to compartmentalize the complexity of reliably identifying namespace packages defined using pkgutil.

Note

The pkgutil_namespaces property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

python_name

The name of the Python package (a string).

python_requirements[source]

Find the installation requirements of the Python package.

Returns:A list of pkg_resources.Requirement objects.

This property used to be implemented by manually parsing the requires.txt file generated by pip when it unpacks a distribution archive.

While this implementation was eventually enhanced to supported named extras, it never supported environment markers.

Since then this property has been reimplemented to use pkg_resources.Distribution.requires() so that environment markers are supported.

If the new implementation fails the property falls back to the old implementation (as a precautionary measure to avoid unexpected side effects of the new implementation).

Note

The python_requirements property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

python_requirements_fallback[source]

Fall-back implementation of python_requirements.

Note

The python_requirements_fallback property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

python_version

The version of the Python package (a string).

setuptools_namespaces[source]

Namespace packages declared through setuptools.

Returns:A list of dotted names (strings).

For details about this type of namespace packages please refer to <https://packaging.python.org/guides/packaging-namespace-packages/#pkg-resources-style-namespace-packages>.

Note

The setuptools_namespaces property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

vcs_revision[source]

The VCS revision of the Python package.

This works by parsing the .hg_archival.txt file generated by the hg archive command so for now this only supports Python source distributions exported from Mercurial repositories.

Note

The vcs_revision property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

convert()[source]

Convert current package from Python package to Debian package.

Returns:The pathname of the generated *.deb archive.
determine_package_architecture(has_shared_object_files)[source]

Determine binary architecture that Debian package should be tagged with.

Parameters:has_shared_objects – True if the package contains *.so files, False otherwise.
Returns:The architecture string, ‘all’ or one of the values of debian_architecture.

If a package contains *.so files we’re dealing with a compiled Python module. To determine the applicable architecture, we take the Debian architecture reported by debian_architecture.

find_egg_info_file(pattern='')[source]

Find pip metadata files in unpacked source distributions.

Parameters:pattern – The glob pattern to search for (a string).
Returns:A list of matched filenames (strings).

When pip unpacks a source distribution archive it creates a directory pip-egg-info which contains the package metadata in a declarative and easy to parse format. This method finds such metadata files.

generate_maintainer_script(filename, python_executable, function, **arguments)[source]

Generate a post-installation or pre-removal maintainer script.

Parameters:
  • filename – The pathname of the maintainer script (a string).
  • python_executable – The absolute pathname of the Python interpreter on the target system (a string).
  • function – The name of the function in the py2deb.hooks module to be called when the maintainer script is run (a string).
  • arguments – Any keyword arguments to the function in the py2deb.hooks are serialized to text using repr() and embedded inside the generated maintainer script.
load_control_field_overrides(control_fields)[source]

Apply user defined control field overrides.

Parameters:control_fields – The control field defaults constructed by py2deb (a deb_pkg_tools.deb822.Deb822 object).
Returns:The merged defaults and overrides (a deb_pkg_tools.deb822.Deb822 object).

Looks for an stdeb.cfg file inside the Python package’s source distribution and if found it merges the overrides into the control fields that will be embedded in the generated Debian binary package.

This method first applies any overrides defined in the DEFAULT section and then it applies any overrides defined in the section whose normalized name (see package_names_match()) matches that of the Python package.

transform_binary_dist(interpreter)[source]

Build Python package and transform directory layout.

Parameters:interpreter – The absolute pathname of the Python interpreter that should be referenced by executable scripts in the binary distribution (a string).
Returns:An iterable of tuples with two values each:
  1. A tarfile.TarInfo object;
  2. A file-like object.

Builds the Python package (using pip-accel) and changes the names of the files included in the package to match the layout corresponding to the given conversion options.

update_shebang(handle, interpreter)[source]

Update the shebang of executable scripts.

Parameters:
  • handle – A file-like object containing an executable.
  • interpreter – The absolute pathname of the Python interpreter that should be referenced by the script (a string).
Returns:

A file-like object.

Normally pip-accel is responsible for updating interpreter references in executable scripts, however there’s a bug in pip-accel where it assumes that the string ‘python’ will appear literally in the shebang (which isn’t true when running on PyPy).

Note

Of course this bug should be fixed in pip-accel however that project is in limbo while I decide whether to reinvigorate or kill it (the second of which implies needing to make a whole lot of changes to py2deb).

__str__()[source]

The name, version and extras of the package encoded in a human readable string.

py2deb.tests

The py2deb.tests module contains the automated tests for py2deb.

The makefile in the py2deb git repository uses pytest to run the test suite because of pytest’s great error reporting. Nevertheless the test suite is written to be compatible with the unittest module (part of Python’s standard library) so that the test suite can be run without additional external dependencies.

py2deb.tests.execute()

Execute an external command and make sure it succeeded.

Parameters:
  • command – All positional arguments are passed on to the constructor of ExternalCommand.
  • options – All keyword arguments are passed on to the constructor of ExternalCommand.
Returns:

Refer to execute_prepared().
Raises:

ExternalCommandFailed when the command exits with a nonzero exit code (and check is True).

If asynchronous is True then execute() will automatically start the external command for you using start() (but it won’t wait for it to end). If you want to create an ExternalCommand object instance without immediately starting the external command then you can use ExternalCommand directly.

Some examples

By default the status code of the external command is returned as a boolean:

>>> from executor import execute
>>> execute('true')
True

However when an external command exits with a nonzero status code an exception is raised, this is intended to “make it easy to do the right thing” (never forget to check the status code of an external command without having to write a lot of repetitive code):

>>> execute('false')
Traceback (most recent call last):
  File "executor/__init__.py", line 124, in execute
    cmd.start()
  File "executor/__init__.py", line 516, in start
    self.wait()
  File "executor/__init__.py", line 541, in wait
    self.check_errors()
  File "executor/__init__.py", line 568, in check_errors
    raise ExternalCommandFailed(self)
executor.ExternalCommandFailed: External command failed with exit code 1! (command: false)

What’s also useful to know is that exceptions raised by execute() expose command and returncode attributes. If you know a command is likely to exit with a nonzero status code and you want execute() to simply return a boolean you can do this instead:

>>> execute('false', check=False)
False
py2deb.tests.setUpModule()[source]

Prepare the test suite.

This function does two things:

  1. Sets up verbose logging to the terminal. When a test fails the logging output can help to perform a post-mortem analysis of the failure in question (even when its hard to reproduce locally). This is especially useful when debugging remote test failures, whether they happened on Travis CI or a user’s local system.
  2. Creates temporary directories where the pip download cache and the pip-accel binary cache are located. Isolating the pip-accel binary cache from the user’s system is meant to ensure that the tests are as independent from the user’s system as possible. The function tearDownModule() is responsible for cleaning up the temporary directory after the test suite finishes.
py2deb.tests.tearDownModule()[source]

Clean up temporary directories created by setUpModule().

py2deb.tests.create_temporary_directory()[source]

Create a temporary directory for the test suite to use.

The created temporary directory will be cleaned up by tearDownModule() when the test suite is being torn down.

Returns:The pathname of the created temporary directory (a string).
class py2deb.tests.PackageConverterTestCase(*args, **kw)[source]

unittest compatible container for the test suite of py2deb.

create_isolated_converter()[source]

Instantiate an isolated package converter.

test_argument_validation()[source]

Test argument validation done by setters of py2deb.converter.PackageConverter.

test_version_reformatting()[source]

Test reformatting of Python version strings.

test_conversion_of_simple_package()[source]

Convert a simple Python package without any dependencies.

Converts coloredlogs and sanity checks the result. Performs several static checks on the metadata and contents of the resulting package archive.

test_custom_conversion_command()[source]

Convert a simple Python package that requires a custom conversion command.

Converts Fabric and sanity checks the result. For details please refer to py2deb.converter.PackageConverter.set_conversion_command().

test_duplicate_files_check()[source]

Ensure that py2deb checks for duplicate file conflicts within dependency sets.

Converts a version of Fabric that bundles Paramiko but also includes Paramiko itself in the dependency set, thereby causing a duplicate file conflict, to verify that py2deb recognizes duplicate file conflicts.

test_conversion_of_package_with_dependencies()[source]

Convert a non trivial Python package with several dependencies.

Converts deb-pkg-tools to a Debian package archive and sanity checks the result. Performs static checks on the metadata (dependencies) of the resulting package archive.

test_conversion_of_extras()[source]

Convert a package with extras.

Converts raven[flask]==3.6.0 and sanity checks the result.

test_conversion_of_environment_markers()[source]

Convert a package with installation requirements using environment markers.

Converts weasyprint==0.42 and sanity checks that the cairosvg dependency is present.

test_python_requirements_fallback()[source]

Test the fall-back implementation of the python_requirements property.

test_namespace_package_parsing()[source]

Test parsing of namespace_package.txt files.

test_conversion_of_binary_package()[source]

Convert a package that includes a *.so file (a shared object file).

Converts setproctitle==1.1.8 and sanity checks the result. The goal of this test is to verify that pydeb properly handles packages with binary components (including dpkg-shlibdeps magic). This explains why I chose the setproctitle package:

  1. This package is known to require a compiled shared object file for proper functioning.
  2. Despite requiring a compiled shared object file the package is fairly lightweight and has little dependencies so including this test on every run of the test suite won’t slow things down so much that it becomes annoying.
  3. The package is documented to support Python 3.x as well which means we can run this test on all supported Python versions.
test_converted_package_installation()[source]

Install a converted package on the test system and verify that it works.

This test only runs on Travis CI, it’s a functional test that uses py2deb to convert a Python package to a Debian package, installs that package on the local system and verifies that the system wide Python installation can successfully import the installed package.

test_conversion_of_binary_package_with_executable()[source]

Convert a package that includes a binary executable file.

Converts uwsgi==2.0.17.1 and sanity checks the result. The goal of this test is to verify that pydeb preserves binary executables instead of truncating them as it did until issue 9 was reported.

test_install_requires_version_munging()[source]

Convert a package with a requirement whose version is “munged” by pip.

Refer to py2deb.converter.PackageConverter.transform_version() for details about the purpose of this test.

test_conversion_with_system_package()[source]

Convert a package and map one of its requirements to a system package.

test_conversion_of_isolated_packages()[source]

Convert a group of packages with a custom name and installation prefix.

Converts pip-accel and its dependencies to a group of “isolated Debian packages” that are installed with a custom name prefix and installation prefix and sanity check the result. Also tests the --rename=FROM,TO command line option. Performs static checks on the metadata and contents of the resulting package archive.

test_conversion_with_configuration_file()[source]

Convert a group of packages based on the settings in a configuration file.

Repeats the same test as test_conversion_of_isolated_packages() but instead of using command line options the conversion process is configured using a configuration file.

check_converted_pip_accel_packages(directory)[source]

Check a group of packages converted with a custom name and installation prefix.

Check the results of test_conversion_of_isolated_packages() and test_conversion_with_configuration_file().

test_python_callback_from_api()[source]

Test Python callback logic (registered through the Python API).

test_python_callback_from_dotted_path()[source]

Test Python callback logic (through a dotted path expression).

test_python_callback_from_filename()[source]

Test Python callback logic (through a filename expression).

check_python_callback(expression)[source]

Test for Python callback logic manipulating the build of a package.

test_find_installed_files()[source]

Test the py2deb.hooks.find_installed_files() function.

test_bytecode_generation()[source]

Test byte code generation and cleanup.

This tests the generate_bytecode_files() and cleanup_bytecode_files() functions.

test_namespace_initialization()[source]

Test namespace package initialization and cleanup.

This tests the initialize_namespaces() and cleanup_namespaces() functions.

test_pkgutil_namespaces()[source]

Test compatibility with pkgutil style namespace packages.

This test fails on py2deb <= 4.0 because the two packages involved both define the same pkgutil-style namespace package and this causes a file conflict that’s detected by py2deb, in the form of a DuplicateFilesFound exception:

deb_pkg_tools.checks.DuplicateFilesFound: Found 1 duplicate file in 2 package archives!
-------------------------------------------------------------------------------
Found 1 conflict between 2 packages:
  1. /tmp/tmpgqz6ettd/python3-backports-functools-lru-cache_1.6.1_all.deb
  2. /tmp/tmpgqz6ettd/python3-configparser_3.7.4_all.deb
These packages contain 1 conflict:
  1. /usr/lib/python3.6/dist-packages/backports/__init__.py
-------------------------------------------------------------------------------
test_post_install_hook()[source]

Test the post_installation_hook() function.

test_pre_removal_hook()[source]

Test the pre_removal_hook() function.

run_post_install_hook(directory, namespace_style)[source]

Helper for test_post_install_hook() and test_pre_removal_hook().

check_test_namespaces(directory)[source]

Make sure the test name spaces are properly initialized.

py2deb.tests.find_package_archive(available_archives, package_name)[source]

Find the *.deb archive of a specific package.

Parameters:
  • available_packages – The pathnames of the available package archives (a list of strings).
  • package_name – The name of the package whose archive file we’re interested in (a string).
Returns:

The pathname of the package archive (a string).
Raises:

exceptions.AssertionError if zero or more than one package archive is found.
py2deb.tests.find_file(contents, pattern)[source]

Find the file matching the given filename pattern.

Searches the dictionary of Debian package archive entries reported by deb_pkg_tools.package.inspect_package().

Parameters:
  • contents – The dictionary of package archive entries.
  • pattern – The filename pattern to match (fnmatch syntax).
Returns:

The metadata of the matched file.
Raises:

exceptions.AssertionError if zero or more than one archive entry is found.
py2deb.tests.fix_name_prefix(name)[source]

Change the name prefix of a Debian package to match the current Python version.

py2deb.tests.python_callback_fn(converter, package, build_directory)[source]

Simple Python function to test support for callbacks.

py2deb.utils

The py2deb.utils module contains miscellaneous code.

py2deb.utils.integer_pattern = <_sre.SRE_Pattern object>

Compiled regular expression to match a consecutive run of digits.

py2deb.utils.PYTHON_EXECUTABLE_PATTERN = <_sre.SRE_Pattern object>

A compiled regular expression to match Python interpreter executable names.

The following are examples of program names that match this pattern:

  • pypy
  • pypy2.7
  • pypy3
  • python
  • python2
  • python2.7
  • python3m
class py2deb.utils.PackageRepository(directory)[source]

Very simply abstraction for a directory containing *.deb archives.

Used by py2deb.converter.PackageConverter to recognize which Python packages have previously been converted (and so can be skipped).

Here’s an overview of the PackageRepository class:

Superclass: PropertyManager
Special methods: __init__()
Public methods: get_package()
Properties: archives and directory

When you initialize a PackageRepository object you are required to provide a value for the directory property. You can set the value of the directory property by passing a keyword argument to the class initializer.

__init__(directory)[source]

Initialize a PackageRepository object.

Parameters:directory – The pathname of a directory containing *.deb archives (a string).
archives[source]

A sorted list of package archives in directory.

The value of archives is computed using deb_pkg_tools.package.find_package_archives().

An example:

>>> from py2deb import PackageRepository
>>> repo = PackageRepository('/tmp')
>>> repo.archives
[PackageFile(name='py2deb', version='0.1', architecture='all',
             filename='/tmp/py2deb_0.1_all.deb'),
 PackageFile(name='py2deb-cached-property', version='0.1.5', architecture='all',
             filename='/tmp/py2deb-cached-property_0.1.5_all.deb'),
 PackageFile(name='py2deb-chardet', version='2.2.1', architecture='all',
             filename='/tmp/py2deb-chardet_2.2.1_all.deb'),
 PackageFile(name='py2deb-coloredlogs', version='0.5', architecture='all',
             filename='/tmp/py2deb-coloredlogs_0.5_all.deb'),
 PackageFile(name='py2deb-deb-pkg-tools', version='1.20.4', architecture='all',
             filename='/tmp/py2deb-deb-pkg-tools_1.20.4_all.deb'),
 PackageFile(name='py2deb-docutils', version='0.11', architecture='all',
             filename='/tmp/py2deb-docutils_0.11_all.deb'),
 PackageFile(name='py2deb-executor', version='1.2', architecture='all',
             filename='/tmp/py2deb-executor_1.2_all.deb'),
 PackageFile(name='py2deb-html2text', version='2014.4.5', architecture='all',
             filename='/tmp/py2deb-html2text_2014.4.5_all.deb'),
 PackageFile(name='py2deb-humanfriendly', version='1.8.2', architecture='all',
             filename='/tmp/py2deb-humanfriendly_1.8.2_all.deb'),
 PackageFile(name='py2deb-pkginfo', version='1.1', architecture='all',
             filename='/tmp/py2deb-pkginfo_1.1_all.deb'),
 PackageFile(name='py2deb-python-debian', version='0.1.21-nmu2', architecture='all',
             filename='/tmp/py2deb-python-debian_0.1.21-nmu2_all.deb'),
 PackageFile(name='py2deb-six', version='1.6.1', architecture='all',
             filename='/tmp/py2deb-six_1.6.1_all.deb')]

Note

The archives property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

directory[source]

The pathname of a directory containing *.deb archives (a string).

Note

The directory property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named directory (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

get_package(package, version, architecture)[source]

Find a package in the repository.

Parameters:
  • package – The name of the package (a string).
  • version – The version of the package (a string).
  • architecture – The architecture of the package (a string).
Returns:

A deb_pkg_tools.package.PackageFile object or None.

Here’s an example:

>>> from py2deb import PackageRepository
>>> repo = PackageRepository('/tmp')
>>> repo.get_package('py2deb', '0.1', 'all')
PackageFile(name='py2deb', version='0.1', architecture='all', filename='/tmp/py2deb_0.1_all.deb')
class py2deb.utils.TemporaryDirectory(**options)[source]

Easy temporary directory creation & cleanup using the with statement.

Here’s an example of how to use this:

with TemporaryDirectory() as directory:
    # Do something useful here.
    assert os.path.isdir(directory)
__init__(**options)[source]

Initialize context manager that manages creation & cleanup of temporary directory.

Parameters:options – Any keyword arguments are passed on to tempfile.mkdtemp().
__enter__()[source]

Create the temporary directory.

__exit__(exc_type, exc_value, traceback)[source]

Destroy the temporary directory.

py2deb.utils.compact_repeating_words(words)[source]

Remove adjacent repeating words.

Parameters:words – An iterable of words (strings), assumed to already be normalized (lowercased).
Returns:An iterable of words with adjacent repeating words replaced by a single word.

This is used to avoid awkward word repetitions in the package name conversion algorithm. Here’s an example of what I mean:

>>> from py2deb import compact_repeating_words
>>> name_prefix = 'python'
>>> package_name = 'python-mcrypt'
>>> combined_words = [name_prefix] + package_name.split('-')
>>> print(list(combined_words))
['python', 'python', 'mcrypt']
>>> compacted_words = compact_repeating_words(combined_words)
>>> print(list(compacted_words))
['python', 'mcrypt']
py2deb.utils.convert_package_name(python_package_name, name_prefix=None, extras=())[source]

Convert a Python package name to a Debian package name.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • name_prefix – The name prefix to apply (a string or None, in which case the result of default_name_prefix() is used instead).
Returns:

A Debian package name (a string).
py2deb.utils.default_name_prefix()[source]

Get the default package name prefix for the Python version we’re running.

Returns:One of the strings python, python3 or pypy.
py2deb.utils.detect_python_script(handle)[source]

Detect whether a file-like object contains an executable Python script.

Parameters:handle – A file-like object (assumed to contain an executable).
Returns:True if the program name in the shebang of the script references a known Python interpreter, False otherwise.
py2deb.utils.embed_install_prefix(handle, install_prefix)[source]

Embed Python snippet that adds custom installation prefix to module search path.

Parameters:
  • handle – A file-like object containing an executable Python script.
  • install_prefix – The pathname of the custom installation prefix (a string).
Returns:

A file-like object containing the modified Python script.
py2deb.utils.extract_shebang_command(handle)[source]

Extract the shebang command line from an executable script.

Parameters:handle – A file-like object (assumed to contain an executable).
Returns:The command in the shebang line (a string).

The seek position is expected to be at the start of the file and will be reset afterwards, before this function returns. It is not an error if the executable contains binary data.

py2deb.utils.extract_shebang_program(command)[source]

Extract the program name from a shebang command line.

Parameters:command – The result of extract_shebang_command().
Returns:The program name in the shebang command line (a string).
py2deb.utils.normalize_package_name(python_package_name)[source]

Normalize Python package name to be used as Debian package name.

Parameters:python_package_name – The name of a Python package as found on PyPI (a string).
Returns:The normalized name (a string). 
>>> from py2deb import normalize_package_name
>>> normalize_package_name('MySQL-python')
'mysql-python'
>>> normalize_package_name('simple_json')
'simple-json'
py2deb.utils.normalize_package_version(python_package_version, prerelease_workaround=True)[source]

Normalize Python package version to be used as Debian package version.

Parameters:
  • python_package_version – The version of a Python package (a string).
  • prerelease_workaround – True to enable the pre-release handling documented below, False to restore the old behavior.

Reformats Python package versions to comply with the Debian policy manual. All characters except alphanumerics, dot (.) and plus (+) are replaced with dashes (-).

The PEP 440 pre-release identifiers ‘a’, ‘b’, ‘c’ and ‘rc’ are prefixed by a tilde (~) to replicate the intended ordering in Debian versions, also the identifier ‘c’ is translated into ‘rc’. Refer to issue #8 for details.

py2deb.utils.package_names_match(a, b)[source]

Check whether two Python package names are equal.

Parameters:
  • a – The name of the first Python package (a string).
  • b – The name of the second Python package (a string).
Returns:

True if the package names match, False if they don’t.

Uses normalize_package_name() to normalize both names before comparing them for equality. This makes sure differences in case and dashes versus underscores are ignored.

py2deb.utils.python_version()[source]

Find the version of Python we’re running.

Returns:A string like python2.7, python3.8, pypy or pypy3.

This specifically returns a name that matches both of the following:

  • The name of the Debian package providing the current Python version.
  • The name of the interpreter executable for the current Python version.
py2deb.utils.tokenize_version(version_number)[source]

Tokenize a string containing a version number.

Parameters:version_number – The string to tokenize.
Returns:A list of strings.

Change log

The change log lists notable changes to the project:

Changelog

The purpose of this document is to list all of the notable changes to this project. The format was inspired by Keep a Changelog. This project adheres to semantic versioning.

Release 5.0 (2020-08-05)

  • Added support for pkgutil style namespace packages. This should be considered experimental because it hasn’t seen any real world use yet.
  • Explicitly documented Python compatibility in the readme (see also #17, #18, #27 and #31) to avoid more issues being reported about Python 3.8+ not being supported.

Release 4.0 (2020-08-04)

Note

While I don’t consider this a major release feature wise, the major version number was bumped because this change is backwards incompatible (although clearly an improvement).

Merged pull request #22 to stop py2deb from normalizing “local version labels” as defined by PEP 440. One important thing to note is that the “Debian revision” safe guard is applied after the “local version label” is restored, which means the “local version label” may not be the final part of the generated Debian version string.

Release 3.2 (2020-08-04)

Merged pull request #25 which adds support for the $DEBFULLNAME and $DEBEMAIL environment variables to override package maintainer metadata.

Release 3.1 (2020-08-04)

Merged pull request #20 which adds a Provides Debian control field for converted packages that have “extras” encoded in their name.

One caveat to point out: Provides is a second-class citizen in the Debian packaging ecosystem in the sense that it satisfies only unversioned relationships.

Nevertheless this may prove useful, so it was merged 🙂.

Release 3.0.1 (2020-08-04)

Release 3.0 was yanked from PyPI just minutes after uploading, because I forgot to include a python_requires definition in the setup.py script, which means Python 2.6 and 3.4 installations could end up downloading incompatible py2deb releases. This has since been added.

Release 3.0 (2020-08-04)

Note

While I don’t consider this a major release feature wise, the major version number was bumped because of the compatibility changes (dropping 2.6 and 3.4).

Updated compatibility:

  • PyPy 3 is now officially supported (and tested on Travis CI). This was triggered by pull requests #29 and #30.
  • Python 2.6 and 3.4 are no longer supported (nor tested on Travis CI) following the same change in my other 20+ open source Python projects (some of which are requirements of py2deb).

Project maintenance:

  • Spent several days stabilizing the test suite on Travis CI, to avoid finding myself in a situation where I’m releasing new features without the safety net provided by a test suite that runs automatically and shouts loudly when breakage is found 😇.
  • Spent several days getting PyPy 3 testing to work on Travis CI, due to fatal incompatibilities between the most recent release of pip and PyPy 3. For more then you ever wanted to know consult these commits and the related Travis CI build failures (some of which are linked in commit messages).
  • Updated some imports to be compatible with humanfriendly 8.0.

Miscellaneous changes:

  • Merged pull request #21 which fixes a typo in the hooks module.

Release 2.3 (2020-07-28)

Merged pull request #30:

  • Added support pypy3 in replacement hashbangs.
  • Added support for pypy3 package name prefix.

Release 2.2 (2020-07-28)

Addded support for pypy3 hashbangs via pull request #29.

Release 2.1.1 (2020-05-26)

Defensively pin pip-accel requirement.

I intend to revive pip-accel based on the latest pip release, offering a minimal conceptual subset of previous functionality of pip-accel, just enough for py2deb to use for downloading and unpacking distribution archives.

However this will surely take some time to flesh out - possibly multiple releases of both projects. I’m not even sure yet what will be involved in getting pip-accel and py2deb running on the latest version of pip (I can however already tell that large architectural changes will be required in pip-accel and consequently also py2deb).

In the mean time I don’t want any users (including my employer) run into breakage caused by this endeavor. Alpha / beta releases on PyPI should be able to avoid this problem, however I’ve never published those myself, so I’m opting for “defense in depth” 😇.

Release 2.1 (2018-12-16)

Enable optional backwards compatibility with the old version number conversion up to release 0.25 in which pre-release identifiers didn’t receive any special treatment.

My reason for adding this backwards compatibility now is that it will allow me to upgrade py2deb on the build server of my employer to the latest version without being forced to switch to the new version number format at the same time. This simplifies the transition significantly.

Release 2.0 (2018-11-18)

New features:

  • Added support for Python 3.7 🎉 (configured Travis CI to run the test suite on Python 3.7 and updated the project metadata and documentation).
  • Added support for PyPy 🎉 (configured Travis CI to run the test suite on PyPy, changed the test suite to accommodate PyPy, fixed several incompatibilities in the code base, updated the project metadata and documentation).
  • Make it possible for callers to change Lintian overrides embedded in the generated binary packages. Also, update the default overrides.

Bug fixes:

  • Make the default name prefix conditional on the Python version that’s running py2deb (this is backwards incompatible although clearly the correct behavior):

    • On PyPy the default name prefix is now pypy.
    • On Python 2 the default name prefix is still python.
    • On Python 3 the default name prefix is now python3.

    The old behavior of using the python name prefix on Python 3 and PyPy was definitely wrong and quite likely could lead to serious breakage, but even so this change is of course backwards incompatible.

  • Don’t raise an exception from transform_version() when a partial requirement set is converted using pip’s --no-deps command line option (this is a valid use case that should be supported).

Documentation changes:

  • Added this changelog 🎉. The contents were generated by a Python script that collects tags and commit messages from the git repository. I manually summarized and converted the output to reStructuredText format (which was a whole lot work 😛).
  • Changed the theme of the documentation from classic to nature. The classic theme is heavily customized by Read the Docs whereas the nature theme more closely matches what is rendered locally by Sphinx versus what is rendered ‘remotely’ on Read the Docs.
  • Changed the location of the intersphinx mapping for setuptools (it now uses Read the Docs).

Internal improvements:

  • Move the finding of shared object files and the dpkg-shlibdeps integration to deb-pkg-tools (strictly speaking this is backwards incompatible). This functionality originated in py2deb but since then I’d wanted to reuse it outside of py2deb several times and so I eventually reimplemented it in deb-pkg-tools. Switching to that implementation now made sense (in order to reduce code duplication and simplify the py2deb code base). Strictly speaking this is backwards incompatible because methods have been removed but this only affects those who extend PackageToConvert which I don’t expect anyone to have actually done 🙂.
  • Switched from cached-property to property-manager. The py2deb project comes from a time (2013) when Python descriptors were still magic to me and so I chose to use cached-property. However since then I created the property-manager project (2015). At this point in time (2018) several of the dependencies of py2deb (other projects of mine) already use property-manager and the integration of property-manager in py2deb can help to improve the project, so this seemed like the logical choice 😇.

Release 1.1 (2018-02-24)

  • Add support for conditional dependencies via environment markers.
  • Include the documentation in source distributions (the *.tar.gz files).

Release 1.0 (2017-08-08)

  • Fixed issue #8: Support PEP 440 pre-release versions.
  • Document Python 3.6 support, configure Travis CI to test Python 3.6.
  • Merged pull request #11: Update comparison with fpm to remove invalid statement about the lack of support for converting multiple packages at once.

Since release 0.25 I’ve only made bug fixes (i.e. no features were added) however the change related to #8 is backwards incompatible, which is why I’ve decided to bump the major version number.

Release 0.25 (2017-05-23)

Make it possible to “replace” specific Python packages (installation requirements) with a user defined system package using the new command line option --use-system-package=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME.

The package PYTHON_PACKAGE_NAME will be excluded from the convertion process. Converted packages that depended on PYTHON_PACKAGE_NAME will have their dependencies updated to refer to DEBIAN_PACKAGE_NAME instead.

Release 0.24.4 (2017-01-17)

  • Fixed a bug in py2deb.utils.embed_install_prefix() (reported in issue #9 and fixed in pull request #10) that accidentally truncated binary executables when using a custom installation prefix.
  • Fixed a broken import in the documentation (reported in issue #6).
  • Added Python 3.5 to versions tested on Travis CI (but don’t look at the build logs just yet, for example Lintian complains with python-module-in-wrong-location, to be investigated if and how this can be ‘improved’).
  • Improved docs/conf.py and added humanfriendly.sphinx usage.
  • Refactored setup script (added docstring and classifiers) and Makefile and related files.

Release 0.24.3 (2016-04-15)

Refactor setup.py script, improving Python 3 support:

  • Counteract a possible UnicodeDecodeError when setup.py loads README.rst to populate the long_description field.
  • Could have fixed this with a two line diff, but noticed some other things I wanted to improve, so here we are 🙂.

Release 0.24.2 (2016-01-19)

Bug fix: Restore compatibility with latest coloredlogs (fixes #4).

Release 0.24.1 (2015-09-24)

Bug fix to restore Python 3 compatibility (execfile() versus exec).

Release 0.24 (2015-09-24)

Added support for Python callbacks that enable arbitrary manipulation during packaging.

Release 0.23.2 (2015-09-04)

  • Strip trailing zeros in required versions when necessary (improves compatibility with pip).
  • Document ideas for future improvements.

Release 0.23.1 (2015-06-28)

Moved usage message munging to humanfriendly package.

Release 0.23 (2015-04-22)

Make it possible to disable automatic Lintian checks.

Release 0.22 (2015-04-12)

  • Refactor maintainer scripts into a proper Python module:

    The post-installation and pre-removal scripts that py2deb bundled with generated Debian packages were lacking functionality and were not easy to extend. I’ve now refactored these scripts into a Python module with proper coding standards (documentation, tests, readable and maintainable code) and some additional features:

  • Use executor.quote() instead of pipes.quote().

  • Always clean up temporary directories created by pip and pip-accel.

  • Remove redundant temporary directory creation.

Release 0.21.1 (2015-04-05)

Update usage instructions in readme (and automate the process for the future).

Release 0.21 (2015-04-04)

Upgraded dependencies: pip-accel 0.25 and pip 6.

Release 0.20.11 (2015-03-18)

Switched to deb_pkg_tools.utils.find_debian_architecture().

Release 0.20.10 (2015-03-04)

Move control field override handling to separate, documented method.

Release 0.20.9 (2015-03-04)

Normalize package names during stdeb.cfg parsing.

Release 0.20.8 (2015-03-01)

  • Include a detailed comparison to stdeb, dh-virtualenv and fpm in the documentation (for details see #1).
  • Clarify in the readme that py2deb builds binary Debian packages and that Lintian is an optional dependency.

Release 0.20.7 (2015-03-01)

This was a “vanity release” that contained no code changes relevant to users: I’d finally gotten the full test suite to pass on Travis CI (see issue #3 for details) and I wanted to add badges to the readme 😇.

Release 0.20.6 (2015-03-01)

Improve PackageToConvert.determine_package_architecture().

In the previous release I added the armv6l to armhf mapping to PackageConverter and I just noticed that PackageToConvert didn’t respect this change.

I’m not sure why PackageConverter and PackageToConvert both ended up having separate ways to detect the current Debian architecture (I guess this was left over from a previous refactoring) but clearly this logic should be contained in a single place, not spread over multiple places like it was before this change.

Release 0.20.5 (2015-02-27)

  • Improved Python 3.4 compatibility, also bumped deb-pkg-tools requirement to improve Python 3 compatibility.
  • Replaced the use of uname -m with os.uname() and added an armv6l to armhf mapping (to enable support for Raspbian).
  • Start running the test suite on Travis CI against Python 2.6, 2.7 and 3.4 and collect coverage statistics on Coveralls.

Release 0.20.4 (2015-02-25)

Give up on conversion of package descriptions using docutils:

  1. It was always just a nice to have.
  2. I’m never going to get it working reliably.
  3. Right now it adds several “dead weight” dependencies (because the feature was disabled in release 0.18.6).
  4. This “dead code” was reducing test coverage.

This release was the first release to be published on PyPI.

Release 0.20.3 (2014-12-09)

Add a log message when the control field overrides file is not found.

Release 0.20.2 (2014-11-29)

Bug fix: Change initialization order.

Release 0.20.1 (2014-11-28)

Re-enable auto-install runtime/configuration option.

Release 0.20 (2014-11-28)

Upgraded to the newest pip-accel (0.19.2).

Release 0.19.1 (2014-11-18)

Release 0.19 (2014-11-12)

Load configuration files and environment variables by default (with an escape hatch should it ever turn out to be problematic 😇).

Release 0.18.9 (2014-11-09)

Upgrade to pip-accel 0.14.1.

Release 0.18.8 (2014-07-23)

Avoid Lintian complaining about debian-revision-should-not-be-zero.

Release 0.18.7 (2014-07-15)

Bug fix for custom installation prefix embedding in executable scripts.

Release 0.18.6 (2014-07-15)

Disable package description conversion until I find out what’s wrong with it:

  • Starting from release 0.16 py2deb would use docutils to convert the long_description of each Python package to HTML which was then translated to plain text in order to generate a readme text that was embedded in the metadata of the binary package.
  • However lots of packages on PyPI (including mine) automatically embed their README.rst as the long_description in the setup.py script, making for rather complex documents to transform.
  • This interaction caused “Unable to parse package file” warnings from apt-get during installation of packages (given input packages with complex enough long descriptions).

Given that this was a “nice to have” and I had more important things on my plate I decided to just disable this feature for now.

Release 0.18.5 (2014-07-15)

Bug fix: Make sure the “Debian revision” part of converted version numbers contains a digit.

Release 0.18.4 (2014-07-15)

Bug fix: Tildes in Debian binary package versions considered harmful!

Because of the special semantics of ~ in Debian binary pakcage versions I’ve decided to switch from ~ to - as the separator between tokens in the version string.

About those special semantics:

$ dpkg --compare-versions '0.21.1~paylogic' '>=' '0.21.1'; echo $?
1

$ dpkg --compare-versions '0.21.1~paylogic' '>=' '0.21.1'; echo $?
1

$ dpkg --compare-versions '0.21.1-paylogic' '>=' '0.21.1'; echo $?
0

$ dpkg --compare-versions '0.21.1-paylogic-0' '>=' '0.21.1'; echo $?
0

Release 0.18.3 (2014-07-15)

Bug fix: Cleanup temporary source directories.

These are created when you tell pip to install from a directory containing an unpacked source distribution: pip copies the complete directory to /tmp before doing anything with it, but because this directory cannot be set using --build-directory py2deb never cleaned up directories created in this manner.

Release 0.18.2 (2014-07-02)

Automatically add the Vcs-Hg control field when possible.

This works by parsing the .hg_archival.txt file generated by the hg archive command so for now this only supports Python source distributions exported from Mercurial repositories.

Release 0.18.1 (2014-06-27)

This release consists of more than 10 commits that were part of an effort to prepare the py2deb project for open sourcing under the name of Paylogic. Here’s a short summary:

  • Bumped pip-accel requirement (to pull in an upstream bug fix) and minor changes to be compatible with the new version.
  • Support for default configuration files (/etc/py2deb.ini and ~/.py2deb.ini).
  • Don’t copy files during builds (performance optimization).
  • Add logging in order to debug handling of postinst/prerm scripts.
  • Explicitly iterate postinst/prerm scripts (explicit is better than implicit).
  • Bug fix: Include postinst/prerm scripts during installation!
  • Bug fix: Reformat version strings to comply with Debian policy manual.
  • Make converter.convert() return list of generated package archives.
  • Check for duplicate files in converted dependency sets.
  • Improved the documentation.

Release 0.18 (2014-06-16)

This release consists of about 15 commits that were part of an effort to prepare the py2deb project for open sourcing under the name of Paylogic. Here’s a short summary:

  • Support for environment variables.
  • Make py2deb compatible with Python 3.4.
  • Explicitly document that py2deb invokes pip.
  • Improve PackageToConvert.python_requirements.
  • Improve PackageToConvert.debian_dependencies.
  • Rename find_package() to get_package().
  • Rename find_python_version() to python_version().
  • Improve compact_repeating_words().
  • Add comparison between py2deb and stdeb to readme.
  • Bring test coverage up to 92%.

Release 0.17 (2014-06-07)

This release consists of almost 50 commits that were part of an effort to prepare the py2deb project for open sourcing under the name of Paylogic. Here’s a short summary:

  • Implemented PEP-8 and PEP-257 compatibility and code style checks.
  • Implemented --report-dependencies option.
  • Encode Python requirement ‘extras’ in Debian package names.
  • Document the -- trick in the usage message.
  • Document several missing installation requirements.
  • Restore compatibility with stdeb.cfg configuration files (for now there’s no reason not to use the same file, since the file serves the exact same purpose - if and when I need non-compatible behavior I can switch to or add py2deb.cfg support).
  • Bug fix: Don’t move generated archives if already in target directory.
  • Big refactoring: Split main module into several sub modules.
  • Significantly improve test coverage.
  • Enable Sphinx viewcode extension.

Release 0.16 (2014-06-05)

Remove the stdeb backend and focus fully on the pip-accel backend:

  • I don’t need something that’s refined and elegant but only supports a subset of packages (stdeb).

    I see stdeb as the more idealistic choice.

  • What I need instead is something that supports all or most packages, and when it does, then it doesn’t matter if the way in which it works isn’t the most elegant way to do things.

    I see the pip-accel backend as the pragmatic choice.

Release 0.15 (2014-06-01)

Abusing update-alternatives for fun and profit?

This makes it possible to create a package with an isolated installation prefix that nevertheless installs global executables in the default executable search path ($PATH).

Release 0.14.9 (2014-05-31)

  • Update dependencies.
  • Update tests to use new version of deb-pkg-tools (including support for relationship parsing and matching).
  • Bug fix: Exclude other architectures from *.deb filename matching.

Release 0.14.8 (2014-05-26)

  • Rename packages_to_rename → name_mapping.
  • Update requirements (python-debian 0.1.21-nmu2 for Python 3.x compatibility).
  • Replace configuration (global state) with function arguments (local state).

Release 0.14.7 (2014-05-24)

Bug fix for last commit.

Release 0.14.6 (2014-05-24)

Don’t implicitly forbid automatic installation by pip-accel.

Release 0.14.5 (2014-05-22)

Release 0.14.4 (2014-05-16)

Implement py2deb --inject-deps=CTRL_FILE option.

Release 0.14.3 (2014-05-07)

  • Implement --no-name-prefix=PKG option, use it in the automated tests.
  • Test conversion of isolated packages and the --rename=FROM,TO option.

Release 0.14.2 (2014-05-07)

  • Bug fixes for --rename=FROM,TO functionality.
  • Bug fix for stdeb backend.
  • Start writing new tests that cover both backends.
  • Start using Sphinx for documentation.
  • Add a test involving a package with Python dependencies as well as system dependencies (stdeb.cfg).

Release 0.14.1 (2014-05-05)

Bug fix for py2deb.util.apply_script().

Release 0.14 (2014-05-05)

Introduce the --rename=FROM,TO option to make things more robust.

Release 0.13.15 (2014-05-04)

Switch from deb_pkg_tools.utils.execute() to executor.execute() (today I decided to extract this functionality into a separate package called executor).

Release 0.13.14 (2014-05-03)

Support for default configuration files (~/.py2deb.ini and /etc/py2deb.ini).

Release 0.13.13 (2014-05-03)

Support for environment variables ($PY2DEB_CONFIG, $PY2DEB_REPO and $PY2DEB_VERBOSE).

Release 0.13.12 (2014-04-23)

Check command line options for non-empty arguments (feedback from Bart :-).

Release 0.13.11 (2014-04-22)

Ignore overridden Debian package names when building isolated packages.

Release 0.13.10 (2014-04-11)

  • Don’t make the post-installation script error out on syntax errors reported by py_compile.
  • Bug fix for apply-script command in pip-accel backend.

Release 0.13.9 (2014-04-11)

Bug fix for order of unpack/apply script/cleanup commands in pip-accel backend.

Release 0.13.8 (2014-04-11)

Release 0.13.7 (2014-04-09)

Bug fix: Never use the root logger.

Release 0.13.6 (2014-04-09)

Bug fix: Remove output redirection, change --print-deps to --report-deps=PATH.

Release 0.13.5 (2014-04-01)

Bug fix: Don’t patch control files of isolated packages.

Release 0.13.4 (2014-03-31)

Bug fix: Move output redirection to main() function (where it belongs).

Release 0.13.3 (2014-03-27)

Reset primary package name when building name/install prefixed packages.

Release 0.13.2 (2014-03-20)

Cleanup handling & documentation of command line arguments.

Release 0.13.1 (2014-03-20)

Add a post-installation script to generate *.pyc files.

Release 0.13 (2014-03-20)

Initial support for isolated packages (not in the default sys.path).

Release 0.12.3 (2014-02-01)

Bump pip-accel requirement (another upstream bug fixed).

Release 0.12.2 (2014-01-30)

Bump pip-accel requirement (upstream bug fixed).

Release 0.12.1 (2013-11-03)

Bug fix: Don’t fail when a PKG-INFO file can’t be parsed.

Release 0.12 (2013-11-03)

Improve the pip-accel backend (use a prerm script to cleanup left over byte code files).

Release 0.11.2 (2013-11-03)

Improve the pip-accel backend (the maintainer field is now preserved).

Release 0.11.1 (2013-11-03)

Improve logging of pip-accel backend.

Release 0.11 (2013-11-03)

  • Improve the pip-accel backend (for example it now respects stdeb.cfg).
  • Move generation of tagged descriptions to common function.
  • Make Python >= 2.6 dependency explicit in stdeb.cfg.

Release 0.10.8 (2013-11-03)

  • Add a test case for converting packages with dependencies on replacements.
  • Increase the verbosity of the stdeb logger.

Release 0.10.7 (2013-11-02)

Bug fix: Properly convert dependencies on packages with replacements (and add a test case for converting packages with dependencies).

Release 0.10.6 (2013-11-02)

  • Bug fix: Make convert() report direct dependencies but not transitive ones.
  • Add a first test case to the test suite, use py.test to run it.

Release 0.10.5 (2013-11-01)

  • Bug fix for logging in py2deb.backends.stdeb_backend.patch_control().
  • Add make reset target to (re)create virtual environment

Release 0.10.4 (2013-10-22)

Bug fix for pip-accel backend (fallback on e.g. Jaunty and Karmic) by rewriting /site-packages/ to /dist-packages/.

Release 0.10.3 (2013-10-22)

Remove automatic dependency installation (way too much magic, a silly idea in retrospect).

Release 0.10.2 (2013-10-21)

Add a missing Debian dependency: python-setuptools.

Release 0.10.1 (2013-10-20)

Bug fix for last commit.

Release 0.10 (2013-10-20)

Fall back to alternative backend when requested backend fails.

Release 0.9.10 (2013-10-20)

Enable compatiblity with Ubuntu 9.04 (Jaunty) by changing from sort --version-sort to sort --general-numeric-sort.

Release 0.9.9 (2013-10-20)

Bug fix: Don’t assume iterable arguments are lists (they might be tuples).

Release 0.9.8 (2013-10-20)

Fix recursive import error between __init__.py and bootstrap.py.

Release 0.9.7 (2013-10-20)

Automatic installation of required system packages.

Release 0.9.6 (2013-10-17)

Bug fix: Send the output of Lintian to stderr! (otherwise --print-deps is broken)

Release 0.9.5 (2013-10-12)

Bump some requirements.

Release 0.9.4 (2013-10-12)

Bug fix for py2deb.bootstrap.install().

Release 0.9.3 (2013-10-12)

Bug fix for py2deb.converter.convert().

Release 0.9.2 (2013-10-12)

Bug fix for py2deb --install.

Release 0.9.1 (2013-10-12)

Bug fix for broken import.

Release 0.9 (2013-10-12)

  • Created a shell script that uses magic in deb-pkg-tools to convert py2deb using itself and install the resulting *.deb packages on the local system. This shell script was then converted to Python and is available from the command line interface using py2deb --install.
  • Bug fix: Don’t error out when repository directory matches archive directory.

Release 0.8.6 (2013-09-29)

Make it simpler to call py2deb from Python (by moving logic from py2deb.main() to py2deb.converter.convert()).

Release 0.8.5 (2013-09-29)

Cleanup handling of logging.

Release 0.8.4 (2013-09-14)

Be compatible with upstream Debianized packages (e.g. Kazoo).

Release 0.8.3 (2013-09-14)

Process required packages in alphabetical sort order.

Release 0.8.2 (2013-08-13)

  • Improved decision process for choosing stdeb version:

    And here’s for a very peculiar bug fix… I was trying to convert PyXML 0.8.4 to a Debian package and the setup.py script kept failing with error: invalid command 'debianize'. After much digging:

    • py2deb runs python setup.py --command-packages=stdeb.command debianize which implies that from stdeb.command import debianize is run.

    • import stdeb actually imports the module bundled with py2deb (which automatically pick the right version of stdeb for the current platform) and this module imported py2deb -> pip-accel -> pip -> html5lib (bundled with pip) which then blows up with:

      >>> import xml.etree.ElementTree as default_etree
ImportError: No module named etree.ElementTree

    • Turns out PyXML 0.8.4 indeed contains an xml module… This all happens because Python implicitly imports from the current working directory before the rest of the entries in sys.path and PyXML actually depends on this; take a look at the setup.py script.

    Lesson learned: I guess it’s wise to restrict our bundled fake stdeb module to standard library module imports :-).

  • Improved py2deb.util.patch_control_file().

Release 0.8.1 (2013-08-13)

  • Implement control overrides for pip-accel backend (also: refactor configuration handling).
  • Make it possible to override individual Debian package names.
  • Backends shouldn’t know about “replacements”.

Release 0.8 (2013-08-13)

Start work on a backend using pip-accel instead of stdeb:

  • After working with stdeb for over four months it had become painfully clear that it would never be able to convert the huge dependency trees I had in mind for it because it was simply way too fragile.
  • At the same time I knew from working on pip-accel that python setup.py bdist was much more reliable / robust and gave usable results, even if completely specific to the major and minor version of the running Python interpreter.

This is how I decided to start working on an alternative package conversion backend for py2deb.

Release 0.7.7 (2013-08-11)

  • Remove reference to stdeb from py2deb.ini (bundled with py2deb anyway).
  • Log external command execution.
  • Fix copy/paste error in setup.py.
  • Improve stdeb version selection.

Release 0.7.6 (2013-08-11)

Use coloredlogs.increase_verbosity() (always keep logger at full verbosity).

Release 0.7.5 (2013-08-11)

Release 0.7.4 (2013-08-11)

Compatibility with pip-accel 0.9.4.

Release 0.7.3 (2013-08-11)

Improve the setup.py script and move the installation requirements to a separate requirements.txt file.

Release 0.7.2 (2013-08-07)

Tweak the requirements.

Release 0.7.1 (2013-08-05)

  • Compatibility with the latest version of pip-accel (0.9.12).
  • Compatibility with the latest version of deb-pkg-tools.
  • Restore release tag in pinned versions only.
  • Abuse “Description” field to advertise py2deb.
  • Make py2deb -v imply DH_VERBOSE=1 (pass verbosity to debian-helper scripts).

Release 0.7 (2013-07-23)

This is a snapshot in the middle of a big refactoring…

I’d love to use py2deb in a dozen places but was blocked from doing so because of a handful of unrelated issues that remained to be solved. After lots of testing, failed attempts and frustration I now have something that seems to work (although I have to clean it up and there are still some minor issues that I’m aware of):

  • My original goal with py2deb was to use two name spaces for the names of generated packages: The real name space pl-python-... would be very explicit but dependencies would refer to virtual packages named python-.... Then the pl-python-... packages could have Provides: fields giving the python-... names.

    It turns out this cannot work the way I want it to; virtual packages are second class citizens in Debian :-(. AFAICT the only way to get everything working properly is to just use the python-... name space directly, so that’s what the new code is slowly working towards.

  • Merging of control files was not working properly, however some months ago (I think before py2deb was born) I wrote my own control file merger. I’ve now extracted that from the project where it originated and moved it to a package called deb-pkg-tools, which hasn’t been released yet but will be soon. py2deb now uses deb-pkg-tools to patch/merge control files.

  • The Python == version matching operator was copied verbatim to the Debian control files which is invalid. This is now fixed.

  • stdeb 0.6.0 is required on Ubuntu 10.04, stdeb 0.6.0+git is required on Ubuntu 12.04, however stdeb 0.6.0+git hasn’t been released yet. Also Python nor Debian can simply/elegantly express this very explicit distinction between stdeb versions and Ubuntu distributions. The only remaining way to keep my sanity was to bundle both versions of stdeb inside py2deb.

    TODO: Add READMEs, LICENSEs.

  • Lots of changes to logging including the version of coloredlogs and the introduction of separate loggers for separate modules.

  • Lots of moving around with code and responsibilities while I tried to make sense of the way py2deb should and could work.

Release 0.6.10 (2013-07-05)

  • Replace nasty rules file patching with an environment variable
  • Improved the README.

Release 0.6.9 (2013-06-27)

Minor changes to logging output (changed severity levels + made logger name visible).

Release 0.6.8 (2013-06-27)

Make it possible to set the repository directory as a command line option.

Release 0.6.7 (2013-06-27)

Sneaking in a minor bug fix.

Release 0.6.6 (2013-06-27)

Redirect pip’s output to stderr.

Release 0.6.5 (2013-06-26)

  • Updated README.
  • Return of the sanity_check

Release 0.6.4 (2013-06-25)

  • Will now correctly remove the script field.
  • Fixed dependency issues.

Release 0.6.2 (2013-06-25)

Temporarily removed sanity checking.

Release 0.6.1 (2013-06-24)

Added sanity check on dependencies using pip-accel.

Release 0.6.0 (2013-06-24)

  • Moved and rewrote converter, package, util to reflect changes to the cli.
  • Fixed check on returncodes from subprocesses.
  • Overhauled command line options.
  • Changed verbosity option.
  • Renamed control.ini.

Release 0.5.41 (2013-06-04)

Try to deal better with packages that have Debian replacements.

Release 0.5.40 (2013-06-04)

Deal with the python-imaging vs. pil vs. pillow mess 😞.

Release 0.5.39 (2013-06-04)

Added pil to control.ini.

Release 0.5.38 (2013-06-04)

Lots of changes to deal with the whole setuptools/distribute contraption…

Release 0.5.37 (2013-06-04)

Added Pillow conflict with python-imaging to control.ini.

Release 0.5.36 (2013-05-30)

  • Mark the python-support package as a requirement of py2deb in the configuration file.
  • Added the command line option -d, --no-deps to ignore dependencies.

Release 0.5.35 (2013-05-17)

Raise an exception if there is no dependency file to recall.

Release 0.5.34 (2013-05-17)

Properly integrate pip-accel 0.8.5 into py2deb and remove the embedded (and simplified) variant of pip-accel from the py2deb code base.

Release 0.5.33 (2013-05-02)

Workaround Fabric bundling Paramiko.

Release 0.5.32 (2013-05-02)

Bug fix: Requirement instance has no attribute ‘specs’.

Release 0.5.31 (2013-05-02)

Remove confusion about py2deb.package.Requirement versus pkg_resources.Requirement.

Release 0.5.30 (2013-05-02)

  • Rename [replace_dependencies] section to [replacements].
  • Add [replacements] workarounds for specific packages to the configuration file.
  • Don’t translate replacement package names.

Release 0.5.29 (2013-05-02)

Make pinned Debian dependencies explicit.

Release 0.5.28 (2013-05-02)

Change the location of the default repository when running as root.

Release 0.5.27 (2013-05-02)

  • Pinned version of python-debian.
  • Support for “replacing” dependencies (for example setuptools versus distribute).
  • Lots of changes and improvements to dependency/requirement handling.

Release 0.5.26 (2013-05-01)

Incorporate release numbers in pinned versions (without this, pl-py2deb --recall reports invalid versions).

Release 0.5.25 (2013-05-01)

  • Make it possible to persist and recall Debianized dependencies.
  • Add a simple command line interface.
  • Place built packages in /tmp if user is not root.
  • Make sure python setup.py debianize runs inside the virtual environment.

Release 0.5.24 (2013-05-01)

Report dependencies as well as required versions.

Release 0.5.23 (2013-04-29)

Another bug fix.

Release 0.5.22 (2013-04-29)

Another bug fix.

Release 0.5.21 (2013-04-29)

Another bug fix.

Release 0.5.20 (2013-04-29)

Sorry, forgot to call the function…

Release 0.5.19 (2013-04-29)

Bug fix for previous release.

Release 0.5.18 (2013-04-29)

Bug fix for dependency introspection.

Release 0.5.17 (2013-04-29)

Remove merge_dicts usage.

Release 0.5.16 (2013-04-29)

Don’t print empty Depends: fields.

Release 0.5.15 (2013-04-29)

Bug fix for deb822 usage (merge_fields doesn’t work if you start with an empty field).

Release 0.5.14 (2013-04-29)

Bug fix for release 0.5.13.

Release 0.5.13 (2013-04-29)

Print the Depends: fields of built packages.

Release 0.5.12 (2013-04-25)

Code style noise.

Release 0.5.11 (2013-04-25)

Bug fix: Use pkg_resources.Requirement.parse() to properly parse requirement expressions.

Release 0.5.10 (2013-04-25)

Don’t silence the output of dpkg-buildpackage.

Release 0.5.9 (2013-04-25)

  • Ignore GPG signing when building packages.
  • Don’t cleanup build directory on exceptions (allows post-mortem debugging).
  • Added a readme and todo list.

Release 0.5.8 (2013-04-25)

Yet another bug fix for release 0.5.5…

Release 0.5.7 (2013-04-25)

Another bug fix for release 0.5.5.

Release 0.5.6 (2013-04-25)

Bug fix for release 0.5.5.

Release 0.5.5 (2013-04-25)

Fixes for installation of global build dependencies.

Release 0.5.4 (2013-04-25)

Don’t silence the output of apt-get when installing build dependencies.

Release 0.5.3 (2013-04-25)

Use system wide pip-accel cache directories when running as root.

Release 0.5.2 (2013-04-25)

Add dependency on chardet which is imported by python-debian but not included in its installation requirements.

Release 0.5.1 (2013-04-25)

  • Properly nest all Python modules under pydeb.* namespace.

  • Renamed command line entry point from py2deb to pl-py2deb.

    Context: py2deb is developed at Paylogic where a lot of our internal command line tools use the pl-* namespace inspired by the mk-* / pt-* namespace that Percona Toolkit uses.

Release 0.5.0 (2013-04-24)

The initial release, very much a rough work in progress 😇.

The py2deb project was kicked off by Arjan, an intern at Paylogic at the time, in collaboration with Peter (who guided Arjan’s internship). The abstract idea that we set out to create was as follows:

  • Use pip to download a Python package from PyPI and recursively gather installation requirements until we can satisfy all dependencies.
  • Use stdeb to batch convert all of the downloaded Python packages to Debian packages.