Documentation for the py2deb API

On this page you can find the complete API documentation of py2deb 0.24.4.

A note about backwards compatibility

On the one hand the py2deb project has been in development since April 2013, on the other hand it was only made public in February 2015 (a couple of days at the of writing :-). Also note that a 1.0 version number hasn’t been reached yet. What I mean to say is that I’m not committing to API stability yet. As the project matures I will likely nominate a 1.0 version and decide to freeze the API for backwards incompatible changes. That time hasn’t come yet.

The Python API of py2deb

Here are the relevant Python modules that make up py2deb:

py2deb - Project metadata

The top level py2deb module contains only a version number.

py2deb.__version__

The version number of the pydeb package (a string).

py2deb.cli - Command line interface

Usage: py2deb [OPTIONS] ...

Convert Python packages to Debian packages according to the given command line options (see below). The command line arguments are the same as accepted by the “pip install” command because py2deb invokes pip during the conversion process. This means you can name the package(s) to convert on the command line but you can also use “requirement files” if you prefer.

If you want to pass command line options to pip (e.g. because you want to use a custom index URL or a requirements file) then you will need to tell py2deb where the options for py2deb stop and the options for pip begin. In such cases you can use the following syntax:

$ py2deb -r /tmp -- -r requirements.txt

So the “–” marker separates the py2deb options from the pip options.

Supported options:

Option Description
-c, --config=FILENAME

Load a configuration file. Because the command line arguments are processed in the given order, you have the choice and responsibility to decide if command line options override configuration file options or vice versa. Refer to the documentation for details on the configuration file format.

The default configuration files /etc/py2deb.ini and ~/.py2deb.ini are automatically loaded if they exist. This happens before environment variables and command line options are processed.

Can also be set using the environment variable $PY2DEB_CONFIG.

-r, --repository=DIRECTORY

Change the directory where *.deb archives are stored. Defaults to the system wide temporary directory (which is usually /tmp). If this directory doesn’t exist py2deb refuses to run.

Can also be set using the environment variable $PY2DEB_REPOSITORY.

--name-prefix=PREFIX

Set the name prefix used during the name conversion from Python to Debian packages. Defaults to “python”. The name prefix and package names are always delimited by a dash.

Can also be set using the environment variable $PY2DEB_NAME_PREFIX.

--no-name-prefix=PYTHON_PACKAGE_NAME Exclude a Python package from having the name prefix applied during the package name conversion. This is useful to avoid awkward repetitions.
--rename=PYTHON_PACKAGE_NAME,DEBIAN_PACKAGE_NAME Override the package name conversion algorithm for the given pair of package names. Useful if you don’t agree with the algorithm :-)
--install-prefix=DIRECTORY

Override the default system wide installation prefix. By setting this to anything other than “/usr” or “/usr/local” you change the way py2deb works. It will build packages with a file system layout similar to a Python virtual environment, except there will not be a Python executable: The packages are meant to be loaded by modifying Python’s module search path. Refer to the documentation for details.

Can also be set using the environment variable $PY2DEB_INSTALL_PREFIX.

--install-alternative=LINK,PATH Use Debian’s “update-alternatives” system to add an executable that’s installed in a custom installation prefix (see above) to the system wide executable search path. Refer to the documentation for details.
--python-callback=EXPRESSION

Set a Python callback to be called during the conversion process. Refer to the documentation for details about the use of this feature and the syntax of EXPRESSION.

Can also be set using the environment variable $PY2DEB_CALLBACK.

--report-dependencies=FILENAME Add the Debian relationships needed to depend on the converted package(s) to the given control file. If the control file already contains relationships the additional relationships will be added to the control file; they won’t overwrite existing relationships.
-y, --yes

Instruct pip-accel to automatically install build time dependencies where possible. Refer to the pip-accel documentation for details.

Can also be set using the environment variable $PY2DEB_AUTO_INSTALL.

-v, --verbose Make more noise :-).
-h, --help Show this message and exit.
py2deb.cli.main()[source]

Command line interface for the py2deb program.

py2deb.converter - Package converter

The py2deb.converter module contains the high level conversion logic.

This module defines the PackageConverter class which provides the intended way for external Python code to interface with py2deb. The separation between the PackageConverter and PackageToConvert classes is somewhat crude (because neither class can work without the other) but the idea is to separate the high level conversion logic from the low level conversion logic.

py2deb.converter.MACHINE_ARCHITECTURE_MAPPING = {'x86_64': 'amd64', 'armv6l': 'armhf', 'i686': 'i386'}

Mapping of supported machine architectures (a dictionary).

The keys are the names reported by os.uname() and the values are machine architecture labels used in the Debian packaging system.

class py2deb.converter.PackageConverter(load_configuration_files=True, load_environment_variables=True)[source]

The external interface of py2deb, the Python to Debian package converter.

alternatives

A set of tuples with two strings each (the strings passed to install_alternative()). Used by create_alternatives() and cleanup_alternatives() during installation and removal of the generated package.

__init__(load_configuration_files=True, load_environment_variables=True)[source]

Initialize a Python to Debian package converter.

Parameters:
set_repository(directory)[source]

Set pathname of directory where py2deb stores converted packages.

Parameters:directory – The pathname of a directory (a string).
Raises:ValueError when the directory doesn’t exist.
set_name_prefix(prefix)[source]

Set package name prefix to use during package conversion.

Parameters:prefix – The name prefix to use (a string).
Raises:ValueError when no name prefix is provided (e.g. an empty string).
rename_package(python_package_name, debian_package_name)[source]

Override package name conversion algorithm for given pair of names.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • debian_package_name – The name of the converted Debian package (a string).
Raises:

ValueError when a package name is not provided (e.g. an empty string).

set_install_prefix(directory)[source]

Set installation prefix to use during package conversion.

The installation directory doesn’t have to exist on the system where the package is converted.

Parameters:directory – The pathname of the directory where the converted packages should be installed (a string).
Raises:ValueError when no installation prefix is provided (e.g. an empty string).
set_auto_install(enabled)[source]

Enable or disable automatic installation of build time dependencies.

Parameters:enabled – Any value, evaluated using coerce_boolean().
set_lintian_enabled(enabled)[source]

Enable or disable automatic Lintian checks after package building.

Parameters:enabled – Any value, evaluated using coerce_boolean().
install_alternative(link, path)[source]

Install system wide link for program installed in custom installation prefix.

Use Debian’s update-alternatives system to add an executable that’s installed in a custom installation prefix to the system wide executable search path using a symbolic link.

Parameters:
  • link – The generic name for the master link (a string). This is the first argument passed to update-alternatives --install.
  • path – The alternative being introduced for the master link (a string). This is the third argument passed to update-alternatives --install.
Raises:

ValueError when one of the paths is not provided (e.g. an empty string).

If this is a bit vague, consider the following example:

$ py2deb --name-prefix=py2deb \
         --no-name-prefix=py2deb \
         --install-prefix=/usr/lib/py2deb \
         --install-alternative=/usr/bin/py2deb,/usr/lib/py2deb/bin/py2deb \
         py2deb==0.1

This example will convert py2deb and its dependencies using a custom name prefix and a custom installation prefix which means the py2deb program is not available on the default executable search path. This is why update-alternatives is used to create a symbolic link /usr/bin/py2deb which points to the program inside the custom installation prefix.

set_conversion_command(python_package_name, command)[source]

Set shell command to be executed during conversion process.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • command – The shell command to execute (a string).
Raises:

ValueError when the package name or command is not provided (e.g. an empty string).

The shell command is executed in the directory containing the Python module(s) that are to be installed by the converted package.

Warning

This functionality allows arbitrary manipulation of the Python modules to be installed by the converted package. It should clearly be considered a last resort, only for for fixing things like packaging issues with Python packages that you can’t otherwise change.

For example old versions of Fabric bundle a copy of Paramiko. Most people will never notice this because Python package managers don’t complain about this, they just blindly overwrite the files... Debian’s packaging system is much more strict and will consider the converted Fabric and Paramiko packages as conflicting and thus broken. In this case you have two options:

  1. Switch to a newer version of Fabric that no longer bundles Paramiko;
  2. Use the conversion command rm -rf paramiko to convert Fabric (yes this is somewhat brute force :-).
set_python_callback(expression)[source]

Set a Python callback to be called during the conversion process.

Parameters:expression

One of the following:

  1. A callable object (to be provided by Python API callers).
  2. A string containing the pathname of a Python script and the name of a callable, separated by a colon. The Python script will be loaded using exec.
  3. A string containing the “dotted path” of a Python module and the name of a callable, separated by a colon. The Python module will be loaded using importlib.import_module().
  4. Any value that evaluates to False will clear an existing callback (if any).
Raises:ValueError when the given expression does not result in a valid callable. ImportError when the expression contains a dotted path that cannot be imported.

The callback will be called at the very last step before the binary package’s metadata and contents are packaged as a *.deb archive.

This allows arbitrary manipulation of resulting binary packages, e.g. changing package metadata or files to be packaged.

An example use case:

  • Consider a dependency set (group of related packages) that has previously been converted and deployed.
  • A new version of the dependency set switches from Python package A to Python package B, where the two Python packages contain conflicting files (installed in the same location). This could happen when switching to a project’s fork.
  • A deployment of the new dependency set will conflict with existing installations due to “unrelated” packages (in the eyes of apt and dpkg) installing the same files.
  • By injecting a custom Python callback the user can mark package B as “replacing” and “breaking” package A. Refer to section 7.6 of the Debian policy manual for details about the required binary control fields (hint: Replaces: and Breaks:).

Warning

The callback is responsible for not making changes that would break the installation of the converted dependency set!

load_environment_variables()[source]

Load configuration defaults from environment variables.

The following environment variables are currently supported:

  • $PY2DEB_CONFIG
  • $PY2DEB_REPOSITORY
  • $PY2DEB_NAME_PREFIX
  • $PY2DEB_INSTALL_PREFIX
  • $PY2DEB_AUTO_INSTALL
  • $PY2DEB_LINTIAN
load_configuration_file(configuration_file)[source]

Load configuration defaults from a configuration file.

Parameters:configuration_file – The pathname of a configuration file (a string).
Raises:Exception when the configuration file cannot be loaded.

Below is an example of the available options, I assume that the mapping between the configuration options and the setters of PackageConverter is fairly obvious (it should be :-).

# The `py2deb' section contains global options.
[py2deb]
repository = /tmp
name-prefix = py2deb
install-prefix = /usr/lib/py2deb
auto-install = on
lintian = on

# The `alternatives' section contains instructions
# for Debian's `update-alternatives' system.
[alternatives]
/usr/bin/py2deb = /usr/lib/py2deb/bin/py2deb

# Sections starting with `package:' contain conversion options
# specific to a package.
[package:py2deb]
no-name-prefix = true

Note that the configuration options shown here are just examples, they are not the configuration defaults (they are what I use to convert py2deb itself). Package specific sections support the following options:

no-name-prefix:
A boolean indicating whether the configured name prefix should be applied or not. Understands true and false (false is the default and you only need this option to change the default).
rename:
Gives an override for the package name conversion algorithm (refer to rename_package() for details).
script:
Set a shell command to be executed during the conversion process (refer to set_conversion_command() for details).
load_default_configuration_files()[source]

Load configuration options from default configuration files.

The following default configuration file locations are checked:

  • /etc/py2deb.ini
  • ~/.py2deb.ini
Raises:Exception when a configuration file exists but cannot be loaded.
convert(pip_install_arguments)[source]

Convert one or more Python packages to Debian packages.

Parameters:pip_install_arguments – The command line arguments to the pip install command.
Returns:A tuple with two lists:
  1. A list of strings containing the pathname(s) of the generated Debian package package archive(s).
  2. A list of strings containing the Debian package relationship(s) required to depend on the converted package(s).
Raises:DuplicateFilesFound if two converted package archives contain the same files (certainly not what you want within a set of dependencies).

Here’s an example of what’s returned:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> archives, relationships = converter.convert(['py2deb'])
>>> print(archives)
['/tmp/python-py2deb_0.18_all.deb']
>>> print(relationships)
['python-py2deb (=0.18)']
get_source_distributions(pip_install_arguments)[source]

Use pip_accel to download and unpack Python source distributions.

Retries several times if a download fails (so it doesn’t fail immediately when a package index server returns a transient error).

Parameters:pip_install_arguments – The command line arguments to the pip install command.
Returns:A generator of PackageToConvert objects.
Raises:When downloading fails even after several retries this function raises pip.exceptions.DistributionNotFound. This function can also raise other exceptions raised by pip because it uses pip_accel to call pip (as a Python API).
transform_name(python_package_name, *extras)[source]

Transform Python package name to Debian package name.

Parameters:
  • python_package_name – The name of a Python package as found on PyPI (a string).
  • extras – Any extras requested to be included (a tuple of strings).
Returns:

The transformed name (a string).

Examples:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> converter.transform_name('example')
'python-example'
>>> converter.set_name_prefix('my-custom-prefix')
>>> converter.transform_name('example')
'my-custom-prefix-example'
>>> converter.set_name_prefix('some-web-app')
>>> converter.transform_name('raven', 'flask')
'some-web-app-raven-flask'
transform_version(package_to_convert, python_requirement_name, python_requirement_version)[source]

Transform a Python requirement version to a Debian version number.

Parameters:
  • package_to_convert – The PackageToConvert whose requirement is being transformed.
  • python_requirement_name – The name of a Python package as found on PyPI (a string).
  • python_requirement_version – The required version of the Python package (a string).
Returns:

The transformed version (a string).

This method is a wrapper for normalize_package_version() that takes care of one additional quirk to ensure compatibility with pip. Explaining this quirk requires a bit of context:

  • When package A requires package B (via install_requires) and package A absolutely pins the required version of package B using one or more trailing zeros (e.g. B==1.0.0) but the actual version number of package B (embedded in the metadata of package B) contains less trailing zeros (e.g. 1.0) then pip will not complain but silently fetch version 1.0 of package B to satisfy the requirement.
  • However this doesn’t change the absolutely pinned version in the install_requires metadata of package A.
  • When py2deb converts the resulting requirement set, the dependency of package A is converted as B (= 1.0.0). The resulting packages will not be installable because apt considers 1.0 to be different from 1.0.0.

This method analyzes the requirement set to identify occurrences of this quirk and strip trailing zeros in install_requires metadata that would otherwise result in converted packages that cannot be installed.

debian_architecture

Find the Debian architecture of the current environment.

This logic was originally implemented in py2deb but has since been moved to deb_pkg_tools.utils.find_debian_architecture(). This property remains as a convenient shortcut.

py2deb.package - Package conversion model

The py2deb.package module contains the low level conversion logic.

This module defines the PackageToConvert class which implements the low level logic of converting a single Python package to a Debian package. The separation between the PackageConverter and PackageToConvert classes is somewhat crude (because neither class can work without the other) but the idea is to separate the high level conversion logic from the low level conversion logic.

class py2deb.package.PackageToConvert(converter, requirement)[source]

Abstraction for Python packages to be converted to Debian packages.

Contains a pip_accel.req.Requirement object, has a back reference to the PackageConverter and provides all of the Debian package metadata implied by the Python package metadata.

__init__(converter, requirement)[source]

Initialize a package to convert.

Parameters:
__str__()[source]

The name, version and extras of the package encoded in a human readable string.

python_name

The name of the Python package (a string).

debian_name

The name of the converted Debian package (a string).

python_version

The version of the Python package (a string).

vcs_revision

The VCS revision of the Python package.

This works by parsing the .hg_archival.txt file generated by the hg archive command so for now this only supports Python source distributions exported from Mercurial repositories.

debian_version

The version of the Debian package (a string).

Reformats python_version using normalize_package_version().

debian_maintainer

Get the package maintainer’s name and e-mail address.

The name and e-mail address are combined into a single string that can be embedded in a Debian package.

debian_description

Get a minimal description for the converted Debian package.

Includes the name of the Python package and the date at which the package was converted.

metadata

Get the Python package metadata.

The metadata is loaded from the PKG-INFO file generated by pip when it unpacked the source distribution archive. Results in a pkginfo.UnpackedSDist object.

namespace_packages

Get the Python namespace packages defined by the Python package.

This property returns the same value that was originally passed to the namespace_packages keyword argument of setuptools.setup() (albeit in a very indirect way, but nonetheless the same value :-).

Returns:A list of dotted names (strings).
namespaces

Get the Python namespace packages defined by the Python package.

Returns:A list of unique tuples of strings. The tuples are sorted by increasing length (the number of strings in each tuple) so that e.g. zope is guaranteed to sort before zope.app.

This property processes the result of namespace_packages into a more easily usable format. Here’s an example of the difference between namespace_packages and namespaces:

>>> from py2deb.converter import PackageConverter
>>> converter = PackageConverter()
>>> package = next(converter.get_source_distributions(['zope.app.cache']))
>>> package.namespace_packages
['zope', 'zope.app']
>>> package.namespaces
[('zope',), ('zope', 'app')]

The value of this property is used by initialize_namespaces() and cleanup_namespaces() during installation and removal of the generated package.

has_custom_install_prefix

Check whether package is being installed under custom installation prefix.

Returns:True if the package is being installed under a custom installation prefix, False otherwise.

A custom installation prefix is an installation prefix whose bin directory is (likely) not available on the default executable search path (the environment variable $PATH)

python_requirements

Find requirements of Python package.

Returns:A list of pkg_resources.Requirement objects, read from the requires.txt file generated by pip when it unpacks a source distribution archive.
debian_dependencies

Find Debian dependencies of Python package.

Converts Python version specifiers to Debian package relationships.

Returns:A list with Debian package relationships (strings) in the format of the Depends: line of a Debian package control file. Based on python_requirements.
existing_archive

Find *.deb archive for current package name and version.

Returns:The pathname of the found archive (a string) or None if no existing archive is found.
convert()[source]

Convert current package from Python package to Debian package.

Returns:The pathname of the generated *.deb archive.
transform_binary_dist()[source]

Build Python package and transform directory layout.

Builds the Python package (using pip_accel) and changes the names of the files included in the package to match the layout corresponding to the given conversion options.

Returns:An iterable of tuples with two values each:
  1. A tarfile.TarInfo object;
  2. A file-like object.
find_shared_object_files(directory)[source]

Search directory tree of converted package for shared object files.

Runs strip --strip-unneeded on all *.so files found.

Parameters:directory – The directory to search (a string).
Returns:A list with pathnames of *.so files.
find_system_dependencies(shared_object_files)[source]

(Ab)use dpkg-shlibdeps to find dependencies on system libraries.

Parameters:shared_object_files – The pathnames of the *.so file(s) contained in the package (a list of strings).
Returns:A list of strings in the format of the entries on the Depends: line of a binary package control file.
determine_package_architecture(has_shared_object_files)[source]

Determine binary architecture that Debian package should be tagged with.

If a package contains *.so files we’re dealing with a compiled Python module. To determine the applicable architecture, we take the Debian architecture reported by debian_architecture.

Parameters:has_shared_objectsTrue if the package contains *.so files, False otherwise.
Returns:The architecture string, ‘all’ or one of the values of debian_architecture.
load_control_field_overrides(control_fields)[source]

Apply user defined control field overrides.

Looks for an stdeb.cfg file inside the Python package’s source distribution and if found it merges the overrides into the control fields that will be embedded in the generated Debian binary package.

This method first applies any overrides defined in the DEFAULT section and then it applies any overrides defined in the section whose normalized name (see package_names_match()) matches that of the Python package.

Parameters:control_fields – The control field defaults constructed by py2deb (a debian.deb822.Deb822 object).
Returns:The merged defaults and overrides (a debian.deb822.Deb822 object).
generate_maintainer_script(filename, python_executable, function, **arguments)[source]

Generate a post-installation or pre-removal maintainer script.

Parameters:
  • filename – The pathname of the maintainer script (a string).
  • python_executable – The absolute pathname of the Python interpreter on the target system (a string).
  • function – The name of the function in the py2deb.hooks module to be called when the maintainer script is run (a string).
  • arguments – Any keyword arguments to the function in the py2deb.hooks are serialized and embedded inside the generated maintainer script.
find_egg_info_file(pattern='')[source]

Find pip metadata files in unpacked source distributions.

When pip unpacks a source distribution archive it creates a directory pip-egg-info which contains the package metadata in a declarative and easy to parse format. This method finds such metadata files.

Parameters:pattern – The glob pattern to search for (a string).
Returns:A list of matched filenames (strings).

py2deb.utils - Utility classes/functions

The py2deb.utils module contains miscellaneous code.

py2deb.utils.integer_pattern = <_sre.SRE_Pattern object>

Compiled regular expression to match a consecutive run of digits.

class py2deb.utils.PackageRepository(directory)[source]

Very simply abstraction for a directory containing *.deb archives.

Used by py2deb.converter.PackageConverter to recognize which Python packages have previously been converted (and so can be skipped).

__init__(directory)[source]

Initialize a PackageRepository object.

Parameters:directory – The pathname of the directory containing *.deb archives (a string).
archives

Find archive(s) in package repository / directory.

Returns:A sorted list of package archives, same as the return value of deb_pkg_tools.package.find_package_archives().

An example:

>>> from py2deb import PackageRepository
>>> repo = PackageRepository('/tmp')
>>> repo.archives
[PackageFile(name='py2deb', version='0.1', architecture='all',
             filename='/tmp/py2deb_0.1_all.deb'),
 PackageFile(name='py2deb-cached-property', version='0.1.5', architecture='all',
             filename='/tmp/py2deb-cached-property_0.1.5_all.deb'),
 PackageFile(name='py2deb-chardet', version='2.2.1', architecture='all',
             filename='/tmp/py2deb-chardet_2.2.1_all.deb'),
 PackageFile(name='py2deb-coloredlogs', version='0.5', architecture='all',
             filename='/tmp/py2deb-coloredlogs_0.5_all.deb'),
 PackageFile(name='py2deb-deb-pkg-tools', version='1.20.4', architecture='all',
             filename='/tmp/py2deb-deb-pkg-tools_1.20.4_all.deb'),
 PackageFile(name='py2deb-docutils', version='0.11', architecture='all',
             filename='/tmp/py2deb-docutils_0.11_all.deb'),
 PackageFile(name='py2deb-executor', version='1.2', architecture='all',
             filename='/tmp/py2deb-executor_1.2_all.deb'),
 PackageFile(name='py2deb-html2text', version='2014.4.5', architecture='all',
             filename='/tmp/py2deb-html2text_2014.4.5_all.deb'),
 PackageFile(name='py2deb-humanfriendly', version='1.8.2', architecture='all',
             filename='/tmp/py2deb-humanfriendly_1.8.2_all.deb'),
 PackageFile(name='py2deb-pkginfo', version='1.1', architecture='all',
             filename='/tmp/py2deb-pkginfo_1.1_all.deb'),
 PackageFile(name='py2deb-python-debian', version='0.1.21-nmu2', architecture='all',
             filename='/tmp/py2deb-python-debian_0.1.21-nmu2_all.deb'),
 PackageFile(name='py2deb-six', version='1.6.1', architecture='all',
             filename='/tmp/py2deb-six_1.6.1_all.deb')]
get_package(package, version, architecture)[source]

Find a package in the repository.

Here’s an example:

>>> from py2deb import PackageRepository
>>> repo = PackageRepository('/tmp')
>>> repo.get_package('py2deb', '0.1', 'all')
PackageFile(name='py2deb', version='0.1', architecture='all', filename='/tmp/py2deb_0.1_all.deb')
Parameters:
  • package – The name of the package (a string).
  • version – The version of the package (a string).
  • architecture – The architecture of the package (a string).
Returns:

A deb_pkg_tools.package.PackageFile object or None.

class py2deb.utils.TemporaryDirectory(**options)[source]

Easy temporary directory creation & cleanup using the with statement.

Here’s an example of how to use this:

with TemporaryDirectory() as directory:
    # Do something useful here.
    assert os.path.isdir(directory)
__init__(**options)[source]

Initialize context manager that manages creation & cleanup of temporary directory.

Parameters:options – Any keyword arguments are passed on to tempfile.mkdtemp().
__enter__()[source]

Create the temporary directory.

__exit__(exc_type, exc_value, traceback)[source]

Destroy the temporary directory.

py2deb.utils.python_version()[source]

Find the version of Python we’re running.

This specifically returns a name matching the format of the names of the Debian packages providing the various available Python versions.

Returns:A string like python2.6 or python2.7.
py2deb.utils.normalize_package_name(python_package_name)[source]

Normalize Python package name to be used as Debian package name.

Parameters:python_package_name – The name of a Python package as found on PyPI (a string).
Returns:The normalized name (a string).
>>> from py2deb import normalize_package_name
>>> normalize_package_name('MySQL-python')
'mysql-python'
>>> normalize_package_name('simple_json')
'simple-json'
py2deb.utils.normalize_package_version(python_package_version)[source]

Normalize Python package version to be used as Debian package version.

Parameters:python_package_version – The version of a Python package (a string).

Reformats Python package versions to comply with the Debian policy manual. All characters except alphanumerics, dot (.) and plus (+) are replaced with dashes (-).

py2deb.utils.tokenize_version(version_number)[source]

Tokenize a string containing a version number.

Parameters:version_number – The string to tokenize.
Returns:A list of strings.
py2deb.utils.package_names_match(a, b)[source]

Check whether two Python package names are equal.

Uses normalize_package_name() to normalize both names before comparing them for equality. This makes sure differences in case and dashes versus underscores are ignored.

Parameters:
  • a – The name of the first Python package (a string).
  • b – The name of the second Python package (a string).
Returns:

True if the package names match, False if they don’t.

py2deb.utils.compact_repeating_words(words)[source]

Remove adjacent repeating words.

Parameters:words – An iterable of words (strings), assumed to already be normalized (lowercased).
Returns:An iterable of words with adjacent repeating words replaced by a single word.

This is used to avoid awkward word repetitions in the package name conversion algorithm. Here’s an example of what I mean:

>>> from py2deb import compact_repeating_words
>>> name_prefix = 'python'
>>> package_name = 'python-mcrypt'
>>> combined_words = [name_prefix] + package_name.split('-')
>>> print(list(combined_words))
['python', 'python', 'mcrypt']
>>> compacted_words = compact_repeating_words(combined_words)
>>> print(list(compacted_words))
['python', 'mcrypt']
py2deb.utils.embed_install_prefix(handle, install_prefix)[source]

Embed Python snippet that adds custom installation prefix to module search path.

Parameters:
  • handle – A file-like object containing an executable Python script.
  • install_prefix – The pathname of the custom installation prefix (a string).
Returns:

A file-like object containing the modified Python script.

py2deb.hooks - Maintainer scripts

The py2deb.hooks module contains post-installation and pre-removal hooks.

This module is a bit special in the sense that it is a part of the py2deb code base but it is embedded in the Debian binary packages generated by py2deb as a post-installation and pre-removal hook.

Because this module is embedded in generated packages it can’t use the external dependencies of the py2deb project, it needs to restrict itself to Python’s standard library.

My reasons for including this Python script as a “proper module” inside the py2deb project:

  • It encourages proper documentation of the functionality in this module, which enables users to read through the documentation without having to dive into py2deb’s source code.
  • It makes it easier to unit test the individual functions in this script without jumping through too many hoops (I greatly value test suite coverage).

The generate_maintainer_script() method is responsible for converting this module into a post-installation or pre-removal script. It does so by reading this module’s source code and appending a call to post_installation_hook() or pre_removal_hook() at the bottom.

py2deb.hooks.post_installation_hook(package_name, alternatives, modules_directory, namespaces)[source]

Generic post-installation hook for packages generated by py2deb.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).

Uses the following functions to implement everything py2deb needs from the post-installation maintainer script:

py2deb.hooks.pre_removal_hook(package_name, alternatives, modules_directory, namespaces)[source]

Generic pre-removal hook for packages generated by py2deb.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by py2deb.package.PackageToConvert.namespaces).

Uses the following functions to implement everything py2deb needs from the pre-removal maintainer script:

py2deb.hooks.initialize_logging()[source]

Initialize logging to the terminal and system log.

py2deb.hooks.find_installed_files(package_name)[source]

Find the files installed by a Debian system package.

Uses the dpkg -L command.

Parameters:package_name – The name of the system package (a string).
Returns:A list of absolute filenames (strings).
py2deb.hooks.generate_bytecode_files(package_name, installed_files)[source]

Generate Python byte code files for the *.py files installed by a package.

Uses py_compile.compile() to generate bytecode files.

Parameters:
  • package_name – The name of the system package (a string).
  • installed_files – A list of strings with the absolute pathnames of installed files.
py2deb.hooks.cleanup_bytecode_files(package_name, installed_files)[source]

Cleanup Python byte code files generated when a package was installed.

Parameters:
  • package_name – The name of the system package (a string).
  • installed_files – A list of strings with the absolute pathnames of installed files.
py2deb.hooks.cleanup_bytecode_helper(filenames)[source]

Cleanup Python byte code files.

Parameters:filenames – A list of strings with the absolute pathnames of installed files.
Returns:The number of files that were removed (an integer).
py2deb.hooks.remove_empty_directory(directory)[source]

Remove a directory if it is empty.

Parameters:directory – The pathname of the directory (a string).
py2deb.hooks.find_bytecode_files(python_file)[source]

Find the byte code file(s) generated from a Python file.

Parameters:python_file – The pathname of a *.py file (a string).
Returns:A generator of pathnames (strings).

Starting from Python 3.2 byte code files are written according to PEP 3147 which also defines imp.cache_from_source() to locate (optimized) byte code files. When this function is available it is used, when it’s not available the corresponding *.pyc and/or *.pyo files are located manually by find_bytecode_files().

py2deb.hooks.create_alternatives(package_name, alternatives)[source]

Use update-alternatives to install a global symbolic link to a program.

Install a program available inside the custom installation prefix in the system wide executable search path using the Debian alternatives system.

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
py2deb.hooks.cleanup_alternatives(package_name, alternatives)[source]

Cleanup the alternatives that were previously installed by create_alternatives().

Parameters:
  • package_name – The name of the system package (a string).
  • alternatives – The relevant subset of values in alternatives.
py2deb.hooks.initialize_namespaces(package_name, modules_directory, namespaces)[source]

Initialize Python namespace packages so they can be imported in the normal way.

The setuptools project introduced the concept of “namespace packages” and the use of such packages is now fairly widespread. Because the contents of multiple namespace packages will (by definition) overlap the relevant __init__.py files cannot be included in the Debian binary packages generated by py2deb (this would result in packages whose contents conflict).

To support namespace packages without conflicts py2deb generates the relevant __init__.py files during package installation.

Parameters:
  • package_name – The name of the system package (a string).
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).
py2deb.hooks.cleanup_namespaces(package_name, modules_directory, namespaces)[source]

Clean up Python namespace packages previously initialized using initialize_namespaces().

Parameters:
  • package_name – The name of the system package (a string).
  • modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
  • namespaces – The namespaces used by the package (a list of tuples in the format generated by namespaces).
py2deb.hooks.touch(filename)[source]

The equivalent of the UNIX touch program in Python.

Parameters:filename – The absolute pathname of the file to touch (a string).
class py2deb.hooks.NameSpaceReferenceCount(modules_directory)[source]

Persistent reference counting for initialization of namespace packages.

__init__(modules_directory)[source]

Initialize a NameSpaceReferenceCount object.

Parameters:modules_directory – The absolute pathname of the directory where Python modules are installed (a string).
__enter__()[source]

Load the persistent data file (if it exists).

__exit__(exc_type=None, exc_value=None, traceback=None)[source]

Save the persistent data file.

__getitem__(key)[source]

Get the reference count of a namespace (defaults to zero).

__setitem__(key, value)[source]

Set the reference count of a namespace.

py2deb.tests - Test suite

The py2deb.tests module contains the automated tests for py2deb.

The makefile in the py2deb git repository uses pytest to run the test suite because of pytest’s great error reporting. Nevertheless the test suite is written to be compatible with the unittest module (part of Python’s standard library) so that the test suite can be run without additional external dependencies.

py2deb.tests.setUpModule()[source]

Prepare the test suite.

This function does two things:

  1. Sets up verbose logging to the terminal. When a test fails the logging output can help to perform a post-mortem analysis of the failure in question (even when its hard to reproduce locally). This is especially useful when debugging remote test failures, whether they happened on Travis CI or a user’s local system.
  2. Creates temporary directories where the pip download cache and the pip-accel binary cache are located. Isolating the pip-accel binary cache from the user’s system is meant to ensure that the tests are as independent from the user’s system as possible. The function tearDownModule() is responsible for cleaning up the temporary directory after the test suite finishes.
py2deb.tests.tearDownModule()[source]

Clean up temporary directories created by setUpModule().

py2deb.tests.create_temporary_directory()[source]

Create a temporary directory for the test suite to use.

The created temporary directory will be cleaned up by tearDownModule() when the test suite is being torn down.

Returns:The pathname of the created temporary directory (a string).
class py2deb.tests.PackageConverterTestCase(methodName='runTest')[source]

unittest compatible container for the test suite of py2deb.

create_isolated_converter()[source]

Instantiate an isolated package converter.

test_argument_validation()[source]

Test argument validation done by setters of py2deb.converter.PackageConverter.

test_version_reformatting()[source]

Test reformatting of Python version strings.

test_conversion_of_simple_package()[source]

Convert a simple Python package without any dependencies.

Converts coloredlogs and sanity checks the result. Performs several static checks on the metadata and contents of the resulting package archive.

test_custom_conversion_command()[source]

Convert a simple Python package that requires a custom conversion command.

Converts Fabric and sanity checks the result. For details please refer to py2deb.converter.PackageConverter.set_conversion_command().

test_duplicate_files_check()[source]

Ensure that py2deb checks for duplicate file conflicts within dependency sets.

Converts a version of Fabric that bundles Paramiko but also includes Paramiko itself in the dependency set, thereby causing a duplicate file conflict, to verify that py2deb recognizes duplicate file conflicts.

test_conversion_of_package_with_dependencies()[source]

Convert a non trivial Python package with several dependencies.

Converts deb-pkg-tools to a Debian package archive and sanity checks the result. Performs static checks on the metadata (dependencies) of the resulting package archive.

test_conversion_of_extras()[source]

Convert a package with extras.

Converts raven[flask]==3.6.0 and sanity checks the result.

test_namespace_package_parsing()[source]

Test parsing of namespace_package.txt files.

test_conversion_of_binary_package()[source]

Convert a package that includes a *.so file (a shared object file).

Converts setproctitle==1.1.8 and sanity checks the result. The goal of this test is to verify that pydeb properly handles packages with binary components (including dpkg-shlibdeps magic). This explains why I chose the setproctitle package:

  1. This package is known to require a compiled shared object file for proper functioning.
  2. Despite requiring a compiled shared object file the package is fairly lightweight and has little dependencies so including this test on every run of the test suite won’t slow things down so much that it becomes annoying.
  3. The package is documented to support Python 3.x as well which means we can run this test on all supported Python versions.
test_conversion_of_binary_package_with_executable()[source]

Convert a package that includes a binary executable file.

Converts uwsgi==2.0.14 and sanity checks the result. The goal of this test is to verify that pydeb preserves binary executables instead of truncating them as it did until issue 9 was reported.

test_install_requires_version_munging()[source]

Convert a package with a requirement whose version is “munged” by pip.

Refer to py2deb.converter.PackageConverter.transform_version() for details about the purpose of this test.

test_conversion_of_isolated_packages()[source]

Convert a group of packages with a custom name and installation prefix.

Converts pip-accel and its dependencies to a group of “isolated Debian packages” that are installed with a custom name prefix and installation prefix and sanity check the result. Also tests the --rename=FROM,TO command line option. Performs static checks on the metadata and contents of the resulting package archive.

test_conversion_with_configuration_file()[source]

Convert a group of packages based on the settings in a configuration file.

Repeats the same test as test_conversion_of_isolated_packages() but instead of using command line options the conversion process is configured using a configuration file.

check_converted_pip_accel_packages(directory)[source]

Check a group of packages converted with a custom name and installation prefix.

Check the results of test_conversion_of_isolated_packages() and test_conversion_with_configuration_file().

test_python_callback_from_api()[source]

Test Python callback logic (registered through the Python API).

test_python_callback_from_dotted_path()[source]

Test Python callback logic (through a dotted path expression).

test_python_callback_from_filename()[source]

Test Python callback logic (through a filename expression).

check_python_callback(expression)[source]

Test for Python callback logic manipulating the build of a package.

test_find_installed_files()[source]

Test the py2deb.hooks.find_installed_files() function.

test_bytecode_generation()[source]

Test byte code generation and cleanup.

This tests the generate_bytecode_files() and cleanup_bytecode_files() functions.

test_namespace_initialization()[source]

Test namespace package initialization and cleanup.

This tests the initialize_namespaces() and cleanup_namespaces() functions.

test_post_install_hook()[source]

Test the post_installation_hook() function.

test_pre_removal_hook()[source]

Test the pre_removal_hook() function.

run_post_install_hook(directory)[source]

Helper for test_post_install_hook() and test_pre_removal_hook().

check_test_namespaces(directory)[source]

Make sure the test name spaces are properly initialized.

py2deb.tests.py2deb(*arguments)[source]

Test everything including command line parsing by running py2deb’s main function.

We want the test suite to cover as much of py2deb as possible, so including the command line interface, however we don’t want to run py2deb as a subprocess because that would break test coverage measurements. This explains the purpose of the py2deb() function.

Parameters:arguments – The command line arguments to pass to py2deb (one or more strings).
py2deb.tests.find_package_archive(available_archives, package_name)[source]

Find the *.deb archive of a specific package.

Parameters:
  • available_packages – The pathnames of the available package archives (a list of strings).
  • package_name – The name of the package whose archive file we’re interested in (a string).
Returns:

The pathname of the package archive (a string).

Raises:

exceptions.AssertionError if zero or more than one package archive is found.

py2deb.tests.find_file(contents, pattern)[source]

Find the file matching the given filename pattern.

Searches the dictionary of Debian package archive entries reported by deb_pkg_tools.package.inspect_package().

Parameters:
  • contents – The dictionary of package archive entries.
  • pattern – The filename pattern to match (fnmatch syntax).
Returns:

The metadata of the matched file.

Raises:

exceptions.AssertionError if zero or more than one archive entry is found.

py2deb.tests.python_callback_fn(converter, package, build_directory)[source]

Simple Python function to test support for callbacks.