Archive for August, 2010

Łukasz Czyżykowski

tox – New Tool For Testing Python Libraries

What is tox?

tox is a tool for testing python libraries (or more generally Python projects) against multiple Python implementations. Before tox, this would have been quite involved and any project wanting to check had to create its own solution. With tox it’s just a matter of creating a simple .ini file and running it as often as you need.

It uses virtualenv to properly manage and separate environments.

It’s compatible with all Python versions starting from 2.4, currently up to n 3.2. Support for Jython and PyPy is also included.

How to use it?

First thing is to create tox.ini file in your project’s directory, next to the setup.py. Format is quite simple. Here’s example:

[tox]
envlist = py26,py27

[testenv]
deps = nose
commands = nosetests

Where envlist is a list of all Python implementations you want the tests (py26, py27 are built-in, one can also create custom environments) to run in. deps is a list of all dependencies your package requires, other than packages specified in setup.py file. Finally commands is a list of all commands which tox will run in that test environment. Here it’s only running nose, but you can also have commands for building documentation or anything else that you want to ensure that it works.

With this file in package directory running tox is just a matter of:

$ tox

It will create each environment, install required packages (using pip), run all the commands and finally report success or failure. It saves all output to log files and caches all downloaded files, so no need worry about that.

Real life example

To test tox on a real project I’ve chosen configglue, our configuration library. It is always a good idea to make sure it runs properly on various Python versions. So, there will be no surprises in the future.

First step was to download the source code ($ bzr branch lp:configglue). Next in the project’s directory. I’ve created very simple tox.ini file displayed below:

[tox]
envlist = py25,py26,jython

[testenv]
deps = nose
commands = nosetests []

[testenv:jython]
basepython=/path/to/jython

As you can see, one addition is testing against Jython.

Another small caveat with configglue is that one file in the source code requires small addition to be compatible with Python 2.5 (from __future__ import with_statement). That doesn’t ensure that all tests will pass, it just prevents syntax error from being raised.

As you can see you can customize each test environment separately by using testenv:pyver section name.

Running tox with configglue reveals that it works perfectly on Python 2.5 and 2.6 but one test fails on Jython (offending line _open = __builtins__['open']).

Final Notes

Additionally tox has support for working with Hudson and should be fairly straightforward to integrate with other CI solutions.

Because this project is really young (released on June 2010) there’s not so many real life examples. One of them is from mock library; it’s tox.ini is fairly complex, as it includes building documentation using Sphinx.

You can find an even more complicated example in the kombu source code.

tox.ini in the tox source code has an example of integration with Hudson.

Of course, tox site contains a list of examples.