Dusty's Diverse Domain

Holger Krekel just released the 1.0 version of py.test. Py.test is a functional and unit testing tool that cuts out a lot of the annoying boilerplate found in the unittest module included in the Python standard library.

I do a lot of Django coding. Django has a built-in test engine based on unittest. Its annoying, but it does a few things (such as capturing e-mails and creating a test database) automatically so I’ve tended to use it rather than setting up py.test to take care of these things. Today I decided I’d rather use py.test for my latest project. Turns out its not that complicated:

# there are better ways to do these first three lines
import os, sys
sys.path.append('.')
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from django.conf import settings
from django.test.client import Client
from django.test.utils import setup_test_environment, teardown_test_environment
from django.core.management import call_command
 
def pytest_funcarg__django_client(request):
    old_name = settings.DATABASE_NAME
    def setup():
        setup_test_environment()
        settings.DEBUG = False
        from django.db import connection
        connection.creation.create_test_db(1, True)
        return Client()
    def teardown(client):
        teardown_test_environment()
        from django.db import connection
        connection.creation.destroy_test_db(old_name, 1)
    return request.cached_setup(setup, teardown, "session")
 
def pytest_funcarg__client(request):
    def setup():
        return request.getfuncargvalue('django_client')
    def teardown(client):
        call_command('flush', verbosity=0, interactive=False)
        mail.outbox = []
    return request.cached_setup(setup, teardown, "function")

Put that in your conftest.py and you can write tests like this:

def test_something(client):
    response = client.post('/some_url', {'someparam': 'somevalue'})
    assert "somestring" in response.content
    # Other assertions...

This uses the innovative py.test ‘funcarg’ mechanism to create a test database when testing starts, and refer to that database throughout the test run. The ‘django_client’ funcarg sets up a database when the session starts and deletes it when it finishes. The ‘client’ funcarg creates a similar client, but also resets the database after each test is run to ensure there are no interaction effects between tests.

I haven’t fully tested it yet, but its nice to know I can get the most useful django test functionality so cheaply in py.test.

When I attended Pycon 2009, I saw a lightning talk on a neat little tool called Argparse. Its a python option parsing tool designed to be easier to use than optparse.

In the first few seconds of the talk, I thought to myself “I know where this is going. It’s brilliant.” I said so to the guy next to me. Then I took it back because while argparse is a pretty cool idea, its not the brilliant idea I thought it was.

See, I thought the tool would automatically introspect a script’s main method and use the docstring to provide any missing information. That’s what I thought would be brilliant.

Since it didn’t do that, I did it. Its certainly not full-featured yet, but it works for basic options and arguments. For 90% of scripts, that’s all you need. I called it opterator and put it on github:

http://github.com/buchuki/opterator/tree/master.

git clone git://github.com/buchuki/opterator.git

Here’s an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

from opterator import opterate
@opterate
def main(filename1, filename2, recursive=False, backup=False,
        suffix='~', *other_filenames):
    '''An example copy script with some example parameters that might
    be used in a copy command.
 
    @param recursive store_true -r --recursive copy directories
        recursively
    @param backup store_true -b --backup backup any files you copy over
    @param suffix store -S --suffix override the usual backup
        suffix '''
    filenames = [filename1, filename2] + list(other_filenames)
    destination = filenames.pop()
 
    print "You asked to move %s to %s" % (filenames, destination)
    if recursive:
        print "You asked to copy directories recursively."
    if backup:
        print "You asked to backup any overwritten files."
        print "You would use the suffix %s" % suffix
 
if __name__ == '__main__':
    main()

Obviously, that only parses the options, its not doing the actual copying because implementing that would do nothing to illustrate how opterator is working. Here’s one run of it:

dusty:opterator $ python copy.pyc -b a b c d -r
You asked to move ['a', 'b', 'c'] to d
You asked to copy directories recursively.
You asked to backup any overwritten files.
You would use the suffix ~

Here’s the automatically generated help:

dusty:opterator $ python copy.py -h
Usage: copy.py [options] filename1 filename2 [other_filenames]

An example copy script with some example parameters that might
    be used in a copy command.

Options:
  -h, --help            show this help message and exit
  -r, --recursive       copy directories recursively
  -b, --backup          backup any files you copy over
  -S SUFFIX, --suffix=SUFFIX
                        override the usual backup suffix

I named it opterator because I wanted a name that didn’t have ‘parse’ in it. I was going to go for operator but I realized if you heard about this app called operator and searched google for ‘python operator’ you’d get swamped. ‘opterator’ is unique and highlights the option part of the task. Plus it allowed me to invent a new verb: ‘opterate’. Days when I get to invent new verbs are always good days.

I used py.test for testing (another gem I discovered at pycon). You don’t need to run the tests, but the tests are useful examples to see what opterator can do, so have a look.

So I hope somebody finds this useful or even awesome and contributes some patches to make it more of the above. Enjoy!