Posts tagged ‘python’

Creating an Application in Kivy: Part 1

This is the first in what I expect to be a series of tutorials on creating user interfaces in Kivy. Kivy is a cross platform user interface framework that allows us to write applications in Python that run on various desktop operating systems, Android, and IOS.

I’ve wanted to write a tutorial on Kivy since hacking on Python 3 support during the dev sprints at Pycon 2013. My goal is to create a useful application that runs on multiple devices and highlights use of the KV Language to design user interfaces. I also want this multi-part series to describe end-to-end how to develop, maintain, and deploy a Kivy application. I intend it to be a model for newcomers to Kivy and even Python to develop their own cross platform or mobile applications in Python. Thus, it doesn’t just include code, but also deployment instructions and information on how to use the git version control system for basic history management.

Therefore, this tutorial series is written at a more basic level than a lot of my technical articles. My intended audience is new programmers who have some basic Python experience; perhaps having read The Python Tutorial and Learn Python The Hard Way, but possibly not the beginner-intermediate topics covered in my book, Python 3 Object Oriented Programming.

Having decided to write a tutorial, I needed to decide what kind of application to develop. This actually took quite a bit of thought. I decided on a Jabber client, as it has a user interface with reasonable complexity, and I am hoping that most of the difficult bits can be abstracted away into the SleekXMPP library. Finally, I had to settle on a name for the app; I chose Orkiv. I do not know why.

Table Of Contents

A blog isn’t the best platform for publishing a multi-part tutorial. I’ll include a Table Of Contents section in each part with references to all the other parts. Hopefully I’ll even keep it up to date!

  1. Part 1: Introduction to Kivy
  2. Part 2: Creating a Form
  3. Part 3: Handling Events
  4. Part 4: Code and Interface Improvements
  5. Part 5: Rendering a Buddy List
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Width based layout
  9. Part 9: Deploying your Kivy application

Prerequisites

Start by installing the following pre-requisites using standard operating system tools.

Python is the programming language and interpreter that Kivy programs are written in. Git is a version control system that we will use to track changes to our code. Virtualenv is a tool for creating isolated Python environments. Pip is an installer for installing python packages, in this case into the isolated virtualenv.

Note: Use of git is optional if you are more interested in learning to code than learning to manage a project. You are also welcome to use another version control system of your choice. At the level we will be working, the commands are virtually interchangeable.

Setting up the environment

Run the following commands in a terminal. (Note this tutorial was written using Arch Linux, and will probably work flawlessly on MacOS and other Linux distributions. However, you may need to do some experimenting to make Windows cooperate).

mkdir orkiv
cd orkiv
git init
virtualenv -p python2.7 venv
echo venv >> .gitignore
echo "*.pyc" >> .gitignore
git add .gitignore
git commit -m "Create .gitignore file. Ignores compiled python files and venv."
source venv/bin/activate

We’re creating a directory to hold our project and then initialize a git repository in there. We next create a virtualenv in a folder named venv. This folder will hold the isolated python files so that you can interact with Kivy without adding cruft to your system Python. I highly recommend creating a separate virtualenv for every project you work on.

Next, we set up a .gitignore file that tells git not to track certain files in our repository; in this instance all compiled python files, which end in .pyc, and the virtualenv directory we just created. We commit the changes to this file as our first commit in the git repository.

Finally, we “turn on” the virtualenv by sourcing its activate script. This essentially tells our shell to “use the isolated python for this project instead of system python”. When you are done working on this project, you should enter the command deactivate. When you come back to the project in the future, reenter the source venv/bin/activate command to turn on the isolated environment.

Note: If you maintain a lot of virtualenvs in a lot of different projects, as I do, you may be interested in a small script I wrote to facilitate switching between them.

Kivy Dependencies

Kivy depends on several Python libraries. Unfortunately, it does not have a setuptools-enabled setup.py to automatically install these dependencies into our virtualenv, so we have to do a little work ourselves. This takes a while, but if you copy paste these commands into your terminal, you should get lucky.

pip install cython
pip install pygame

Sadly, on Arch Linux, the last command, for pygame fails. I suspect it works fine on less bleeding edge operating systems, however, if you encounter an error about linux/videodev.h not existing, applying the Arch Linux patch to pygame may get you to the next step.

wget http://www.pygame.org/ftp/pygame-1.9.1release.tar.gz
wget https://projects.archlinux.org/svntogit/packages.git/plain/trunk/pygame-v4l.patch?h=packages/python-pygame -O pygame-v4l.patch
tar xf pygame-1.9.1release.tar.gz
cd pygame-1.9.1release/
patch -Np1 -i ../pygame-v4l.patch
python setup.py install
cd ..
rm pygame* -r

And now, you should finally be ready to install Kivy itself: This will take a while, so grab a smoothie while it runs:

pip install kivy
python -c 'import kivy'

The latter command should output the kivy version (I’m working with 1.7.1). If it exits without failure, then you have successfully installed Kivy!

Now let’s create a basic Kivy app

Create a directory to hold the application code:

mkdir orkiv

This directory will contain your Python and Kivy source files. Our goal is to be able to always create a zipfile from this directory and be able to run it using python orkiv.zip. As long as the required dependencies are installed on the target system (as described above), the program should run. It will therefore be nice and self-contained.

There is a relatively unknown feature of recent versions of Python to support this. If a directory or zipfile contains a __main__.py, that module will be executed if python is called with that directory or zipfile as an argument. First, create a file inside the new orkiv directory named __main__.py. Create a simple “hello world” in this file to test that
it’s working:

(commit)

print("hello world!")

Now from the parent directory, run the command python orkiv. If you want to test it with a zipfile, try this:

cd orkiv
zip ../orkiv.zip *
cd ..
python orkiv.zip

It is possible to code Kivy applications in pure Python, but in my opinion, it is much better to use the amazing KV Language to do layouts. The basic approach is to have a Python file (in our case __main__.py above) that contains the logic for the application, and a KV Language file that contains the layout information. The KV Language is very similar to Python and you can learn it as you go.

Lets start by removing the print statement from our __main__.py and replacing it with the most basic possible Kivy application:

(commit)

from kivy.app import App


class Orkiv(App):
    pass

Orkiv().run()

This simply imports a Kivy App class and creates a subclass of it using Python’s notoriously simple inheritance syntax. Inheritance is a useful feature of Object Oriented Programming that basically means that a class can be defined that has all the functionality of the parent class. In this case, since we didn’t add anything to the subclass, that’s ALL it has. However, the true beauty of inheritance is that we can add new properties and methods (functions attached to objects) to the subclass or even change the functionality that comes with the parent class (App). The best part is, we don’t really have to know what is going on inside the App, and can assume the Kivy devs know what they are doing.

It doesn’t even add anything to the subclass! Then it instantiates an instance of that subclass and calls the run() method on the newly created object.

If you run python orkiv with the new code saved, you’ll see an empty window pop up.

As far as boilerplate goes, that’s pretty damn concise, don’t you think? The amazing thing is that we can start laying out KV Language widgets in a separate file without touching this boilerplate at all! Don’t believe me? Try saving the following as orkiv/orkiv.kv:

(commit)

Label:
    text: "hello world"

Now if you run python orkiv, you should see the “hello world” label centered in the window that pops up. It’s almost like magic, and if you’re like me, you’re probably wondering how that’s even possible.

When you create a subclass of a kivy.app.App, it introspects the name of the new class (we named it Orkiv). Then it looks for a file in the same directory that follows these rules:

  1. Ends with the .kv extension
  2. Starts with the name of the class converted to lowercase and with any
    trailing App stripped.

The Kivy documentation would have named the class OrkivApp and still used the orkiv.kv filename. Personally, I don’t see the redundant need to add App to the class name, since the inheritance structure clearly indicates that Orkiv is a App.

We’ll be working with the Kivy Language a lot as we proceed through each part of this tutorial. You’ll soon see that it’s possible to do more, a lot more, than add a label to a window!

A note on version control

At this point, you’ve created a logical related set of changes that are in a working state. I want to encourage you to experiment with these files, maybe change the contents of the label, try a different application and KV Language filename, or see if you can figure out how to group two labels into a single layout. But before doing that, you should record this known state that your repository is currently in so it’s easy to get back to. That way, you can be free to explore without fear of getting completely lost; you’ll always have a path straight back to your current state. You can do that from the command line using git:

git add orkiv/
git commit -m "A basic Kivy boiler plate"

You just made a commit on the master branch in git. First you listed the changes you wanted to include in the commit (the entirety of every file in the new orkiv directory), then you told git to record the current state of those files forever. It is generally recommended that you only save consistent, known-to-be-working state on the master branch. For the purposes of this tutorial, I suggest that you only save the code you copied from the tutorial to the master branch, and do your other development on other branches.

Are you confused by what a branch is, exactly? Think of it like walking down a forest path in a national park. You are following signs for a marked trail; that’s this tutorial. As you walk, you see things that you want to remember, so you take a photo. That’s like making a commit on the master branch. However, you also see side paths that you would like to explore on your own. You can go down that path as far as you want, or even take other random paths without fear of getting lost. You can even take photos on those paths, knowing they won’t get mixed in with the photos from the main trail. Then when you want to return to the point where you left the main trail, you can magically teleport back to it (or indeed, to any of the other branches you walked down in your exploration).

You probably won’t be doing it for this tutorial, but the most import aspect of version control is that if you are in unmarked forest rather than a national park, you can walk down all the paths and decide which one you’re going to choose as the main path. You can even merge paths back into each other so that the photos (code commits) taken on each of them end up in the same final presentation.

Enough digressing! For your purposes, the first thing you should do is create a new branch in git:

git checkout -b my_exploration

You are now on a new branch named my_exploration (you can call each branch whatever you want, of course) that is exactly the same as the master branch as you last saw it. But you can make changes on that branch to your heart’s content; you can add and commit them as above if you want to remember them for later. When you’re ready to come back to the tutorial, you can use this command to return to the current state of the master branch:

git checkout master

From there, you could continue with the next part of the tutorial, or you could create a new branch and explore in a different direction.

Browsing the examples

I am maintaining my own version of the code developed in this tutorial at https://github.com/buchuki/orkiv/. However, it’s not likely to be the same as your repository, since I’m not making commits at the same time that I’m suggesting you make commits. Instead, my master branch contains a separate commit for each example in the tutorial. I also create git tags for each part of the tutorial so it’s fairly easy to see from commit history what was covered in each part. If you browse the Commit History you can see each example and how it evolved from top to bottom. You can view the list of tags here.

Monetary feedback

When I wrote this tutorial, I didn’t expect it to be big or popular enough to publish as a book. However, the series caught O’Reilly’s eye, and I have since written an entire book on Kivy. If you want to support me and learn more about Kivy, you and I both will be delighted if you buy it.

If you like the tutorial and would like to see similar work published in the future, this isn’t the only way to support me. I hope to one day reduce my working hours at my day job to have more time to devote to open source development and technical writing. If you think this article was valuable enough that you would have paid for it, please consider thanking me in one or more of the following ways:

If you aren’t in the mood or financial position to help fund this work, at least share it on your favorite social platforms!

Next Year: addressing the more subtle sexism at Pycon

From a gender equality point of view, I’d call Pycon 2013 a success, though perhaps a better word is “progress”. The gender ratio apparently doubled to 20% this year. If that continues (depending if it’s a constant, linear, or quadratic curve) we could be at parity in as little as two to three years.

Another indicator of forward momentum is that women this year were able to behave, to use a sexist term, “like women,” without fear of reprisal. That is to say, I saw makeup, heels, and skirts, and they did not seem out of place. This is not to suggest that women should or must dress up to attend conferences; the traditional vendorware is equally acceptable for all genders. However, it is progress that more women felt they had a choice in the matter and did not feel an obligation hide their physical differences. I think the fact that there was more variety among the male dress styles is also a sign of success.

The worst charges coming out of Pycon 2013 were the use of inappropriate jokes. To the best of my knowledge, there were no reports of women being harassed, assaulted, or groped during the conference. I’m not aware that this has been an issue at past Pycons either, but I think it’s a sign that the Python community is better behaved in this regard than some of her sister tech conferences.

So all in all, while it wasn’t completely successful, Pycon 2013 was a terrific step on the road to eliminating sexism and creating gender equality. Let’s make next year an equal sized step. Here are a few topics I think we, as a community, can work on to engender further progress.

I’m going to start with a couple links. Ruth Burr wrote a terrific essay titled Things You Think Aren’t Sexist, But Really Are a couple weeks ago. In summary: Men, you are instructed to go to conferences to network and interact, not to date or hook up.

Yes, you might meet the love of your life at Pycon, but don’t expect, intend, or plan for that to happen (at PyCon or anywhere). Behave professionally, and consider each woman you meet for her knowledge and intelligence, not her figure or her marital status, just as you would when interacting with a male.

Ned Batchelder, with his amazing talent for understanding and explaining problems wrote another great article on the root issues. Summarized, he says friction is inevitable, and that education is better than shunning when people make mistakes.

I started planning this article while attending Pycon this year. I had to scrap those plans after the so-memed “donglegate” fiasco. I don’t have anything to add to that discussion, but as Ned illustrated, there were many many instances of similarly inappropriate comments at Pycon. I myself made such a joke, most of my friends did so, I overheard a conference organizer say something inappropriate. This is not sexism per se, but sexual and other potentially offensive comments need to be reduced. Comments laced with sexual innuendo do not belong in a professional or a family setting, and Pycon is intended to be both.

Let’s start with interation between the genders. In her article, Ruth Burr mentioned that women can feel awkward when they interact with men because the men assume they are flirting rather than interested in the topic at hand. Many men see cleavage and assume the speaker can’t possibly know as much as them about whatever topic that is. Some girls avoid interacting with men because of this awkward sensation, and if they do, the interactions are tainted.

I had a related problem. While I comfortably talked to men of varying skill levels at Pycon, I felt uncomfortable addressing women because I was afraid of sending some vibe that I was being flirtatious. So I tended to ignore the female attendees. This is unfortunate for everyone who missed out on that potential conversation. Pycon 2014 needs to increase the interaction between the genders, not just the attendance ratios. Ladies, talk to the shy guys, they need to be taught that you aren’t that intimidating. Guys, talk to the women, they need to learn that we are interested in what they have to say about tech, and not their figures. As it stands, only the sexist guys are interacting with the women, and it makes us all look bad.

The gender ratio really fell during the developer sprints. I don’t think there were any female sprint leaders presenting projects to hack on. Worse, I’d estimate between one and two percent of the sprint attendees were women. This is a huge area for improvement. To the various diversity groups out there, please encourage your members to stay an extra day or two and attend the Pycon dev sprints. Get them involved with Open Hatch if they are unsure how to contribute to open source projects. Sprint leaders, make sure female attendees are welcome, especially if they are new coders (indeed, make all new coders welcome).

I noticed a lot of Impostor syndrome among the female attendees and speakers. Programming really is easy. The fact that you enjoy it and find it quite simple is not a sign that you don’t know the “hard stuff” and therefore you’re not a “real programmer.” Quite the reverse, in fact. It’s not necessary to apologize for your lack of knowledge if you’ve been invited to do a talk or are about to join an open source project for the first time. Find ways to build your confidence (contributing to open source and getting feedback is a great start), and start believing in your skills. Even if you don’t believe it yourself, try to project confidence in your coding abilities; it will send a much more effective signal that women are capable and here to stay. Don’t worry if you over-present yourself; faking your way through it is a great way to find out that you’re actually better than you thought!

I’d like to close with an admonishment to those people who open their articles on sexism with a discussion of their gender. These usually take one of two forms: the disclaimer and the shocker. The disclaimer is most often used by men and sounds like, “I am a privileged white male so I don’t understand what it’s like for women, but I still think I have something valid to say on this topic”. The shocker is used by both genders and sounds like, “I am a man/woman, so you’re going to be amazed that my opinion on this topic is different from other members of my gender.” Yes, it is useful to state your bias and frame of reference. However, be cautious that your purpose in doing so is to remove distortion from the lens you are applying to the discussion, not to add to it.

And now, feel free to analyze my bias in this article. I am a privileged white male from a blue-collar family. I’ve had many amazing opportunities in the tech industry. I have very little experience as the target of discrimination, outside my mental illness. My interest in feminist issues comes from my sister’s master’s degree on the subject; I proofread most of her undergraduate and graduate level essays, and gained a relatively deep understanding of the topic. I acknowledge that I am a racist, sexist jerk by culture and conditioning. Sometimes I forget to compensate for that. I find it exceptionally easy to overlook the patriarchy when it’s doing me favors, and I will never be as keenly aware of its negative impact as someone who is directly experiencing it every day.

Guido Van Rossum Should Retire (and focus on python)

At the two previous Pycons I’ve attended (2009 and 2012), Guido Van Rossum’s keynotes sounded bored and uninterested, even though the content was meaningful. I was actually wondering if this would be the year that he would step down from BDFL of Python. Thankfully, I was dead wrong.

Instead, he presented a highly technical and very exciting addition to the Python language. Alfredo told me this started when he took a month off between stepping down at Google and starting at DropBox. Now, when normal people take a month off, they relax or travel or visit friends and family. Not our BDFL. He writes a callback-free asynchronous event loop API and reference implementation that is expected to massively alleviate Python’s oft-maligned lack of a consistent, unhackish concurrency solution.

Let’s have more of that. What if Mr. Van Rossum could hack on Python full time? Would we see quantum progress in Python every month?

Anyone who knows about the Gittip project likely thinks they can guess where this is going. We, the people, can each tip our BDFL a few cents or dollars per week so he can focus on whatever he deems worthy. It’s safe to assume that a man who spends his vacation time drafting a new Python library would choose to work on Python full time if we funded him.

This plan is great, and I actually think that Guido could easily earn enough to quit his day job if he endorsed Gittip and invited individuals to tip him. But I’d like to discuss a different plan: Not individuals, but companies should tip Guido the maximum gittip amount as a sort of “partial salary”. At $1248 per year, most companies wouldn’t even notice this expense, and they would get a better programming language and standard library in return. The rate of accelerated development would be even higher if each of these companies chose to invest an entire salary, split between a hundred Python core and library developers. If a hundred companies chose to do this, those hundred people could work on Python full time. The language and library would improve so vastly and so rapidly that the return on investment for each of those companies would be far greater than if they had paid that same salary to a single developer working on their in-house product, full time.

It might take some convincing to justify such a strategy to these corporations. Companies tend to like to know what is happening to their money, and simply throwing a hefty developer salary at Gittip would be hard to justify. Obviously “goodwill” could support some of it, in the same way that so many companies sponsored Pycon in exchange for exposure.

Cutthroat CEOs should perhaps consider not just the value that having Guido working exclusively on Python is, but also the cost of having him work for the competition. I’m sure Box.com CEO Aaron Levie was a little nervous when he found out that the first and greatest Python programmer of all time had recently hired on at a major competitor. Perhaps Box.com can’t afford to steal Guido from Dropbox, but if all the companies currently involved in cloud storage were to tip Guido $24 per week on Gittip, this incredible programmer could be working on an open source product that directly and indirectly benefits their company rather than improving the competing product on a full-time basis.

Most of the arguments that Gittip will fail are based on the premise that not enough money can be injected into the platform to sustain full time development by open source programmers. However, if an open and caring relationship can be built such that the corporate world is also funding the system, I think it can become extremely successful. Everyone will benefit: Open source projects will improve at a rapid pace. Exceptional developers will get to pursue their passions. End users will get better products. The overall level of happiness in the world will be higher.

I would like to see a world where brilliant young software engineers are not risking their mental health (and consequently, their lives) on startup ideas in the hopes of being bought out for a few billion dollars. I would like to see a world where those engineers are not working for large corporations that have neither their employees nor their end users (but rather, their stockholders and advertisers) interests at heart. I would like to see a world where those developers can choose to invest their passion in open source products that will change the world.

Excluding tests with py.test 2.3.4 using -k selection

When I use py.test, I often rely on the -k switch to swiftly select the test I want to run. Instead of having to type the full module, class, and test path as is required with unittest and nose, I can just type a few characters that uniquely match the name of the test.

For example, if I have a test file containing methods test_basic_clone and test_basic_clone_notes, I can run the latter test simply by calling py.test -k clone_no.

However, I often create multiple tests that have similar names. This can make it difficult to run just one test if the test name is a prefix of a longer test name. If I want to run just test_basic_clone, any substring will also be a substring of the test_basic_clone_notes test, and both tests are matched by -k.

Since pytest version 2.3.4, the -k keyword supports expressions. So I can build an expression like this:

py.test -k "basic_clone and not notes"

This selects all tests matching “basic_clone”, then excludes any containing the word “notes”. Thus, I run only the test I’m interested in without having to fix my crappy naming scheme. It’s more typing than is normally the case, but is still less cognitive load than trying to remember what module and class I’m editing and constructing a selector based on those attributes.

Gitifyhg is now awesome.

Gitifyhg is a git client for pushing to and pulling from Mercurial repositories. I described the first implementation, which used hg-git internally, last month. However, I found that it didn’t work as well in practice as my initial tests had indicated, and I morosely reverted to using hg directly.

While researching ways to improve the app, I stumbled across git-remote-hg by Felipe Contreras. It claimed to be a cure-all to the git-hg bridging problem that “just worked”. So I downloaded and tried it, but it didn’t live up to the author’s hype. While it could handle basic cloning, pushing to and pulling from the default branch, it failed for me when working with named upstream branches, something I have to do regularly in my day job. I submitted several issues and pull requests, most of which were ignored. The more deeply involved I became with the code, the more I felt a complete rewrite was in order.

I had some free time during the holiday season, and started my version of a git remote as an exercise, with no intent to create anything useful. To my surprise, something useful emerged. In fact, I believe gitifyhg is the most robust and functional git to hg client currently available. More importantly, it is eminently hackable code: well tested and fairly well documented. I hope this will make it easy to contribute to, and that my inbox will soon be full of pull requests.

This is the real deal. The best part is that you don’t have to learn any new commands. It’s just basic git with mercurial as a remote. The only command that has changed from normal git usage is clone:

pip install gitifyhg
git clone gitifyhg::http://selenic.com/hg
cd hg

By adding gitifyhg:: before the mercurial url, you can git clone most mercurial repositories. If you can’t, it’s a bug. Other complex repositories I have successfully cloned include py.test and pypy.

You can easily use gitifyhg in the fashion of git-svn. All named branches are available as remote branches. default maps to master. Other branches map to branches/<branchname>.

If you want to commit to the master branch, I suggest a workflow like this:

git clone gitifyhg::<any mercurial url>
cd repo_name
git checkout -b working  # make a local branch so master stays prestine
# hack and commit, hack and commit
git checkout master
git pull  # Any new commits that other people have added to upstream mercurial are now on master
git rebase master working  # rebase the working branch onto the end of master
git checkout master
git push

Working on a named mercurial branch, for example feature1, is easy:

git checkout --track origin/branches/feature1
git checkout -b working  # make a local branch so feature1 stays prestine for easy pulling and rebasing
# hack and commit, hack and commit
git checkout branches/feature1
git pull  # New commits from upstream on the feature1 branch
git rebase branches/feature1 working  # rebase the working branch onto the end of feature1
git checkout master
git push  #push your changes back to the upstream feature1 branch

It is even possible to create new named branches (assuming my_new_branch doesn’t exist yet in Mercurial):

git checkout -b "branches/my_new_branch"
# hack add commit
git push --set_upstream origin branches/my_new_branch

These basic workflows have been working flawlessly for me all week. In contrast to my previous attempts to use git to hg bridges, I have found it easier to use gitifyhg than to work in the mercurial commands that I have become expert with, but not used to, in the past year.

Gitify hg is not yet perfect. There are a few issues that still need to be ironed out. There are failing tests for most of these in the gitifyhg test suite if you would like to contribute with some low-hanging fruit:

  • Anonymous branches are dropped when cloned. Only the tip of a named branch is kept.
  • Tags can be cloned and pulled, but not pushed.
  • Bookmarks can be cloned and pushed, but not pulled reliably. I suspect this is related to the anonymous branch issue.

So give it a shot. Gitifyhg is just one easy_install gitifyhg away.

Opterator, revisited

A few years ago, I wrote a simple decorator that introspects a main() method signature and docstring to generate an option parser. I never really used it, mostly because back then, I wasn’t too keen on third party dependencies. Nowadays with distribute being maintained, well-documented, and working (plus I know how to use it properly), I no longer have this concern.

A friend recently linked me to the brilliantly designed docopt and reminded me of my interest in opterator. I revisited my code and decided it’s a pretty neat little design. So I ported it to Python 3. This took a matter of minutes, largely because I originally wrote it shortly after taking the pytest tutorial at Pycon 2009 and it was easy to find the failing code. It now supports Python 2.6, 2.7, and 3.3, according to tox.

I originally designed opterator to be a full replacement for optparse and friends. However, my main purpose for it now is to create quick and dirty command line applications, often for my personal use. These apps usually have a couple of options and it doesn’t seem worth the trouble of setting up optparse. Yet, mucking around with sys.argv is also annoying. Opterator minimizes the boilerplate. Check out this basic example:

from opterator import opterate                      


@opterate                                           
def main(filename, color='red', verbose=False):     
    print(filename, color, verbose)

main()                                              

with three lines of boilerplate code, a function can be turned into a command line program that can be called like so:

  $ python examples/basic.py this_file
  this_file red False

or so:

  python examples/basic.py this_file --color=blue
  this_file blue False

or even so:

  $ python examples/basic.py --color=purple another_file --verbose
  another_file purple True

And you even get a not totally (but somewhat) useless helpfile:

  $ python examples/basic.py -h
  Usage: basic.py [options] filename



Options:
  -h, --help            show this help message and exit
  -c COLOR, --color=COLOR
  -v, --verbose

Building a Python Kivy App in Android (Easier than it looks, but harder than it needs to be)

When I introduced Kivy a while back, I mentioned that I had not been able to deploy a Kivy app on Android. I’ve managed to do this now, using the VirtualBox image provided by the Python For Android project. Using this image was substantially less work than trying to figure out how to install and set up all the prerequisites under Arch Linux. So here’s how to do it:

First download the VirtualBox image from Google Drive. Sadly I cannot find an adequate wget command to download this. Shame on you, Google!

You may want to double check the documentation to see if a later image has been made available.

It’s a big honkin’ file (1.4GB), so grab a cup of coffee, lunch, or go to bed, depending on your bandwidth and the current time of day.

Extract the file using p7zip. This takes a while, so you might want to grab another coffee:

7z e Python\ for\ Android\ -\ Ubuntu\ 12.04.7z

Make sure VirtualBox is installed and the kernel module is loaded. The commands on Arch Linux are:

pacman -S virtualbox virtualbox-host-modules
sudo modprobe vboxdrv

Fire up VirtualBox and hit Machine->Add.

Navigate to the folder the vbox image was extractedto. Select the Python for Android - Ubuntu 12.04.vbox machine and hit open.

If you have enough RAM, select the Python for Android machine and hit settings. Browse to the System tab and increase the RAM to a comfortable level (I set it to 2GB on my 4GB machine). You might also want to increase the number of processors and the video RAM. Hit Start to power on the machine.

Once Ubuntu has loaded, review the README for some important information. You may want to update the VirtualBox Guest Additions install, although I’m sure it will work fine without it.

Now you’re ready to build an APK from a Python project. Open a terminal from the menu on the sidebar. The first thing you’ll want to do is update the python-for-android project to the latest git checkout. This is a bit messy because there are a couple edits in the current checkout that were later fixed upstream. So you’ll want to stash the changes before pulling:

cd android/python-for-android
git stash
git pull
git stash apply

Unfortunately, this introduces a merge conflict in distribute.sh. The upstream project has made some of the customizations obsolete by including more generic code. Luckily, it’s a trivial fix. Open distribute.sh in an editor (gedit distribute.sh works, but I used vim.) and search for the characters =======. It’s at line 170. Delete the line above that says <<<<<<< Updated upstream. Leave the export line that was above the =======. Then delete the ======= and >>>>>>> Stashed changes and the outdated export line in between. Save the file and close the editor.

Now you need a Kivy application to install. Presumably you’ve written your own, but for this example, I’m going to use a cool one supplied by the Kivy team. The Kivy Remote Shell creates an ssh server on the phone that allows you to log in from your computer over ssh and execute Python commands on the phone. You can even run Java commands using the beta pyjnius connector.

These commands will prepare a python for android distribution ready for installation:

cd ..
git clone git://github.com/kivy/kivy-remote-shell
cd python-for-android
./distribute.sh -m 'openssl pycrypto pyasn1 pyjnius twisted kivy'

This will prompt to see if you want to overwrite the previous python for android distribution. Press <enter> to do so. Give it some time to connect to the network, download the dependencies from Pypi, and compile them.

Now you need a rather ugly command to actually build the apk. The options are summarized in the Python for Android documentation.

cd dist/default
./build.py --package org.kivy.sshshell --name "Kivy Remote Shell" \
  --version 1 --dir ../../../kivy-remote-shell/ \
  --icon ../../../kivy-remote-shell/icon.png --permission INTERNET debug

This will create the file bin/KivyRemoteShell-1-.debug.apk. Now let’s install it to the phone! First make sure USB debugging is enabled on your phone. Go to the Settings-->Developer Options menu and make sure Developer Options are enabled and that Android Debugging is enabled.

Plug the phone into your computer via the USB cord. Click the little “USB” icon in the VirtualBox status bar and check the box next to the name for your phone.

Run the command ~/android/android-sdk-linux_x86/platform-tools/adb devices to make sure it outputs the identifier of your phone. If it works, simply run ~/android/android-sdk-linux_x86/platform-tools/adb install bin/KivyRemoteShell-1-debug.apk to install the app.

(If you can’t get the phone to connect to the VM, you can also copy the .apk to the host machine to install it. Use VirtualBox’s shared folders feature).

You can now shut down the virtual machine from Ubuntu’s shutdown menu. For security, it’s a good idea to turn off USB debugging on your phone until you need it again. Run the “Kivy Remote Shell” app that is now in your phone’s app drawer. Type the command that comes up onto your screen into a terminal on your computer, and you will have access to a Python prompt running on your phone!

This same process can be used to install self-developed Kivy apps to your phone. Congratulations!

Gitifyhg: Accessing Mercurial repos from GIT

My company uses Mercurial for internal hosting. Other than that, it’s an absolutely terrific place to work.

About three quarters of my colleagues are sick of hearing the rest of us complain about Mercurial’s inadequacies. They mention tutorials like this one that simply don’t work in real life. I’ve done my research, and I have not been able to find a viable git to hg pipeline anywhere. There is a git-hg repo, but it appears to be un-maintained and not overly well documented. It’s also written in bash, so I’m not eager to take over maintenance.

Instead, I’ve been spending some of my non-working hours trying to make a python wrapper around hg-git named gitifyhg. It started out as an automated version of the hg-git tutorials, but is now expanding to provide additional support and tools.

Right now I’m following a git-svn style of workflow, where your git master branch can be synced up with the hg default branch. It seems to be working somewhat ok. I think at this point, the pain of using gitifyhg is about equal to the pain of using hg alone. Hopefully future improvements will reduce that pain, and as always, patches are most welcome!

The instructions are pretty clear and the unit tests provide a pretty good description of basic usage. Non-basic usage probably doesn’t work (yet). Here’s a tutorial of what does work:

Acme corporation uses Mercurial for all their repositories. Fred and Wilma are developing a project named AcmeAdmin together. Fred is content to use Mercurial for hosting, but Wilma is a git advocate who is going crazy with the restrictions Mercurial places on her. She decides she’s going to try gitifyhg on this new project to see how much trouble it is.

Fred starts the project by creating a Mercurial repo that will serve as the remote repo for both of them:

mkdir acmeadmin
cd acmeadmin
hg init

Fred and Wilma are doing a very strange sort of pair programming where they have separate repositories on the same machine. They are doing this so if you were reading along, you could type in the same commands they are typing and the demo would work. Rest assured that in spite of this queer practice, Fred and Wilma are hotshot programmers. So Fred now clones this remote repo and starts working. In the meantime, Wilma is browsing the gitifyhg README, so she hasn’t cloned anything yet.

cd ..
hg clone acmeadmin fredacme
cd fredacme
echo "Write Documentation" >> TODO
hg add TODO
hg commit -m "write the documentation for this project."
echo "Write Unit Tests" >>TODO
hg commit -m "unit tests are done"
hg push

Wilma’s caught up now and eager to try gitifyhg. She takes over the keyboard and clones the remote repository, which already contains Fred’s changes. Then she runs gitifyhg and verifies that a git repository exists with Fred’s two commits.

cd ..
hg clone acmeadmin wilmaacme
cd wilmaacme
gitifyhg
git log

Fred’s gone for coffee. Wilma now starts her own hacking and pushes the commits using gitifyhg.

echo "Implement code until tests pass" >>TODO
git commit -m "code implemented" -a
git hgpush

Wilma got kind of lucky here because she implemented her code on the master branch and there were no conflicts with Fred’s work. If there had been, I’m not sure what would have happened. Fred’s finished his coffee and pulls in Wilma’s change via Mercurial. He then commits some more changes.

cd ../fredacme
hg pull -u
echo "deploy" >> TODO 
hg commit -m "it's up and running"
hg push

Meanwhile, Wilma is working on a separate feature. She remembers to create a new git branch this time, which is good because she’s going to want to do some rebasing when Fred’s commit comes in.

cd ../wilmaacme
git checkout -b feature_branch
echo "new feature documentation" >> FEATURE
git add FEATURE
git commit -m "start new feature by documenting it"

Before pushing her changes, Wilma pulls to see if Fred has done anything. He has! But the hgpull merges all that into her master branch. She checks out her working branch and rebases it onto master. Then she pushes it upstream.

git hgpull
git checkout feature_branch
git rebase master
git checkout master
git merge feature_branch
git hgpush

And that’s the basic workflow! I’m probably going to add an hgrebase command to take care of that checkout rebase checkout merge step, since I expect it to be common. It should probably use git rebase -i, which is probably the most amazing thing that ever happened to version control.

Like I said, this is how gitifyhg works when all goes well. When things don’t go well it’s still a bit of a mess. I’ve had to manually sync up the hg and git working directories using git reset --hard and hg update -C a couple times. Once, my hgpush ended up putting a bookmarked branch on my hg repo that I didn’t want to go public (luckily, hg push fails by default when there are multiple heads); I ended up having to use hg strip to clean it up. I’m hoping to automate or prevent some of this in the future. For now, try it out and submit patches or at least issues!

Gesture Recognition In Kivy

Kivy is a modern GUI platform that runs on Windows, Linux, Mac, iOS, and Android.

Kivy supports gestures, but the documentation is a as to how to use them. Reviewing the Gesture Board example provides most of the missing pieces if you’re willing to experiment. I have done these experiments and hope this article will make it easier for the next coder.

The kivy.gesture module contains two main classes, the Gesture and GestureDatabase. A Gesture represents the stroke or strokes in a gesture. It maps a sequence of (x,y) coordinates to a normalized representation and can be compared to another sequence of points to determine if the two sequences “match”. In Kivy, gestures can be encoded in base64. This provides an easy way to store and load gestures in source code.

The GestureDatabase is essentially a collection of Gesture objects. Its primary purpose is to compare a new gesture as input by the user to those stored in the GestureDatabase and return the closest matching gesture, if one exists.

Before we can recognize whether a user’s gesture is meaningful, we need some gestures to compare it to. Luckily, the gesture_board.py that ships in Kivy’s examples directory does this for us. Run python gesture_board.py from a terminal. A blank window opens up. Draw a gesture on it.

Have a look in the terminal. There is a variety of output there, but the important one is the long base64 encoded string following the words “gesture_representation:”. Copy that string into a variable in a basic Kivy app:

from kivy.app import App

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"


class TestApp(App):
    pass

TestApp().run()

Now let’s set up a quick and dirty GestureDatabase from the given strings. Normally, I’d put these as an instance variable on the App or a specific widget, but for easy illustration, I’ll just toss them into the module:

from kivy.app import App
from kivy.uix.widget import Widget
from kivy.graphics import Line
from kivy.gesture import Gesture, GestureDatabase

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"

gestures = GestureDatabase()
gesture = gestures.str_to_gesture(down_stroke)
gesture.name = "down_stroke"
gestures.add_gesture(gesture)
gesture = gestures.str_to_gesture(square)
gesture.name = "square"
gestures.add_gesture(gesture)

.
.
.

The next step is recording the gesture that the user makes. This requires keeping track of the touch down, move, and up events. Let’s create a new widget to handle this (Note, for clarity, I’ve omitted collision detection and error conditions):

from kivy.uix.widget import Widget
from kivy.graphics import Line


class TestWidget(Widget):

    def on_touch_down(self, touch):
        touch.ud['gesture_line'] = Line(points=(touch.x, touch.y))

    def on_touch_move(self, touch):
        touch.ud['gesture_line'].points += [touch.x, touch.y]

    def on_touch_up(self, touch):
        # compare the gestures

The final step is to implement on_touch_up to convert the stroke the user created into a gesture and compare that gesture to those in the database:

    def on_touch_up(self, touch):
        gesture = Gesture()
        gesture.add_stroke(
            zip(touch.ud['gesture_line'].points[::2],
                touch.ud['gesture_line'].points[1::2]))
        gesture.normalize()
        match = gestures.find(gesture, minscore=0.70)
        if match:
            print("{} gesture occured".format(match[1].name))
        else:
            print("No gesture recognized")

This code requires some explanation. The Line object stores it’s points in a one dimensional list, where alternate indexes represent the x and y coordinates of each point. However, add_stroke expects a list of tuples of x and y values. In code, Line stores [x1, y1, x2, y2, x3, y3] while add_stroke expects [(x1, y1), (x2, y2), (x3, y3)]. Hence, the rather complicated call to zip.

The gestures.find call accepts a minscore (1.0 would mean the gesture matched perfectly). It will return the gesture that matches with the maximum matching score, but only if that maximum is above minscore. 0.70 seems to be suitable for basic gestures, though I have been able to confuse a ‘square’ with a ‘circle’ in my experiments.

gestures.find returns either None or a tuple of (score, gesture). Thus, if there is a match we need to pull out the Gesture via match[1].

And there you have it: basic gesture recognition in Kivy.

Unfortunately, the touch events are gobbled up by the gesture code, so if you have a gesture widget that contains other widgets, they won’t receive any events. I have taken a stab at creating a GestureBox widget that passes events through to child widgets. It seems to work for touch events, but deciding whether a motion event

Python on Android? First impressions of Kivy

Kivy is a modern cross-platform Python GUI toolkit that runs on mobile devices (Android and IPhone) and supports modern input events (multitouch, accelerometor). I was really skeptical at first, but I spent the weekend developing an app in Kivy, and now I’m hooked.

In the past, I have tried most of the available Python GUI toolkits (even Swing, through jython). In recent history, I’ve mostly done HTML based interfaces, partially because that’s where the money has been, but mostly because pyQT, pyGTK,wxpython, and tk are all painful to work with.

In Kivy, interfaces are defined in a language called (aptly) the Kivy Language. The best way to describe the Kivy Language is “what HTML5 wishes it was.” It is mostly just a layout language, but it also supports binding properties and events using inline Python snippets. It is indentation defined, like Python, and utilizes a minimal amount of syntax with a maximal amount of readability. It is vaguely reminiscent of CoffeeKup but cleaner, and doesn’t suffer from having to be compiled to HTML. I love how it mimics Python syntax, but also clearly separates presentation from control code.

My favourite feature of Kivy is that properties are events. You can easily hook a property on one widget to the property on another. For example, I wrote a simple demo that connects the value of a slider to the value of a progressbar:

BoxLayout:
    orientation: 'vertical'
    ProgressBar:
        id: bar
        value: 14
        max: 20
    Slider:
        id: slider
        max: 200
        value: 140
        on_value: bar.value = self.value / 10

When the value property on the slider changes (whether because the user moved the slider, or because it was adjusted programmatically in response to some other event), the on_value event is fired. This is connected to the value of the progress bar using a Python snippet.

Only a few lines of Python code are required to to get a Kivy application running once its interface has been defined in Kivy language. The Kivy Hello World example is actually overly complicated because it doesn’t actually use the Kivy language. The bare essentials, if the above file is named sample.kv is:

from kivy.app import App
class SampleApp(App): pass
SampleApp().run()

Just three lines of code. Kivy introspects the class name (convention over configuration) and automatically loads sample.kv as the interface for a class named SampleApp.

The Kivy API is very simple. In fact, my earliest impression was that it was almost amateurish. It felt like if I wanted to do anything serious with it, I’d be disappointed. It’s the kind of API I would teach a child. However, the few features of the API (basically properties, events, and graphics instructions) are extremely well thought out and well-defined. They can be combined like Lego bricks to create widgets as complicated as one could possibly wish. The fact that the library includes a ReStructured Text rendering widget is testament to that fact!

Kivy is extremely well documented. I actually found their pong tutorial exciting. Seeing how simple it was to do each step actually made me giggle out loud. The programmer’s guide and API are well presented as well. Further, if you get stuck, the developers on IRC are terrific. I felt like I was an accepted member of the team after asking only one question.

Drawbacks

It took a bit of fiddling to get Kivy to install correctly under Arch Linux. I suppose if I’d followed the instructions, all would have been well. However, I like to do all my development in a virtualenv, and there is no trivial way to track down and install all Kivy’s dependencies. This is especially compounded by the fact that pygame currently needs to be patched under Arch in order to compile.

There are instructions for packaging your Kivy app onto Android, but I wasn’t able to pull it off, yet. I’ll have to revisit it when I have more time.

I tried some sample Kivy apps in the Android market. They work just fine once they are loaded, but it takes each app several seconds to initialize on my Galaxy Nexus. Hopefully the Kivy team can reduce this load time. I don’t know if Kivy supports displaying a splash screen while stuff is loading; that would certainly enhance the usability.

Most of these issues are solvable. I’m sure the team is actively working on some of them. Kivy is a quite young project, but it is remarkably mature.

Kivy Catalog

My weekend project was the Kivy Catalog, an interactive showcase of Kivy widgets. It allows you to view the Kivy language code used to define the widgets and to edit it and see what effects your changes have. I think it will be a tremendous aid to users looking to get started with the Kivy language. Developing it certainly was for me!

I am expecting to have this code included with the Kivy distribution in the examples directory in a future release.