Posts tagged ‘python’

Next Year: addressing the more subtle sexism at Pycon

From a gender equality point of view, I’d call Pycon 2013 a success, though perhaps a better word is “progress”. The gender ratio apparently doubled to 20% this year. If that continues (depending if it’s a constant, linear, or quadratic curve) we could be at parity in as little as two to three years.

Another indicator of forward momentum is that women this year were able to behave, to use a sexist term, “like women,” without fear of reprisal. That is to say, I saw makeup, heels, and skirts, and they did not seem out of place. This is not to suggest that women should or must dress up to attend conferences; the traditional vendorware is equally acceptable for all genders. However, it is progress that more women felt they had a choice in the matter and did not feel an obligation hide their physical differences. I think the fact that there was more variety among the male dress styles is also a sign of success.

The worst charges coming out of Pycon 2013 were the use of inappropriate jokes. To the best of my knowledge, there were no reports of women being harassed, assaulted, or groped during the conference. I’m not aware that this has been an issue at past Pycons either, but I think it’s a sign that the Python community is better behaved in this regard than some of her sister tech conferences.

So all in all, while it wasn’t completely successful, Pycon 2013 was a terrific step on the road to eliminating sexism and creating gender equality. Let’s make next year an equal sized step. Here are a few topics I think we, as a community, can work on to engender further progress.

I’m going to start with a couple links. Ruth Burr wrote a terrific essay titled Things You Think Aren’t Sexist, But Really Are a couple weeks ago. In summary: Men, you are instructed to go to conferences to network and interact, not to date or hook up.

Yes, you might meet the love of your life at Pycon, but don’t expect, intend, or plan for that to happen (at PyCon or anywhere). Behave professionally, and consider each woman you meet for her knowledge and intelligence, not her figure or her marital status, just as you would when interacting with a male.

Ned Batchelder, with his amazing talent for understanding and explaining problems wrote another great article on the root issues. Summarized, he says friction is inevitable, and that education is better than shunning when people make mistakes.

I started planning this article while attending Pycon this year. I had to scrap those plans after the so-memed “donglegate” fiasco. I don’t have anything to add to that discussion, but as Ned illustrated, there were many many instances of similarly inappropriate comments at Pycon. I myself made such a joke, most of my friends did so, I overheard a conference organizer say something inappropriate. This is not sexism per se, but sexual and other potentially offensive comments need to be reduced. Comments laced with sexual innuendo do not belong in a professional or a family setting, and Pycon is intended to be both.

Let’s start with interation between the genders. In her article, Ruth Burr mentioned that women can feel awkward when they interact with men because the men assume they are flirting rather than interested in the topic at hand. Many men see cleavage and assume the speaker can’t possibly know as much as them about whatever topic that is. Some girls avoid interacting with men because of this awkward sensation, and if they do, the interactions are tainted.

I had a related problem. While I comfortably talked to men of varying skill levels at Pycon, I felt uncomfortable addressing women because I was afraid of sending some vibe that I was being flirtatious. So I tended to ignore the female attendees. This is unfortunate for everyone who missed out on that potential conversation. Pycon 2014 needs to increase the interaction between the genders, not just the attendance ratios. Ladies, talk to the shy guys, they need to be taught that you aren’t that intimidating. Guys, talk to the women, they need to learn that we are interested in what they have to say about tech, and not their figures. As it stands, only the sexist guys are interacting with the women, and it makes us all look bad.

The gender ratio really fell during the developer sprints. I don’t think there were any female sprint leaders presenting projects to hack on. Worse, I’d estimate between one and two percent of the sprint attendees were women. This is a huge area for improvement. To the various diversity groups out there, please encourage your members to stay an extra day or two and attend the Pycon dev sprints. Get them involved with Open Hatch if they are unsure how to contribute to open source projects. Sprint leaders, make sure female attendees are welcome, especially if they are new coders (indeed, make all new coders welcome).

I noticed a lot of Impostor syndrome among the female attendees and speakers. Programming really is easy. The fact that you enjoy it and find it quite simple is not a sign that you don’t know the “hard stuff” and therefore you’re not a “real programmer.” Quite the reverse, in fact. It’s not necessary to apologize for your lack of knowledge if you’ve been invited to do a talk or are about to join an open source project for the first time. Find ways to build your confidence (contributing to open source and getting feedback is a great start), and start believing in your skills. Even if you don’t believe it yourself, try to project confidence in your coding abilities; it will send a much more effective signal that women are capable and here to stay. Don’t worry if you over-present yourself; faking your way through it is a great way to find out that you’re actually better than you thought!

I’d like to close with an admonishment to those people who open their articles on sexism with a discussion of their gender. These usually take one of two forms: the disclaimer and the shocker. The disclaimer is most often used by men and sounds like, “I am a privileged white male so I don’t understand what it’s like for women, but I still think I have something valid to say on this topic”. The shocker is used by both genders and sounds like, “I am a man/woman, so you’re going to be amazed that my opinion on this topic is different from other members of my gender.” Yes, it is useful to state your bias and frame of reference. However, be cautious that your purpose in doing so is to remove distortion from the lens you are applying to the discussion, not to add to it.

And now, feel free to analyze my bias in this article. I am a privileged white male from a blue-collar family. I’ve had many amazing opportunities in the tech industry. I have very little experience as the target of discrimination, outside my mental illness. My interest in feminist issues comes from my sister’s master’s degree on the subject; I proofread most of her undergraduate and graduate level essays, and gained a relatively deep understanding of the topic. I acknowledge that I am a racist, sexist jerk by culture and conditioning. Sometimes I forget to compensate for that. I find it exceptionally easy to overlook the patriarchy when it’s doing me favors, and I will never be as keenly aware of its negative impact as someone who is directly experiencing it every day.

Guido Van Rossum Should Retire (and focus on python)

At the two previous Pycons I’ve attended (2009 and 2012), Guido Van Rossum’s keynotes sounded bored and uninterested, even though the content was meaningful. I was actually wondering if this would be the year that he would step down from BDFL of Python. Thankfully, I was dead wrong.

Instead, he presented a highly technical and very exciting addition to the Python language. Alfredo told me this started when he took a month off between stepping down at Google and starting at DropBox. Now, when normal people take a month off, they relax or travel or visit friends and family. Not our BDFL. He writes a callback-free asynchronous event loop API and reference implementation that is expected to massively alleviate Python’s oft-maligned lack of a consistent, unhackish concurrency solution.

Let’s have more of that. What if Mr. Van Rossum could hack on Python full time? Would we see quantum progress in Python every month?

Anyone who knows about the Gittip project likely thinks they can guess where this is going. We, the people, can each tip our BDFL a few cents or dollars per week so he can focus on whatever he deems worthy. It’s safe to assume that a man who spends his vacation time drafting a new Python library would choose to work on Python full time if we funded him.

This plan is great, and I actually think that Guido could easily earn enough to quit his day job if he endorsed Gittip and invited individuals to tip him. But I’d like to discuss a different plan: Not individuals, but companies should tip Guido the maximum gittip amount as a sort of “partial salary”. At $1248 per year, most companies wouldn’t even notice this expense, and they would get a better programming language and standard library in return. The rate of accelerated development would be even higher if each of these companies chose to invest an entire salary, split between a hundred Python core and library developers. If a hundred companies chose to do this, those hundred people could work on Python full time. The language and library would improve so vastly and so rapidly that the return on investment for each of those companies would be far greater than if they had paid that same salary to a single developer working on their in-house product, full time.

It might take some convincing to justify such a strategy to these corporations. Companies tend to like to know what is happening to their money, and simply throwing a hefty developer salary at Gittip would be hard to justify. Obviously “goodwill” could support some of it, in the same way that so many companies sponsored Pycon in exchange for exposure.

Cutthroat CEOs should perhaps consider not just the value that having Guido working exclusively on Python is, but also the cost of having him work for the competition. I’m sure Box.com CEO Aaron Levie was a little nervous when he found out that the first and greatest Python programmer of all time had recently hired on at a major competitor. Perhaps Box.com can’t afford to steal Guido from Dropbox, but if all the companies currently involved in cloud storage were to tip Guido $24 per week on Gittip, this incredible programmer could be working on an open source product that directly and indirectly benefits their company rather than improving the competing product on a full-time basis.

Most of the arguments that Gittip will fail are based on the premise that not enough money can be injected into the platform to sustain full time development by open source programmers. However, if an open and caring relationship can be built such that the corporate world is also funding the system, I think it can become extremely successful. Everyone will benefit: Open source projects will improve at a rapid pace. Exceptional developers will get to pursue their passions. End users will get better products. The overall level of happiness in the world will be higher.

I would like to see a world where brilliant young software engineers are not risking their mental health (and consequently, their lives) on startup ideas in the hopes of being bought out for a few billion dollars. I would like to see a world where those engineers are not working for large corporations that have neither their employees nor their end users (but rather, their stockholders and advertisers) interests at heart. I would like to see a world where those developers can choose to invest their passion in open source products that will change the world.

Excluding tests with py.test 2.3.4 using -k selection

When I use py.test, I often rely on the -k switch to swiftly select the test I want to run. Instead of having to type the full module, class, and test path as is required with unittest and nose, I can just type a few characters that uniquely match the name of the test.

For example, if I have a test file containing methods test_basic_clone and test_basic_clone_notes, I can run the latter test simply by calling py.test -k clone_no.

However, I often create multiple tests that have similar names. This can make it difficult to run just one test if the test name is a prefix of a longer test name. If I want to run just test_basic_clone, any substring will also be a substring of the test_basic_clone_notes test, and both tests are matched by -k.

Since pytest version 2.3.4, the -k keyword supports expressions. So I can build an expression like this:

py.test -k "basic_clone and not notes"

This selects all tests matching “basic_clone”, then excludes any containing the word “notes”. Thus, I run only the test I’m interested in without having to fix my crappy naming scheme. It’s more typing than is normally the case, but is still less cognitive load than trying to remember what module and class I’m editing and constructing a selector based on those attributes.

Gitifyhg is now awesome.

Gitifyhg is a git client for pushing to and pulling from Mercurial repositories. I described the first implementation, which used hg-git internally, last month. However, I found that it didn’t work as well in practice as my initial tests had indicated, and I morosely reverted to using hg directly.

While researching ways to improve the app, I stumbled across git-remote-hg by Felipe Contreras. It claimed to be a cure-all to the git-hg bridging problem that “just worked”. So I downloaded and tried it, but it didn’t live up to the author’s hype. While it could handle basic cloning, pushing to and pulling from the default branch, it failed for me when working with named upstream branches, something I have to do regularly in my day job. I submitted several issues and pull requests, most of which were ignored. The more deeply involved I became with the code, the more I felt a complete rewrite was in order.

I had some free time during the holiday season, and started my version of a git remote as an exercise, with no intent to create anything useful. To my surprise, something useful emerged. In fact, I believe gitifyhg is the most robust and functional git to hg client currently available. More importantly, it is eminently hackable code: well tested and fairly well documented. I hope this will make it easy to contribute to, and that my inbox will soon be full of pull requests.

This is the real deal. The best part is that you don’t have to learn any new commands. It’s just basic git with mercurial as a remote. The only command that has changed from normal git usage is clone:

pip install gitifyhg
git clone gitifyhg::http://selenic.com/hg
cd hg

By adding gitifyhg:: before the mercurial url, you can git clone most mercurial repositories. If you can’t, it’s a bug. Other complex repositories I have successfully cloned include py.test and pypy.

You can easily use gitifyhg in the fashion of git-svn. All named branches are available as remote branches. default maps to master. Other branches map to branches/<branchname>.

If you want to commit to the master branch, I suggest a workflow like this:

git clone gitifyhg::<any mercurial url>
cd repo_name
git checkout -b working  # make a local branch so master stays prestine
# hack and commit, hack and commit
git checkout master
git pull  # Any new commits that other people have added to upstream mercurial are now on master
git rebase master working  # rebase the working branch onto the end of master
git checkout master
git push

Working on a named mercurial branch, for example feature1, is easy:

git checkout --track origin/branches/feature1
git checkout -b working  # make a local branch so feature1 stays prestine for easy pulling and rebasing
# hack and commit, hack and commit
git checkout branches/feature1
git pull  # New commits from upstream on the feature1 branch
git rebase branches/feature1 working  # rebase the working branch onto the end of feature1
git checkout master
git push  #push your changes back to the upstream feature1 branch

It is even possible to create new named branches (assuming my_new_branch doesn’t exist yet in Mercurial):

git checkout -b "branches/my_new_branch"
# hack add commit
git push --set_upstream origin branches/my_new_branch

These basic workflows have been working flawlessly for me all week. In contrast to my previous attempts to use git to hg bridges, I have found it easier to use gitifyhg than to work in the mercurial commands that I have become expert with, but not used to, in the past year.

Gitify hg is not yet perfect. There are a few issues that still need to be ironed out. There are failing tests for most of these in the gitifyhg test suite if you would like to contribute with some low-hanging fruit:

  • Anonymous branches are dropped when cloned. Only the tip of a named branch is kept.
  • Tags can be cloned and pulled, but not pushed.
  • Bookmarks can be cloned and pushed, but not pulled reliably. I suspect this is related to the anonymous branch issue.

So give it a shot. Gitifyhg is just one easy_install gitifyhg away.

Opterator, revisited

A few years ago, I wrote a simple decorator that introspects a main() method signature and docstring to generate an option parser. I never really used it, mostly because back then, I wasn’t too keen on third party dependencies. Nowadays with distribute being maintained, well-documented, and working (plus I know how to use it properly), I no longer have this concern.

A friend recently linked me to the brilliantly designed docopt and reminded me of my interest in opterator. I revisited my code and decided it’s a pretty neat little design. So I ported it to Python 3. This took a matter of minutes, largely because I originally wrote it shortly after taking the pytest tutorial at Pycon 2009 and it was easy to find the failing code. It now supports Python 2.6, 2.7, and 3.3, according to tox.

I originally designed opterator to be a full replacement for optparse and friends. However, my main purpose for it now is to create quick and dirty command line applications, often for my personal use. These apps usually have a couple of options and it doesn’t seem worth the trouble of setting up optparse. Yet, mucking around with sys.argv is also annoying. Opterator minimizes the boilerplate. Check out this basic example:

from opterator import opterate                      


@opterate                                           
def main(filename, color='red', verbose=False):     
    print(filename, color, verbose)

main()                                              

with three lines of boilerplate code, a function can be turned into a command line program that can be called like so:

  $ python examples/basic.py this_file
  this_file red False

or so:

  python examples/basic.py this_file --color=blue
  this_file blue False

or even so:

  $ python examples/basic.py --color=purple another_file --verbose
  another_file purple True

And you even get a not totally (but somewhat) useless helpfile:

  $ python examples/basic.py -h
  Usage: basic.py [options] filename



Options:
  -h, --help            show this help message and exit
  -c COLOR, --color=COLOR
  -v, --verbose

Building a Python Kivy App in Android (Easier than it looks, but harder than it needs to be)

When I introduced Kivy a while back, I mentioned that I had not been able to deploy a Kivy app on Android. I’ve managed to do this now, using the VirtualBox image provided by the Python For Android project. Using this image was substantially less work than trying to figure out how to install and set up all the prerequisites under Arch Linux. So here’s how to do it:

First download the VirtualBox image from Google Drive. Sadly I cannot find an adequate wget command to download this. Shame on you, Google!

You may want to double check the documentation to see if a later image has been made available.

It’s a big honkin’ file (1.4GB), so grab a cup of coffee, lunch, or go to bed, depending on your bandwidth and the current time of day.

Extract the file using p7zip. This takes a while, so you might want to grab another coffee:

7z e Python\ for\ Android\ -\ Ubuntu\ 12.04.7z

Make sure VirtualBox is installed and the kernel module is loaded. The commands on Arch Linux are:

pacman -S virtualbox virtualbox-host-modules
sudo modprobe vboxdrv

Fire up VirtualBox and hit Machine->Add.

Navigate to the folder the vbox image was extractedto. Select the Python for Android - Ubuntu 12.04.vbox machine and hit open.

If you have enough RAM, select the Python for Android machine and hit settings. Browse to the System tab and increase the RAM to a comfortable level (I set it to 2GB on my 4GB machine). You might also want to increase the number of processors and the video RAM. Hit Start to power on the machine.

Once Ubuntu has loaded, review the README for some important information. You may want to update the VirtualBox Guest Additions install, although I’m sure it will work fine without it.

Now you’re ready to build an APK from a Python project. Open a terminal from the menu on the sidebar. The first thing you’ll want to do is update the python-for-android project to the latest git checkout. This is a bit messy because there are a couple edits in the current checkout that were later fixed upstream. So you’ll want to stash the changes before pulling:

cd android/python-for-android
git stash
git pull
git stash apply

Unfortunately, this introduces a merge conflict in distribute.sh. The upstream project has made some of the customizations obsolete by including more generic code. Luckily, it’s a trivial fix. Open distribute.sh in an editor (gedit distribute.sh works, but I used vim.) and search for the characters =======. It’s at line 170. Delete the line above that says <<<<<<< Updated upstream. Leave the export line that was above the =======. Then delete the ======= and >>>>>>> Stashed changes and the outdated export line in between. Save the file and close the editor.

Now you need a Kivy application to install. Presumably you’ve written your own, but for this example, I’m going to use a cool one supplied by the Kivy team. The Kivy Remote Shell creates an ssh server on the phone that allows you to log in from your computer over ssh and execute Python commands on the phone. You can even run Java commands using the beta pyjnius connector.

These commands will prepare a python for android distribution ready for installation:

cd ..
git clone git://github.com/kivy/kivy-remote-shell
cd python-for-android
./distribute.sh -m 'openssl pycrypto pyasn1 pyjnius twisted kivy'

This will prompt to see if you want to overwrite the previous python for android distribution. Press <enter> to do so. Give it some time to connect to the network, download the dependencies from Pypi, and compile them.

Now you need a rather ugly command to actually build the apk. The options are summarized in the Python for Android documentation.

cd dist/default
./build.py --package org.kivy.sshshell --name "Kivy Remote Shell" \
  --version 1 --dir ../../../kivy-remote-shell/ \
  --icon ../../../kivy-remote-shell/icon.png --permission INTERNET debug

This will create the file bin/KivyRemoteShell-1-.debug.apk. Now let’s install it to the phone! First make sure USB debugging is enabled on your phone. Go to the Settings-->Developer Options menu and make sure Developer Options are enabled and that Android Debugging is enabled.

Plug the phone into your computer via the USB cord. Click the little “USB” icon in the VirtualBox status bar and check the box next to the name for your phone.

Run the command ~/android/android-sdk-linux_x86/platform-tools/adb devices to make sure it outputs the identifier of your phone. If it works, simply run ~/android/android-sdk-linux_x86/platform-tools/adb install bin/KivyRemoteShell-1-debug.apk to install the app.

(If you can’t get the phone to connect to the VM, you can also copy the .apk to the host machine to install it. Use VirtualBox’s shared folders feature).

You can now shut down the virtual machine from Ubuntu’s shutdown menu. For security, it’s a good idea to turn off USB debugging on your phone until you need it again. Run the “Kivy Remote Shell” app that is now in your phone’s app drawer. Type the command that comes up onto your screen into a terminal on your computer, and you will have access to a Python prompt running on your phone!

This same process can be used to install self-developed Kivy apps to your phone. Congratulations!

Gitifyhg: Accessing Mercurial repos from GIT

My company uses Mercurial for internal hosting. Other than that, it’s an absolutely terrific place to work.

About three quarters of my colleagues are sick of hearing the rest of us complain about Mercurial’s inadequacies. They mention tutorials like this one that simply don’t work in real life. I’ve done my research, and I have not been able to find a viable git to hg pipeline anywhere. There is a git-hg repo, but it appears to be un-maintained and not overly well documented. It’s also written in bash, so I’m not eager to take over maintenance.

Instead, I’ve been spending some of my non-working hours trying to make a python wrapper around hg-git named gitifyhg. It started out as an automated version of the hg-git tutorials, but is now expanding to provide additional support and tools.

Right now I’m following a git-svn style of workflow, where your git master branch can be synced up with the hg default branch. It seems to be working somewhat ok. I think at this point, the pain of using gitifyhg is about equal to the pain of using hg alone. Hopefully future improvements will reduce that pain, and as always, patches are most welcome!

The instructions are pretty clear and the unit tests provide a pretty good description of basic usage. Non-basic usage probably doesn’t work (yet). Here’s a tutorial of what does work:

Acme corporation uses Mercurial for all their repositories. Fred and Wilma are developing a project named AcmeAdmin together. Fred is content to use Mercurial for hosting, but Wilma is a git advocate who is going crazy with the restrictions Mercurial places on her. She decides she’s going to try gitifyhg on this new project to see how much trouble it is.

Fred starts the project by creating a Mercurial repo that will serve as the remote repo for both of them:

mkdir acmeadmin
cd acmeadmin
hg init

Fred and Wilma are doing a very strange sort of pair programming where they have separate repositories on the same machine. They are doing this so if you were reading along, you could type in the same commands they are typing and the demo would work. Rest assured that in spite of this queer practice, Fred and Wilma are hotshot programmers. So Fred now clones this remote repo and starts working. In the meantime, Wilma is browsing the gitifyhg README, so she hasn’t cloned anything yet.

cd ..
hg clone acmeadmin fredacme
cd fredacme
echo "Write Documentation" >> TODO
hg add TODO
hg commit -m "write the documentation for this project."
echo "Write Unit Tests" >>TODO
hg commit -m "unit tests are done"
hg push

Wilma’s caught up now and eager to try gitifyhg. She takes over the keyboard and clones the remote repository, which already contains Fred’s changes. Then she runs gitifyhg and verifies that a git repository exists with Fred’s two commits.

cd ..
hg clone acmeadmin wilmaacme
cd wilmaacme
gitifyhg
git log

Fred’s gone for coffee. Wilma now starts her own hacking and pushes the commits using gitifyhg.

echo "Implement code until tests pass" >>TODO
git commit -m "code implemented" -a
git hgpush

Wilma got kind of lucky here because she implemented her code on the master branch and there were no conflicts with Fred’s work. If there had been, I’m not sure what would have happened. Fred’s finished his coffee and pulls in Wilma’s change via Mercurial. He then commits some more changes.

cd ../fredacme
hg pull -u
echo "deploy" >> TODO 
hg commit -m "it's up and running"
hg push

Meanwhile, Wilma is working on a separate feature. She remembers to create a new git branch this time, which is good because she’s going to want to do some rebasing when Fred’s commit comes in.

cd ../wilmaacme
git checkout -b feature_branch
echo "new feature documentation" >> FEATURE
git add FEATURE
git commit -m "start new feature by documenting it"

Before pushing her changes, Wilma pulls to see if Fred has done anything. He has! But the hgpull merges all that into her master branch. She checks out her working branch and rebases it onto master. Then she pushes it upstream.

git hgpull
git checkout feature_branch
git rebase master
git checkout master
git merge feature_branch
git hgpush

And that’s the basic workflow! I’m probably going to add an hgrebase command to take care of that checkout rebase checkout merge step, since I expect it to be common. It should probably use git rebase -i, which is probably the most amazing thing that ever happened to version control.

Like I said, this is how gitifyhg works when all goes well. When things don’t go well it’s still a bit of a mess. I’ve had to manually sync up the hg and git working directories using git reset --hard and hg update -C a couple times. Once, my hgpush ended up putting a bookmarked branch on my hg repo that I didn’t want to go public (luckily, hg push fails by default when there are multiple heads); I ended up having to use hg strip to clean it up. I’m hoping to automate or prevent some of this in the future. For now, try it out and submit patches or at least issues!

Gesture Recognition In Kivy

Kivy is a modern GUI platform that runs on Windows, Linux, Mac, iOS, and Android.

Kivy supports gestures, but the documentation is a as to how to use them. Reviewing the Gesture Board example provides most of the missing pieces if you’re willing to experiment. I have done these experiments and hope this article will make it easier for the next coder.

The kivy.gesture module contains two main classes, the Gesture and GestureDatabase. A Gesture represents the stroke or strokes in a gesture. It maps a sequence of (x,y) coordinates to a normalized representation and can be compared to another sequence of points to determine if the two sequences “match”. In Kivy, gestures can be encoded in base64. This provides an easy way to store and load gestures in source code.

The GestureDatabase is essentially a collection of Gesture objects. Its primary purpose is to compare a new gesture as input by the user to those stored in the GestureDatabase and return the closest matching gesture, if one exists.

Before we can recognize whether a user’s gesture is meaningful, we need some gestures to compare it to. Luckily, the gesture_board.py that ships in Kivy’s examples directory does this for us. Run python gesture_board.py from a terminal. A blank window opens up. Draw a gesture on it.

Have a look in the terminal. There is a variety of output there, but the important one is the long base64 encoded string following the words “gesture_representation:”. Copy that string into a variable in a basic Kivy app:

from kivy.app import App

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"


class TestApp(App):
    pass

TestApp().run()

Now let’s set up a quick and dirty GestureDatabase from the given strings. Normally, I’d put these as an instance variable on the App or a specific widget, but for easy illustration, I’ll just toss them into the module:

from kivy.app import App
from kivy.uix.widget import Widget
from kivy.graphics import Line
from kivy.gesture import Gesture, GestureDatabase

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"

gestures = GestureDatabase()
gesture = gestures.str_to_gesture(down_stroke)
gesture.name = "down_stroke"
gestures.add_gesture(gesture)
gesture = gestures.str_to_gesture(square)
gesture.name = "square"
gestures.add_gesture(gesture)

.
.
.

The next step is recording the gesture that the user makes. This requires keeping track of the touch down, move, and up events. Let’s create a new widget to handle this (Note, for clarity, I’ve omitted collision detection and error conditions):

from kivy.uix.widget import Widget
from kivy.graphics import Line


class TestWidget(Widget):

    def on_touch_down(self, touch):
        touch.ud['gesture_line'] = Line(points=(touch.x, touch.y))

    def on_touch_move(self, touch):
        touch.ud['gesture_line'].points += [touch.x, touch.y]

    def on_touch_up(self, touch):
        # compare the gestures

The final step is to implement on_touch_up to convert the stroke the user created into a gesture and compare that gesture to those in the database:

    def on_touch_up(self, touch):
        gesture = Gesture()
        gesture.add_stroke(
            zip(touch.ud['gesture_line'].points[::2],
                touch.ud['gesture_line'].points[1::2]))
        gesture.normalize()
        match = gestures.find(gesture, minscore=0.70)
        if match:
            print("{} gesture occured".format(match[1].name))
        else:
            print("No gesture recognized")

This code requires some explanation. The Line object stores it’s points in a one dimensional list, where alternate indexes represent the x and y coordinates of each point. However, add_stroke expects a list of tuples of x and y values. In code, Line stores [x1, y1, x2, y2, x3, y3] while add_stroke expects [(x1, y1), (x2, y2), (x3, y3)]. Hence, the rather complicated call to zip.

The gestures.find call accepts a minscore (1.0 would mean the gesture matched perfectly). It will return the gesture that matches with the maximum matching score, but only if that maximum is above minscore. 0.70 seems to be suitable for basic gestures, though I have been able to confuse a ‘square’ with a ‘circle’ in my experiments.

gestures.find returns either None or a tuple of (score, gesture). Thus, if there is a match we need to pull out the Gesture via match[1].

And there you have it: basic gesture recognition in Kivy.

Unfortunately, the touch events are gobbled up by the gesture code, so if you have a gesture widget that contains other widgets, they won’t receive any events. I have taken a stab at creating a GestureBox widget that passes events through to child widgets. It seems to work for touch events, but deciding whether a motion event

Python on Android? First impressions of Kivy

Kivy is a modern cross-platform Python GUI toolkit that runs on mobile devices (Android and IPhone) and supports modern input events (multitouch, accelerometor). I was really skeptical at first, but I spent the weekend developing an app in Kivy, and now I’m hooked.

In the past, I have tried most of the available Python GUI toolkits (even Swing, through jython). In recent history, I’ve mostly done HTML based interfaces, partially because that’s where the money has been, but mostly because pyQT, pyGTK,wxpython, and tk are all painful to work with.

In Kivy, interfaces are defined in a language called (aptly) the Kivy Language. The best way to describe the Kivy Language is “what HTML5 wishes it was.” It is mostly just a layout language, but it also supports binding properties and events using inline Python snippets. It is indentation defined, like Python, and utilizes a minimal amount of syntax with a maximal amount of readability. It is vaguely reminiscent of CoffeeKup but cleaner, and doesn’t suffer from having to be compiled to HTML. I love how it mimics Python syntax, but also clearly separates presentation from control code.

My favourite feature of Kivy is that properties are events. You can easily hook a property on one widget to the property on another. For example, I wrote a simple demo that connects the value of a slider to the value of a progressbar:

BoxLayout:
    orientation: 'vertical'
    ProgressBar:
        id: bar
        value: 14
        max: 20
    Slider:
        id: slider
        max: 200
        value: 140
        on_value: bar.value = self.value / 10

When the value property on the slider changes (whether because the user moved the slider, or because it was adjusted programmatically in response to some other event), the on_value event is fired. This is connected to the value of the progress bar using a Python snippet.

Only a few lines of Python code are required to to get a Kivy application running once its interface has been defined in Kivy language. The Kivy Hello World example is actually overly complicated because it doesn’t actually use the Kivy language. The bare essentials, if the above file is named sample.kv is:

from kivy.app import App
class SampleApp(App): pass
SampleApp().run()

Just three lines of code. Kivy introspects the class name (convention over configuration) and automatically loads sample.kv as the interface for a class named SampleApp.

The Kivy API is very simple. In fact, my earliest impression was that it was almost amateurish. It felt like if I wanted to do anything serious with it, I’d be disappointed. It’s the kind of API I would teach a child. However, the few features of the API (basically properties, events, and graphics instructions) are extremely well thought out and well-defined. They can be combined like Lego bricks to create widgets as complicated as one could possibly wish. The fact that the library includes a ReStructured Text rendering widget is testament to that fact!

Kivy is extremely well documented. I actually found their pong tutorial exciting. Seeing how simple it was to do each step actually made me giggle out loud. The programmer’s guide and API are well presented as well. Further, if you get stuck, the developers on IRC are terrific. I felt like I was an accepted member of the team after asking only one question.

Drawbacks

It took a bit of fiddling to get Kivy to install correctly under Arch Linux. I suppose if I’d followed the instructions, all would have been well. However, I like to do all my development in a virtualenv, and there is no trivial way to track down and install all Kivy’s dependencies. This is especially compounded by the fact that pygame currently needs to be patched under Arch in order to compile.

There are instructions for packaging your Kivy app onto Android, but I wasn’t able to pull it off, yet. I’ll have to revisit it when I have more time.

I tried some sample Kivy apps in the Android market. They work just fine once they are loaded, but it takes each app several seconds to initialize on my Galaxy Nexus. Hopefully the Kivy team can reduce this load time. I don’t know if Kivy supports displaying a splash screen while stuff is loading; that would certainly enhance the usability.

Most of these issues are solvable. I’m sure the team is actively working on some of them. Kivy is a quite young project, but it is remarkably mature.

Kivy Catalog

My weekend project was the Kivy Catalog, an interactive showcase of Kivy widgets. It allows you to view the Kivy language code used to define the widgets and to edit it and see what effects your changes have. I think it will be a tremendous aid to users looking to get started with the Kivy language. Developing it certainly was for me!

I am expecting to have this code included with the Kivy distribution in the examples directory in a future release.

Pushing Python Past the Present

This is the first time I’ve inserted myself into an exchange between bloggers I don’t know. This topic interests me and I have something to add. Most importantly, I found an alliterative title. So I thought I’d give it a go.

I first saw Calvin Spealman’s article I am worried about the future of Python. I suspect that upon reading this article, Guido got into his famous time machine to give his Pycon 2012 Keynote speech, accusing Calvin of trolling.

This article was shortly followed by Tim McNamara’s Python is doing just fine which (likely unintentionally) summarizes Guido’s talk.

Finally, Nick Coghlan came out with an extensive summary article called Python’s Future: A Global Perspective.

And now there’s me. All I want to do is push people to start supporting PyPy. The PyPy developers have worked on a variety of technologies that can be made into production-ready products that address most issues (real or imaginary) that people see with Python.

They’ve already solved speed and I suspect the next release of PyPy will solve memory. They’ve made huge but mostly incomplete progress in a lot of other areas, including the ones the above bloggers have mentioned.

Concurrency

I have had a lot of trouble dealing with parallelism in Python. Everything I have tried has either been a hack or required a hack to work with it. In my experience, the multiprocessing module does not work in large-scale production. However, it works a lot better on PyPy than it does on cPython, at least for me. The various async solutions can only use one processor and are therefore, in my opinion, no more exciting than GIL-throttled Threads.

In addition to better multiprocessing, PyPy also supports stackless extensions out of the box. And I haven’t even mentioned Armin Rigo’s mysterious Software Transactional Memory implementation.

Concurrency is a sore point in Python. There are solutions. PyPy is capable of doing those solutions better.

Web

I think it’s unfortunate that the only way to code for a web browser is to use JavaScript or a language that compiles to JavaScript. I feel there is no real reason browsers can’t support arbitrary <script type=""> tags. I discussed this last March.

With development, PyPy can support Python in the browser. The mostly-finished but untested sandboxing feature of PyPy can be adapted for in-browser execution. I know some experimenting was done on this in 2009 that proved it’s possible. We just need to take it up and make it happen.

Mobile

PyPy’s speed makes it theoretically possible to run it effectively on mobile platforms. Further, it can be compiled to arbitrary execution environments including JVM and .Net. There is no reason that with some development, PyPy couldn’t run on the Dalvik JVM. I’m sure it’s even possible to run it on iOS.

Interestingly, one of the trolls Guido mentioned in his talk was “PyPy should become the default Python implementation”. He debunked this readily by asking the audience how many people use PyPy in production. Nobody moved.

PyPy is ready for production use, but it is not widely used. I think this is largely because the primary PyPy developers are way more excited about creating new features than they are about marketing the current platform or polishing up nearly finished features like sandboxing or a Dalvik backend. They are visionaries, not finishers.

I say this with a great deal of respect. I am not calling for these guys to change their focus. What I want is for other people from the Python community to join the PyPy project as finishers. It needs people that can make sure these nearly-finished features are working, production-ready and more importantly: documented.

I’ve been meaning, for months, to become one of these people. Unfortunately, I’ve prioritized other things including my job and my upcoming book. It’s still on my radar, though, and I hope that after reading this article, you, too, are thinking about make PyPy the future of Python.

So, what can you do?

Use PyPy
PyPy is capable of being your default Python 2 interpreter for most common tasks. Use it. If it doesn’t work for a given project, get on #pypy, they will help you fix it. It’s even more exciting if you can use PyPy features that are not currently available in cPython, such as the stackless extensions.
Evangelize PyPy
Tell people how well PyPy is working for you. Write articles like this one.
Document PyPy
The PyPy website contains a lot of documentation, but it’s rather intimidating and unreadable to the uninitiated. PyPy is just Python, but it’s also much more than Python. Somebody with some writing skills needs to get in there and make it digestable.
Design PyPy
Seriously, pypy.org does not need to look like something out of 2005.
Develop PyPy
I’ve hacked on PyPy. It’s not scary. Work on features like numpy or Python 3 support that are the current developer’s focus. Better yet, work on finalizing features like sandboxing or alternative backends that are finished but not quite tested.

If you have an hour or two free this weekend, instead of writing about how Python is great or not great, or not going to continue to be great, do something with PyPy. Make it happen.