Posts tagged ‘kivy’

Gesture Recognition In Kivy

Kivy is a modern GUI platform that runs on Windows, Linux, Mac, iOS, and Android.

Kivy supports gestures, but the documentation is a as to how to use them. Reviewing the Gesture Board example provides most of the missing pieces if you’re willing to experiment. I have done these experiments and hope this article will make it easier for the next coder.

The kivy.gesture module contains two main classes, the Gesture and GestureDatabase. A Gesture represents the stroke or strokes in a gesture. It maps a sequence of (x,y) coordinates to a normalized representation and can be compared to another sequence of points to determine if the two sequences “match”. In Kivy, gestures can be encoded in base64. This provides an easy way to store and load gestures in source code.

The GestureDatabase is essentially a collection of Gesture objects. Its primary purpose is to compare a new gesture as input by the user to those stored in the GestureDatabase and return the closest matching gesture, if one exists.

Before we can recognize whether a user’s gesture is meaningful, we need some gestures to compare it to. Luckily, the gesture_board.py that ships in Kivy’s examples directory does this for us. Run python gesture_board.py from a terminal. A blank window opens up. Draw a gesture on it.

Have a look in the terminal. There is a variety of output there, but the important one is the long base64 encoded string following the words “gesture_representation:”. Copy that string into a variable in a basic Kivy app:

from kivy.app import App

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"


class TestApp(App):
    pass

TestApp().run()

Now let’s set up a quick and dirty GestureDatabase from the given strings. Normally, I’d put these as an instance variable on the App or a specific widget, but for easy illustration, I’ll just toss them into the module:

from kivy.app import App
from kivy.uix.widget import Widget
from kivy.graphics import Line
from kivy.gesture import Gesture, GestureDatabase

down_stroke = "eNq1l91u4zYQhe/1IslNjPkfzgtkb<snip>"
square = "eNq1mEluIzcYRvd1EXsT4Z+HC6i3AXyAwG<snip>"

gestures = GestureDatabase()
gesture = gestures.str_to_gesture(down_stroke)
gesture.name = "down_stroke"
gestures.add_gesture(gesture)
gesture = gestures.str_to_gesture(square)
gesture.name = "square"
gestures.add_gesture(gesture)

.
.
.

The next step is recording the gesture that the user makes. This requires keeping track of the touch down, move, and up events. Let’s create a new widget to handle this (Note, for clarity, I’ve omitted collision detection and error conditions):

from kivy.uix.widget import Widget
from kivy.graphics import Line


class TestWidget(Widget):

    def on_touch_down(self, touch):
        touch.ud['gesture_line'] = Line(points=(touch.x, touch.y))

    def on_touch_move(self, touch):
        touch.ud['gesture_line'].points += [touch.x, touch.y]

    def on_touch_up(self, touch):
        # compare the gestures

The final step is to implement on_touch_up to convert the stroke the user created into a gesture and compare that gesture to those in the database:

    def on_touch_up(self, touch):
        gesture = Gesture()
        gesture.add_stroke(
            zip(touch.ud['gesture_line'].points[::2],
                touch.ud['gesture_line'].points[1::2]))
        gesture.normalize()
        match = gestures.find(gesture, minscore=0.70)
        if match:
            print("{} gesture occured".format(match[1].name))
        else:
            print("No gesture recognized")

This code requires some explanation. The Line object stores it’s points in a one dimensional list, where alternate indexes represent the x and y coordinates of each point. However, add_stroke expects a list of tuples of x and y values. In code, Line stores [x1, y1, x2, y2, x3, y3] while add_stroke expects [(x1, y1), (x2, y2), (x3, y3)]. Hence, the rather complicated call to zip.

The gestures.find call accepts a minscore (1.0 would mean the gesture matched perfectly). It will return the gesture that matches with the maximum matching score, but only if that maximum is above minscore. 0.70 seems to be suitable for basic gestures, though I have been able to confuse a ‘square’ with a ‘circle’ in my experiments.

gestures.find returns either None or a tuple of (score, gesture). Thus, if there is a match we need to pull out the Gesture via match[1].

And there you have it: basic gesture recognition in Kivy.

Unfortunately, the touch events are gobbled up by the gesture code, so if you have a gesture widget that contains other widgets, they won’t receive any events. I have taken a stab at creating a GestureBox widget that passes events through to child widgets. It seems to work for touch events, but deciding whether a motion event

Python on Android? First impressions of Kivy

Kivy is a modern cross-platform Python GUI toolkit that runs on mobile devices (Android and IPhone) and supports modern input events (multitouch, accelerometor). I was really skeptical at first, but I spent the weekend developing an app in Kivy, and now I’m hooked.

In the past, I have tried most of the available Python GUI toolkits (even Swing, through jython). In recent history, I’ve mostly done HTML based interfaces, partially because that’s where the money has been, but mostly because pyQT, pyGTK,wxpython, and tk are all painful to work with.

In Kivy, interfaces are defined in a language called (aptly) the Kivy Language. The best way to describe the Kivy Language is “what HTML5 wishes it was.” It is mostly just a layout language, but it also supports binding properties and events using inline Python snippets. It is indentation defined, like Python, and utilizes a minimal amount of syntax with a maximal amount of readability. It is vaguely reminiscent of CoffeeKup but cleaner, and doesn’t suffer from having to be compiled to HTML. I love how it mimics Python syntax, but also clearly separates presentation from control code.

My favourite feature of Kivy is that properties are events. You can easily hook a property on one widget to the property on another. For example, I wrote a simple demo that connects the value of a slider to the value of a progressbar:

BoxLayout:
    orientation: 'vertical'
    ProgressBar:
        id: bar
        value: 14
        max: 20
    Slider:
        id: slider
        max: 200
        value: 140
        on_value: bar.value = self.value / 10

When the value property on the slider changes (whether because the user moved the slider, or because it was adjusted programmatically in response to some other event), the on_value event is fired. This is connected to the value of the progress bar using a Python snippet.

Only a few lines of Python code are required to to get a Kivy application running once its interface has been defined in Kivy language. The Kivy Hello World example is actually overly complicated because it doesn’t actually use the Kivy language. The bare essentials, if the above file is named sample.kv is:

from kivy.app import App
class SampleApp(App): pass
SampleApp().run()

Just three lines of code. Kivy introspects the class name (convention over configuration) and automatically loads sample.kv as the interface for a class named SampleApp.

The Kivy API is very simple. In fact, my earliest impression was that it was almost amateurish. It felt like if I wanted to do anything serious with it, I’d be disappointed. It’s the kind of API I would teach a child. However, the few features of the API (basically properties, events, and graphics instructions) are extremely well thought out and well-defined. They can be combined like Lego bricks to create widgets as complicated as one could possibly wish. The fact that the library includes a ReStructured Text rendering widget is testament to that fact!

Kivy is extremely well documented. I actually found their pong tutorial exciting. Seeing how simple it was to do each step actually made me giggle out loud. The programmer’s guide and API are well presented as well. Further, if you get stuck, the developers on IRC are terrific. I felt like I was an accepted member of the team after asking only one question.

Drawbacks

It took a bit of fiddling to get Kivy to install correctly under Arch Linux. I suppose if I’d followed the instructions, all would have been well. However, I like to do all my development in a virtualenv, and there is no trivial way to track down and install all Kivy’s dependencies. This is especially compounded by the fact that pygame currently needs to be patched under Arch in order to compile.

There are instructions for packaging your Kivy app onto Android, but I wasn’t able to pull it off, yet. I’ll have to revisit it when I have more time.

I tried some sample Kivy apps in the Android market. They work just fine once they are loaded, but it takes each app several seconds to initialize on my Galaxy Nexus. Hopefully the Kivy team can reduce this load time. I don’t know if Kivy supports displaying a splash screen while stuff is loading; that would certainly enhance the usability.

Most of these issues are solvable. I’m sure the team is actively working on some of them. Kivy is a quite young project, but it is remarkably mature.

Kivy Catalog

My weekend project was the Kivy Catalog, an interactive showcase of Kivy widgets. It allows you to view the Kivy language code used to define the widgets and to edit it and see what effects your changes have. I think it will be a tremendous aid to users looking to get started with the Kivy language. Developing it certainly was for me!

I am expecting to have this code included with the Kivy distribution in the examples directory in a future release.