Creating Apps In Kivy: The Book

I apologize for my near-silence over the past few months. I’ve packed house and moved twice in that time, started a new job, and, relevant to this post, written a book.

My Creating an application in Kivy blog series has been by far the most popular set of posts on this blog. It garners about 200 views per day, and has been doing so for nearly a year. This taught me a very important lesson: posts with controversial titles, (notably this one, but basically anything that has hit hacker news) are much less valuable than posts with meaningful long-term content. Getting a few thousand hits in one day is pretty satisfying, but when most of your “readers” simply comment on the post without having read the article, that hit-count becomes meaningless.

“Creating an application in Kivy”, on the other hand, has had sixteen thousand visits to part 1, and nearly three thousand of those have made their way all the way to part 9. I could definitely achieve more hits per day by writing a new minimum-content controversial article every day, but these visitors keep coming with no effort on my part. This is lucky, as the focus on writing solid content for my book has taken time away from writing solid content for this blog.

And with that, I’d like to announce that Creating Apps In Kivy is now available. This project started when the aforementioned tutorial caught the eye of an O’Reilly editor. They helped me put together a great book that I am proud of and confident you will enjoy.

Creating Apps in Kivy covers many of the topics discussed in my tutorial, but in more depth and with much more polish. Like the tutorial, it takes you from “new to Kivy” to creating and deploying a complete application to your Android or iOS device. It’s a fun, short read and I’m proud to present it to you today. If you liked the tutorial, buying my book would be a great way to support my work!

Have you ever been afraid to get out of bed?

In my past writings about mental health, I’ve mostly discussed depression and suicide. I’ve been meaning to add ‘anxiety’ to that list of topics, but have been waiting until I could write about it from the first person. One of the peculiar things about my personal struggles with mental illness is that when I’m feeling well, it’s really hard to remember what it’s like to be feeling poorly. The inverse is also true, which is why it’s possible to feel suicidal after just a few hours of depression; I can’t imagine what it felt like yesterday when I’d been feeling fine for months on end.

I’ve got a great handle on what I need to do in my life to deal with, prevent, and accommodate for my depressive, suicidal, and bipolar tendencies. I’ve also made great strides in reducing anxiety levels, but, for me, anxiety is a more difficult beast to tame. The following was written a few days ago when I was in the middle of a high-anxiety experience (not an anxiety attack). I’ve edited it a bit for grammar (typing on a phone is a horrible experience), but it’s mostly straight from the heart. I hope it illustrates what anxiety feels like to those who have not experienced it, so you can support those in your life who are suffering. I would also like it to demonstrate to those who may not be aware that they are dealing with anxiety that their symptoms are treatable, and perhaps you will seek further help. And, as always, I want those of you who deal with anxiety on a regular basis to remember that you are not alone.

From Inside Anxiety

Let me tell you what anxiety feels like. It feels like being on a small rock amidst a boiling vat of lava. I have to type this on a phone because the laptop in the other room is no less inaccessible than if I had to jump a giant ravine. Anxiety is fear, pure simple fear. But it’s not normal fear; anxiety is fear of the normal.

Have you ever experienced an adrenaline rush? Some threatening situation: the grill of an oncoming bus, falling down the stairs, being approached by a thug with knife in hand, slipping on ice, a shove from behind, a car accident?

Anxiety is that same feeling, but all the time. Every step is a slip, every car is going to hit you, all your friends and co-workers are thugs with knives. The worst part is that the brain can see that these sensations are not true. I’m not imagining real lava around the bed. I can see the perfectly safe carpet. But my legs feel like numb jello and I cannot step onto it. I’m not hallucinating that all the people outside my street want to kill me. I’m afraid of them even though I know they have no interest in me at all. I know in my mind that my office is full of people I can trust and rely on, yet my body is reacting as though I will be entering a war zone, every sense alert, every muscle tensed against the next attack.

Once upon a time, I lived the majority of my life’s moments in such a constant state of heightened awareness. Fear. It was exhausting. I am lucky that days like this are now rare. I am lucky that today, my anxiety is not accompanied by depression (it’s hard to feel worthy when you spend all your time hiding in bed.).
Some people look for adrenaline rushes outside their normal state of being. They sky dive, they become immersed in video games or movie thrillers, they join extreme sports. While they do this, I look for quiet rooms, I meditate, I practice yoga. Anything to calm the blood down. I will experience the same thrill climbing into the shower this morning that you feel jumping out of that plane. I will feel the same fear walking into my office today that you feel as you step onto a public stage.

And here’s the kicker: nobody thinks I’m brave. My walk to work today will mirror an Indian Jones escape sequence or Aladdin’s carpet ride out of the Cave Of Wonders. I’ll walk into my office and be surrounded by people who do that kind of thing every day. Anxiety is living your life inside a thriller movie. The music is always loud, rushing your heart beat. Something awful is always going to happen around the next corner. This movie doesn’t end and you can’t walk out of the theater.

The world is going to end if I don’t get out of bed and into the shower. How much is a world worth?

Dealing With Anxiety

I have had less success in defeating anxiety than I have had with depression. There are a few types of treatments to deal with it, and you should consult with a medical professional to work out what the best one is for you. The two things that have had the greatest success in taming (if not controlling, yet) my anxiety have been yoga/meditation, and exposure therapy. The latter isn’t as bad as it sounds, although like many cures, it’s not always pleasant. The heart of exposure therapy is, as you can expect from the name, placing yourself in situations that cause anxiety. However, there are a few caveats. You have to place yourself in such situations until you become comfortable with them. If you exit an anxiety-inducing situation while you still feel afraid, you will have reinforced that this is a fearful situation, and that “fight or flight” is an appropriate response.

Instead, you need to pick situations that only cause a small amount of anxiety. A good way to do this is to make a list of as many situations as you can think of that cause anxiety, and then rank them in order of how frightening they are. Over the course of a year or so, put yourself into the easiest situations and work your way up the chain. It’s slow progress, but it’s measurable, and the feedback you give yourself from one encounter can help you refine your technique in a later one.

I’ve been utilizing this technique for three years. As illustrated above, I still have days with a lot of anxiety. However, I lead a functional life, now. I used to work from home, I now work for a high-profile company and I’m doing well at it. I used to avoid all social situations, I now regularly go out with friends or even strangers and generally have a good time. I used to be especially afraid to interact with single women; I’m now in a stable, loving relationship. There are still a few things on my list, such as presenting at conferences and dealing better with conflict, but for the most part my coping skills are better than many people who are considered not to have psychological disorders. It wasn’t easy, but it was worth it.

I’m Not Selling Out The Gittip Dream

A couple weeks ago, I signed a contract with O’Reilly to publish a book inspired by my extremely popular Creating An Application In Kivy blog series. On Monday, I start work at Facebook, a full-time salaried position that includes a substantial raise over my last position.

Signing these two contracts, two very traditional ways of obtaining money in exchange for services, is in stark contrast to the excitement I have held for the Gittip project. I marketed (some might say ‘begged’) for Gittip contributions in return for the service I performed in writing the Kivy articles. Gittip provided; I really didn’t expect to make more than a couple dollars per week, but my current income from Gittip is over $11 per week. I’m touched and grateful for these donations, and I don’t feel I deserve them at all, considering that I’ll be spending most of my time on my new book and job for the next few months.

It would be easy to argue that Gittip has failed. If I had somehow made more money off Gittip, I might have chosen to publish a book under a creative commons license or through other platforms, rather than signing a traditional book contract. If I was making so much money off Gittip that I could have laughed off Facebook’s salary and stayed out of the horrifically over-priced silicon valley, the open source world (and Facebook’s competitors!) would certainly have benefited.

However, there is a better way to look at this. I have committed (to myself) to contribute a substantial portion of the royalties from my new book back to the Kivy team. I will naturally use Gittip to distribute these funds. I am also planning to increase my funding of other projects out of my Facebook salary, and I’ll be using Gittip to fund the Arch Linux development team (if they ever set one up on Gittip) out of the income I already make from Arch Linux Schwag.

I believe one of the biggest problems Gittip has to solve is not increasing it’s member base, but increasing the amount of money being injected into the system. If small tips are just moving from one open source developer to another and back again (I’ve gotten into the unfortunate habit of regifting everything I make plus a few dollars), the money really has no value. It is a small gesture, but it’s not something that can be turned into food on the table if everyone just turns around and gives it away. I would definitely like to live a life, someday, where my writing and coding activities are exciting enough to the world that my entire salary comes from Gittip. However, I think the best thing I can do for the tool right now is to put money in, rather than take it out.

So no, Gittip has not failed me. It is succeeding in a different way from what I originally anticipated. Nor am I selling out. I am excited and passionate about my current and future prospects. Gittip was designed to allow content creators to pursue their passions. I’m able to do that within the traditional framework. I’m very lucky to be able to express my gratitude for my current life in the form of microdonations to other developers.

Arch Linux Schwag site is back up (a comment on privacy)

Hey all,

http://schwag.archlinux.ca has been down for at least a week. I’m really sorry about that. I’ve been working on transferring to a new web host and they were having trouble getting all the Python dependencies I needed.

My old host was based in the United States, and while they provided exceptionally good service for the price, I have become uncomfortable with storing any data in the US. The websites are not private, of course. However I am also hosting e-mail on these domains. As a Canadian citizen, I am not covered by the NSA’s (possibly false) claim that they do not spy on US citizens, so I felt that I should store it in my own country.

I don’t do anything that is currently illegal, but I don’t know that these arrogant governments aren’t going to change their laws to restrict my freedoms. I may not be a person of interest today, but I am outspoken about freedom, privacy, and transparency. This is currently legal and acceptable, but that may not always be the case and I want to be proactive about keeping my private data out of the hands of those who would use it against me.

The importance of “One Obvious Way”

In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.

— Tim Peters, The Zen of Python
>>> import this

While Python has numerous advantages over other languages that make me reach for it first when starting a new project, I feel the two most important are its emphasis on readability and the community belief in Tim’s words above, “there should be one– and preferably only one –obvious way to do it.” While it’s rare to stumble upon a Python coder that doesn’t implicitly “get” this principle, I find that is frequently misinterpreted when I try to explain Python’s charm to outsiders.

This may be because I often misquote the principle as “the one best way” principle. Many languages have a “one best way” principle. Java, for example, insists that the best way to solve any problem is object oriented programming. Similarly, prolog insists that logic programming is the best way, and Haskell programmers suggests that functional is always the way to go.

Python is not like this at all. Python is pragmatically multi-paradigm. Python has a long history of stealing great ideas from other languages (and, given Guido’s genius, making them better). Python does the very opposite of insisting that there is a best way to do everything.

Orthogonally, I have often had my defense of this principle misinterpreted as having said, “Python is always the best solution to any problem.” This makes me sound arrogant and misinformed, and is patently false. I am keenly aware of Python’s shortcomings. My day job has me pushing Python’s limits on a regular basis, and often, it fails me.

No, Python is not always the best way. The “one obvious way” principle refers to something quite different. It’s not about the language or its libraries at all, it’s more about the programmer’s mindset. Given any specific problem, there are invariably multiple ways to solve it. A Python developer will start out by assuming that one of those assorted solutions is the best solution. The goal of both the Python language and the libraries developed by the community is to make that best solution the obvious one to implement.

This is a form of consistency. Just as pep-8 strongly encourages us to be consistent with all other Python developers in our coding style, the “one obvious way” principle encourages us to be consistent in our choices of algorithms, data structures, and design patterns. In an ideal world, this means that any two Python programmers, given the same problem, would write virtually the same solution. Further, the solution that they proposed would be the best possible solution for the given problem, and all other pythonistas would find it elegantly readable.

This never happens. However, I believe that Python programs are closer to this ideal than many (if not most) other languages. This cohesion means that Python code is often more maintainable, not just by the original developer, but by all members of the Python community.

Some might suggest that strict adherence to the “one obvious way” principle is narrow-minded and restricts experimentation. I heartily disagree. There is clearly not one best web framework in Python. There are numerous web engines available, some more suitable to specific purposes than other. And more are being written every day. I think this is because nobody strictly believes that the most obvious implementation has been found, yet. The poem says there should be one obvious way. It doesn’t say that there always is one. The Python community as a whole is working together to find these best solutions and make them into the obvious solution. A few problems are solved. If you need an iterator, Python’s got you covered. Others are still up in the air, as evidenced by the collection of web frameworks or packaging tools or xml parsers on pypi. Some have only been recently solved; the Requests library is a terrific example. And still more are completely unsolved. The most obvious way I’ve found to handle concurrency and parallelism in Python is: Use Golang. Kivy is a terrific library, but the obvious way to do Android development is still (currently) Java.

In summary, while the ideal of “obviousness” is far, far away, I believe that it is a worthy goal to pursue. I think the mindset attracts a certain kind of programmer, and that those programmers are all as delighted when they discover Python as I was almost a decade ago. I don’t think Python is the best tool for all jobs, nor do I think it is the most enjoyable tool for all programmers. However, for those coders who like a certain kind of obvious, readable, elegance, Python is the language of choice.

Creating an Application in Kivy: Part 9

Welcome to the final part of this series on creating a working Jabber client in Kivy. While there are numerous things we could do with our code to take it to Orkiv version 2, I think it’s currently in a lovely state that we can call “finished” for 1.0.

In this article, we’re going to talk about releasing and distributing Kivy to an Android mobile device. We’ll use a tool called buildozer that the Kivy developers have written. These commands will, in the future, extend to other operating systems.

We aren’t going to be writing a lot of code in this part (some adaptation to make it run on android is expected). If you’re more interested in deploying than coding, you can check out my state of the orkiv repository as follows:

git clone https://github.com/buchuki/orkiv.git
git checkout end_part_eight

Remember to create and activate a virtualenv populated with the appropriate dependencies, as discussed in part 1.

Table Of Contents

Here are links to all the articles in this tutorial series:

  1. Part 1: Introduction to Kivy
  2. Part 2: A basic KV Language interface
  3. Part 3: Handling events
  4. Part 4: Code and interface improvements
  5. Part 5: Rendering a buddy list
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Different views for different screens
  9. Part 9: Deploying your kivy application

Restructuring the directory

Back in Part 1, we chose to put all our code in __main__.py so that we could run it from a zip file or directly from the directory. In some ways, this is pretty awesome for making a package that can easily be distributed to other systems (assuming those systems have the dependencies installed). However, Kivy’s build systems presume that the file is called main.py.

I’m not overly happy about this. Kivy sometimes neglects Python best practices and reinvents tools that are widely used by the Python community. That said, it is possible for us to have the best of both worlds. We can move our code into orkiv/main.py and still have a __main__.py that imports it.

First, move the entire __main__.py file into main.py, using git:

    git mv orkiv/__main__.py orkiv/main.py

Next, edit the new main.py so that it can be imported from other modules without automatically running the app. There is a standard idiomatic way to do this in python. Replace the code that calls Orkiv().run() with the following:

(commit)

    def main():
        Orkiv().run()

    if __name__ == "__main__":
        main()

There are two things going on here. First, we moved the code that instantiates the application and starts it running in a function called main(). There is nothing exciting about this function. It does the same thing that used to happen as soon as the module was imported. However, nothing will happen now unless that function is called.

One way to regain the previous functionality of immediately running the app when we type python main.py would be to just call main() at the bottom of the module. However, that is not what we want. Instead, our goal is to make the application automatically run if the user runs python orkiv/main.py, but if they run import main from a different module or the interpreter, nothing is explicitly run.

That’s what the if __name__ conditional does. Every module that is ever imported has a magic __name__ attribute. This is almost always the name of the module file without the .py extension (eg: main.py has a __name__ of main). However, if the module is not an imported module, but is the script that was run from the command line, that __name__ attribute is set to __main__ instead of the name of the module. Thus, we can easily tell if we are an invoked script or an imported script by testing if __name__ is __main__. Indeed, we are taking advantage of the same type of magic when we name a module __main__.py. When we run python zipfile_or_directory it tries to load the __main__ module in that directory.

However, we do not currently have a __main__ module, since we just moved it. Let’s rectify that:

    from main import main
    main()

Save this as a new orkiv/__main__.py and test that running python orkiv from the parent directory still works.

You’ll find that the window runs, however, if you try to connect to a jabber server, you get an exception when it tries to display the buddy list. This is because our orkiv.kv file was explicitly importing from __main__. Fix the import at the top of the file so that it imports from main instead:

#:import la kivy.adapters.listadapter
#:import ok main

OrkivRoot:

While we’re editing code files, let’s also add a __version__ string to the top of our main.py that the Kivy build tools can introspect:

(commit)

    __version__ = 1.0

Deploying to Android with buildozer

Deploying to Android used to be a fairly painful process. Back in December, 2012, I described how to do it using a bulky Ubuntu virtual machine the Kivy devs had put together. Since then, they’ve designed a tool called buildozer that is supposed to do all the heavy lifting. Once again, I have some issues with this plan; it would be much better if Kivy integrated properly with setuptools, the de facto standard python packaging system than to design their own build system. However, buildozer is available and it works, so we’ll use it. The tool is still in alpha, so your mileage may vary. However, I’ve discovered that Kivy products in alpha format are a lot better than final releases from a lot of other projects, so you should be ok.

Let’s take it out for a spin. First activate your virtualenv and install the buildozer app into it:

    . venv/bin/activate
    pip install buildozer

Also make sure you have a java compiler, apache ant, and the android platform tools installed. This is OS specific; these are the commands I used in Arch Linux:

    sudo pacman -S jdk7-openjdk
    sudo pacman -S apache-ant
    yaourt android-sdk-platform-tools

Then run the init command to create a boilerplate buildozer.spec:

    buildozer init

Now edit the buildozer.spec to suit your needs. Most of the configuration is either straightforward (such as changing the application name to Orkiv) or you can keep the default values. One you might overlook is that source.dir should be set to orkiv the directory that contains main.py.

The requirements should include not only kivy and sleekxmpp but also dnspython, a library that sleekxmpp depends on. You can use the pip freeze command to get a list of packages currently installed in your virtualenv. Then make a conscious decision as to whether you need to include each one, remembering that many of those (cython, buildozer, etc) are required for development, but not Android deployment.

For testing, I didn’t feel like creating images for presplash and the application icon, so I just pointed those at icons/available.png. It’s probably prettier than anything I could devise myself, anyway!

If you want to see exactly what changes I made, have a look at the diff.

The next step is to actually build and deploy the app. This takes a while, as buildozer automatically downloads and installs a wide variety of tools on your behalf. I don’t recommend doing it over 3G!

First, Enable USB debugging on the phone and plug it into your development machine with a USB cable. Then tell buildozer to do all the things it needs to do to run a debug build on the phone:

buildozer android debug deploy run

I had a couple false starts before this worked. Mostly I got pertinent error messages that instructed me to install the dependencies I mentioned earlier. Then, to my surprise, the phone showed a Kivy “loading…” screen. Woohoo! It worked!

And then the app promptly crashed without any obvious error messages or debugging information.

Luckily, such information does exist. With the phone plugged into usb, run adb logcat from your development machine. Review the Python tracebacks and you’ll discover that it is having trouble finding the sound file in.wav:

I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/audio/__init__.py", line 135, in on_source
I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/audio/audio_pygame.py", line 84, in load
I/python  (25368):    File "/home/dusty/code/orkiv/.buildozer/android/platform/python-for-android/build/python-install/lib/python2.7/site-packages/android/mixer.py", line 202, in __init__
I/python  (25368):  IOError: [Errno 2] No such file or directory: '/data/data/ca.archlinux.orkiv/files/orkiv/sounds/in.wav'
I/python  (25368): Python for android ended.

The problem here is that the code specifies a relative path to the sound file as orkiv/sounds/in.wav. This worked when we were running python orkiv/ from the parent directory, but python for android is running the code as if it was inside the orkiv directory. We can reconcile this later, but for now, let’s focus on getting android working and just hard code the file location:

(commit)

    self.in_sound = SoundLoader.load("sounds/in.wav")

While we’re at it, we probably also need to remove “orkiv” from the location of the icon files in orkiv.kv:

    source: "icons/" + root.online_status + ".png"

Finally, running buildozer and adb logcat again indicates the error message hasn’t changed. This is because we aren’t explicitly including the .wav file with our distribution. Edit buildozer.spec to change that:

    # (list) Source files to include (let empty to include all the files)
    source.include_exts = py,png,jpg,kv,atlas,wav

Run the buildozer command once again and the app fire up and start running! Seeing your Python app running on an Android phone is a bit of a rush, isn’t it? I was able to log into a jabber account, but selecting a chat window goes into “narrow” mode because the phone’s screen has a much higher pixel density than my laptop. We’ll have to convert our size handler to display pixels somehow.

I was able to send a chat message, which was exciting, but when the phone received a response, it crashed. Hard. adb logcat showed a segmentation fault or something equally horrifying. I initially guessed that some concurrency issue was happening in sleekxmpp, but it turned out that the problem was in Kivy. I debugged this by putting print statements between each line in the handle_xmpp_message method and seeing which ones executed before it crashed. It turned out that Kivy is crashing in its attempt to play the .wav file on an incoming message. Maybe it can’t handle the file format of that particular audio file or maybe there’s something wrong with the media service. Hopefully the media service will be improved in future versions of Kivy. For now, let’s use the most tried and true method of bugfixing: pretend we never needed that feature! Comment out the line:

(commit)

    #self.in_sound.play()

and rerun buildozer android debug deploy run.

Now the chat application interacts more or less as expected, though there are definitely some Android related quirks. Let’s fix the display pixel issue in orkiv.kv:

(commit)

    <OrkivRoot>:
        mode: "narrow" if self.width < dp(600) else "wide"
        AccountDetailsForm:

All I did was wrap the 600 in a call to dp, which converts the 600 display pixels into real pixels so the comparison is accurate. Now when we run the app, it goes into “narrow” mode because the screen is correctly reporting as “not wide enough for wide mode”. However, if you start the app in landscape mode, it does allow side-by-side display. Perfect!

And that’s the app running on Android. Unfortunately, in all honesty, it’s essentially useless. As soon as the phone display goes to sleep, the jabber app closes which means the user is logged out. It might be possible to partially fix this using judicious use of Pause mode. However, since the phone has to shut down internet connectivity to preserve any kind of battery life, there’s probably a lot more involved than that.

On my phone, there also seems to be a weird interaction that the BuddyList buttons all show up in a green color instead of the colors specified in the Kivy language file and the args_converter.

Third, the app crashes whenever we switch orientations. This is probably a bug fixable in our code rather than in Kivy itself, but I don’t know what it is.

Also, touch events are erratic on the phone. Sometimes the scroll view won’t allow me to scroll when I first touch it, but immediately interprets a touch event on the ListItem. Sometimes touch events appear to be forgotten or ignored and I have to tap several times for a button to work. Sometimes trying to select a text box utterly fails. I think there must be some kind of interaction between Kivy’s machinery and my hardware here, but I’m not certain how to fix it.

Occasionally, when I first log in, the BuddyList refuses to act like a ScrollView and the first touch event is interpreted as opening a chat instead of scrolling the window. This is not a problem for subsequent touch events on the buddy list.

Finally, there are some issues with the onscreen keyboard. When I touch a text area, my keyboard (I’m using SwiftKey) pops up, and it works well enough. However, the swipe to delete behavior, which deletes an entire word in standard android apps, only deletes one letter here. More alarmingly, when I type into the password field, while the characters I typed are astrisk’d out in the field, they are still showing up in the SwiftKey autocomplete area. There seems to be some missing hinting between the OS and the app for password fields.

Before closing, I’d like to fix the problem where the app is no longer displaying icons on the laptop when I run python orkiv/. My solution for this is not neat, but it’s simple and it works. However, it doesn’t solve the problem if we put the files in a .zip. I’m not going to worry about this too much, since buildozer is expected, in the future, to be able to make packages for desktop operating systems. It’s probably better to stick with Kivy’s tool. So in the end, this __main__.py feature is probably not very useful and could be removed. (Removing features and useless code is one of the most important parts of programming.) However, for the sake of learning, let’s make it work for now! First we need to add a root_dir field to the Orkiv app. This variable can be accessed as app.root_dir in kv files and as Orkiv.get_running_app().root_dir in python files.

(commit)

    class Orkiv(App):
        def __init__(self, root_dir):
            super(Orkiv, self).__init__()
            self.root_dir = root_dir
            self.xmpp = None

Of course, we also have to change the main() function that invokes the app to pass a value in. However, we can have it default to the current behavior by making the root_dira keyword argument with a default value:

    def main(root_dir=""):
        Orkiv(root_dir).run()

Next, we can change the __main__.py to set this root_dir variable to orkiv/:

    from main import main
    main("orkiv/")

The idea is that whenever a file is accessed, it will be specified relative to Orkiv.root_dir. So if we ran python main.py from inside the orkiv/ directory (this is essentially what happens on android), the root_dir is empty, so the relative path is relative to the directory holding main.py. But when we run python orkiv/ from the parent directory, the root_dir is set to orkiv, so the icon files are now relative to the directory holding orkiv/.

Finally, we have to change the icons line in the kivy file to reference app.root_dir instead of hardcoding the path (a similar fix would be required for the sound file if we hadn’t commented it out):

    Image:
        source: app.root_dir + "icons/" + root.online_status + ".png"

And now, the app works if we run python orkiv/ or python main.py and also works correctly on Android.

There are a million things that could be done with this Jabber client, from persistent logs to managed credentials to IOS deployment. I encourage you to explore all these options and more. However, this is as far as I can guide you on this journey. Thus part 9 is the last part of this tutorial. I hope you’ve enjoyed the ride and have learned much along the way. Most importantly, I hope you’ve been inspired to start developing your own applications and user interfaces using Kivy.

Financial Feedback

Writing this tutorial has required more effort and consumed more time than I expected. I’ve put on average five hours per tutorial (with an intended timeline of one tutorial per week) into this work, for a total of around 50 hours for the whole project.

The writing itself and reader feedback has certainly been compensation enough. However, I’m a bit used up after this marathon series and I’m planning to take a couple months off before writing anything like this tutorial again in my free time.

That said, I have a couple of ideas for further tutorials in Kivy that would build on this one. I may even consider collecting them into a book. It would be ideal to see this funded on gittip, but I’d want to be making $20 / hour (far less than half my normal wage, so I’d still be “donating” much of my time) for four hours per week before I could take half a day off from my day job to work on such pursuits — without using up my free time.

A recent tweet by Gittip founder Chad Whitacre suggests that people hoping to be funded on gittip need to market themselves. I don’t want to do that. I worked as a freelancer for several years, and I have no interest in returning to that paradigm. If I have to spend time “selling” myself, it is time I’m not spending doing what I love. In my mind, if I have to advertise myself, then my work is not exceptional enough to market itself. So I leave it up to the community to choose whether to market my work. If it is, someone will start marketing me and I’ll see $80/week in my gittip account. Then I’ll give those 4 hours per week to tutorials like this or other open source contributions. If it’s not, then I’ll feel more freedom to spend my free time however I like. Either way, I’ll be doing something that I have a lot of motivation to do.

Regardless of any financial value, I really hope this tutorial has been worth the time you spent reading it and implementing the examples! Thank you for being an attentive audience, and I hope to see you back here next time, whenever that is!

Creating Apps In Kivy: The Book

My request for crowd funding above didn’t take off. I’m making just under $12 per week on Gittip, all of which I’m currently regifting. Instead of my open contribution dream being fulfilled, I opted to follow the traditional publishing paradigm and wrote a book Creating Apps In Kivy, which was published with O’Reilly. I’m really excited about this book; I think it’s the highest quality piece of work I have done. If you enjoyed this tutorial, I encourage you to purchase the book, both to support me and because I think you’ll love it!

Bad Guy Discrimination

On my flight back from an absolutely amazing Python Canada 2013, I watched “42″ (Jackie Robinson’s story) and about half of “Skyfall”. The former warmed my heart because the racial discrimination was so disturbing. You’re probably reading that twice and scratching your head, so let me explain: It’s heartwarming that we have made progress in fighting racism, and that things that were considered normal 70 years are truly disgusting today. While nauseating racism, sexism, homophobia, and other forms of othering still exist today, in Canada, at least, it is no longer considered normal. For example, last year, the NHL media was in uproar because of an alleged racial slur against PK Subban. Yes that slur should not have happened, but 60 years ago, it would have been encouraged.

Skyfall naturally reminded me of past decades of James Bond movies. Traditionally, the bad guy in a Bond film is a dude with an accent reflecting whatever country the American media (while Bond is British, the films are Hollywood) was hyperventilating about at the time. This “bad guy with an accent” portrayal is a form of xenophobia that Hollywood has been projecting for decades.

In Skyfall, as well as a majority of the very few other action movies I’ve seen in the past few years, the bad guy is not black like in the 70s, Russian like in the 80s, or middle Eastern like in the 90s. In movies such as Skyfall, Iron Man 2 and 3, all three Batman movies, and a super hero movie that I can’t remember the name of (citation required) the bad guy as a deranged local white man. Some of those insane white dudes have been phenomenal villains. Heath Ledger’s incredible portrayal of The Joker was particularly poignant.

Deranged local white men. These movies are silently teaching us, in the same way that movies of the 60s and 70s taught us that blacks are gangsters and women simper, that mentally ill people are dangerous, different, frightening. And you know what? We are! Even with the empathy I have developed from group therapy sessions and being hospitalized alongside seriously mentally ill individuals, I still get nervous when “crazy people” approach me on the street. I personally identified with The Joker, and my girlfriend at the time even commented that he reminded her of me.

The bad guy in a film normally has to be a little demented. I don’t want to live in a society where the desire to blow up entire cities of peace loving citizens is considered sane. I’m just here to remind people to encourage, rather than suspend, disbelief when watching all movies. Disbelieve the gender roles typified in Disney films. Disbelieve that people with accents are bad guys. Disbelieve that Asians can kill you with their pinky. Disbelieve that crazy men, men like me, are necessarily evil.

Creating an Application in Kivy: Part 8

Sorry for the delay in posting this, folks. I’ve been away on vacation and then at Pycon Canada, which was amazing. And another apology, in advance, that this article is going to be bloody short compared to parts 1 through 7. That’s not only because I’m short on time; it also turned out that what I wanted to do was easier than I expected!

Anyway, time for content! Now that we have a working jabber client, is our job done? The client is certainly functional, but I’d still like to experiment with displaying a different representation depending on the size of the window. Specifically, if it is wider than a certain number of pixels, let’s show the buddy list beside the chat window; otherwise, let’s stick with the switching back and forth that we currently have.

If you haven’t been following along or want to start this example at the same point at which I’m starting, you can clone the github repository with my examples:

git clone https://github.com/buchuki/orkiv.git
git checkout end_part_seven

Remember to create and activate a virtualenv populated with the appropriate dependencies, as discussed in part 1.

Table Of Contents

Here are links to all the articles in this tutorial series:

  1. Part 1: Introduction to Kivy
  2. Part 2: A basic KV Language interface
  3. Part 3: Handling events
  4. Part 4: Code and interface improvements
  5. Part 5: Rendering a buddy list
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Width based layout
  9. Part 9: Deploying your Kivy application

Thinking through design

Since I’m not entirely certain how I want this code (or interface) to look, let’s spend a few paragraphs describing it so we all know we’re developing the same thing. It’s simple to say “we want chat windows to display beside the buddy list when the screen is big enough”, but what does that mean exactly? At what points in the code do we have to handle it? I’m actually “thinking out loud” here so you can see the various paths a brain can follow when programming.

Let’s follow the process from program startup. When the account credentials are displayed, we may as well just show the form no matter how big the screen is. When the Buddy List is first displayed after login, it can take up the whole screen as well. It’s not until we open our first chat that we have to decide if we are going to add the new chat beside the buddy list or to replace it. Of course, it’s not just a matter of adding the chat window, we also have to remove any existing chat windows, since we don’t just want to keep adding windows every time they click a new buddy. One way we could do this is add a ChatWindowContainer and ensure that widget only contains one chat window. Another is to keep track of which widows are loaded and unload them as needed. A third option is to clear all widgets and add both the Buddy List and Chat Window to a clean slate.

Regardless of the width of the screen, when the user clicks the “Buddy List” button in the chat window, it should do the same thing: close all chat windows and show only the buddy list, full width. One option worth exploring is to hide that button when we are in wide screen view. Or perhaps the button should be removed altogether; a gesture might be a better interface. While gestures are all the rage in mobile devices right now, I’m not a fan of them from an affordance perspective. It’s never clear to the end user when or where a gesture is appropriate. While I use them extensively in my highly customized AOKP installation, I wouldn’t want to explain a gesture based interface to my parents.

In addition to deciding what to render when chat windows are added or removed, we need to do something when the window changes size. If it is in wide screen mode while a chat window is displayed, and then gets smaller, we have to remove the buddy list to switch to the original behavior. If it is in narrow mode while a chat window is displayed and it gets wider, we have to add the buddy list back.

I feel like it would be wise to have a “mode” property bound to the window size, so that we always know whether we are currently supposed to be in wide or narrow mode. However binding that property to size means that the mode will change without telling us what the “old” mode was. Thus, we have no way to know whether the mode we are supposed to be in is the one we are currently in. On the other hand, such information can be inferred from the size or contents of the child list on the OrkivRoot object.

Finally, it seems most of our calculations can be simplified by noticing that we only have to do things different if a chat window is currently displayed or about to be displayed.

Now I think I have a plan in my head. You may or may not have come to the same conclusions while reading the above, and it’s possible you’ve had some better ideas. That said, this is how I intend to implement this. It looks like it’s going to be simpler than I originally expected. Hooray for thinking things through in advance!

1. Add a mode property to OrkivRoot that is automatically set from the size of the window
2. Add chat_visible and budddy_list_visible python properties (not kivy properties) that tells us if there are buddy list or chat windows visible.
3. Check the mode when adding a chat window and decide whether to replace the buddy list or not
4. On a size event, if a chat window is currently visible, determine whether the buddy list needs to be added or removed.

Properties

We’ve looked at Kivy properties in past parts of this tutorial, and you may already be familiar with Python properties. It’s somewhat unfortunate that the same word is being used to explain two different programmatic concepts. However, there are only so many words to describe the idea of a “characteristic”, and this idea is extremely common in many different programming libraries. So some overloading of terminology is inevitable. I tend to use the phrase “Kivy property” to refer to the kinds of properties that Kivy supplies. Kivy properties are pretty cool because they can automatically update themselves when other properties change and events can be bound to them so that those other properties also update themselves. In other words, Kivy properties are smart.

Python properties are not stupid, mind you. People sometimes have trouble wrapping their mind around Python properties, but they’re actually pretty simple. It might be convenient to think of python properties as just a way to make a method look like an attribute. Thus, you can access something like node.color and underneath the hood, it’s actually behaving as though node.color() was called. However, they’re even more interesting than that, because you can actually call a completely different method when you set the value of the property. Thus node.color = "red" can do about the same thing as node.color("red").

The confusing thing about python properties is not so much “how do I do this?” as “why would I do this”? There are many problem-specific answers, but two general ones come to mind. The first is the Python idiom that readability counts. In general, when you access a property on an object, you are doing something with an adjective on a noun. In contrast, when you call a method on an object, you are doing something (a verb) to a noun. However, sometimes when you are manipulating an adjective, you need to do a little extra calculation or manipulation when the value is set or accessed. Using properties, it’s possible for the code accessing the property to read “naturally”, as though you were doing something with an adjective on a noun, even though there is actually a little “doing to” going on as well.

The second reason is actually an evolution of the first. Often, in the first version of an API, we just add an attribute to an object and allow client code to manipulate it. Then, in a later iteration of the code, we want to do some kind of calculation, caching, or validation on these attributes. However, we can’t suddenly change the attribute into a method or the old client code will break. However, we can use properties to add these new manipulations and keep the original API, which is probably more readable from the calling code’s perspective, anyway.

At any rate, our OrkivRoot class needs to have one Kivy property and two Python properties in order to perform the calculations for treating the screen differently depending on its size:

(commit)

    class OrkivRoot(BoxLayout):
        mode = StringProperty("narrow")

        @property
        def chat_visible(self):
            return ChatWindow in {c.__class__ for c in self.children}

        @property
        def buddy_list_visible(self):
            return self.buddy_list in self.children

So we have a simple mode property that is exactly the same as any other Kivy property. We also have two read only python properties named buddy_list_visible and chat_visible. The former just checks if the buddy_list object is one of the children. The latter loops through the children of OrkivRoot and checks if any of them are instances of the ChatWindow class. Any time you access orkivroot.chat_visible, this method is executed.

The odd notation in the chat_visible property may need some explanation. (Note: If you’re running python 2.6 or older, it won’t even run) Python has a variety of comprehension syntaxes that allow us to create new iterator objects such as lists, generators, dictionaries, and sets based on existing iterators. This syntax is optimized for efficiency as compared to looping over one iterator and manually adding it to another. It’s also more readable if you know the syntax.

The braces, which you are probably used to denoting a dictionary, in this case actually generate a set, based on the contents of another iterator: self.children. For each element in self.children it adds the class of that element to the new set. Then the in operator is used to determine if a ChatWindow class is in that set.

In theory, we use a set instead of a list for efficiency. However, in this case, I’m actually only using it for pedagogy. The in keyword supports much faster lookups in a large set than it does in a large list. However, in this case, the number of children of a OrkivRoot will never be large. I’m just trying to drive home the point that different data structures have different strengths and weaknesses, and you need to choose them wisely. For example, a few months ago, one of my colleagues shaved nearly an hour off our loading process by converting a list to a set in this very fashion.

I hope you find my digressions educational! As I’ve said before, my goal is to not just explain how to build a Jabber application in Kivy, but to demonstrate the various thought processes behind designing your own application in whatever programming language or library you might choose.

Let’s ignore those python properties for a bit and look back at that Kivy property. While the mode StringProperty has been defined, it’s not yet being set to anything. Let’s fix that in the Kivy language file:

(commit)

    <OrkivRoot>
        mode: "narrow" if self.width < 600 else "wide"
        AccountDetailsForm:

That one new line of code is all that is needed to have a dynamically updating mode attribute. Any time the window size changes, the mode will automatically (it’s almost automagic, in fact) be set to either “narrow” or “wide” depending on what the width is. Kivy properties are very powerful. The width property of widget is automatically updated whenever the window gets resized. Then the Kivy language does a wonderful job of hooking up the event bindings such that any time that width changes, a callback that executes the line of code above is invoked.

For the record, I tested these properties by adding a couple print() statements when the show_buddy_chat method is invoked to render the current mode.

Make it work

Now we need to rework the show_buddy_chat method just a little:

(commit)

    def show_buddy_chat(self, jabber_id):
        self.clear_widgets()
        if self.mode == "wide":
            self.add_widget(self.buddy_list)
        self.add_widget(self.get_chat_window(jabber_id))
        self.buddy_list.new_messages.discard(jabber_id)
        self.buddy_list.force_list_view_update()

Instead of explicitly removing the buddy_list, we now start with a clean slate by calling clear_widgets. Then we check if the mode is “wide” and add the buddy_list if it is. Then the chat window is added just like before, but now if the view is wide, it will peacefully slide in beside the buddy_list instead of replacing it.

The code for changing the window contents when the mode changes is a bit longer. The logic is simple, it’s just verbose. We need to add an on_mode method to the OrkivRoot. This method will only be fired when the mode changes thanks, once again, to the magic of Kivy properties.

(commit)

def on_mode(self, widget, mode):
    if mode == "narrow":
        if self.chat_visible and self.buddy_list_visible:
            self.remove_widget(self.buddy_list)
    else:
        if self.chat_visible and not self.buddy_list_visible:
            self.add_widget(self.buddy_list, index=1)

When I first tested this method, it failed because I didn’t accept any arguments on the event. So I had to figure out what the expected arguments were. I did this by making the method accept *args, which says “put all the non-keyword arguments in a list named args”. Then I printed that list to see what they were. So I knew that the arguments passed were a widget object and the current value of mode. Introspection in Python is so easy!

If the mode is narrow, it checks if both widgets are currently visible and removes the buddy_list if this is the case. The only way both widgets could be visible on an on_mode event is if the mode just switched from wide to narrow.

If the mode is wide, it checks if a chat window is visible and the buddy list is not. My first attempt only checked if the buddy_list was not visible and added it arbitrarily. This failed on startup, since OrkivRoot has a size, and therefore a mode, right from startup, even when the login form is being displayed. Thus, we only do the widget juggling if a chat_window is currently visible.

The index=1 argument to add_widget tells it to insert the buddy_list before the already visible chat_window rather than after.

And that’s it! Test that, try switching back and forth between the buddy list and the chat window, try resizing the window, try switching chats while the buddy list is visible. I’m a little dubious that it’s perfect, but I can’t find a situation where these few lines of code aren’t enough to do what I want. That’s rather unfortunate, because even with all my detours and excursions, this tutorial is incredibly short! However, I’m going to publish it as is, since you’ve all been waiting patiently while I was taking vacations and wandering around at Pycon Canada (where Kivy was mentioned both in the keynote speech and in the first talk of the day). Next week, let’s try to run this sucker on Android!

Financial Feedback

If you are enjoying this tutorial series and would like to see similar work published in the future, please support me. I dream of a future where open companies like gittip and open contributions help the world run in harmony instead of in competition. I believe that my work is high quality and worth backing. If you think this article held enough value that you would have paid for it, please consider backing me in one or more of the following ways:

Creating an Application in Kivy: Part 7

I don’t know about you, but I was pretty excited at the end of Part 6 when I was able to send a message to myself on another account. It’s taken several weeks to get here, but by the end of this part, we’ll have a [mostly|barely] working jabber client that can send and receive messages and isn’t too annoying to interact with.

If you haven’t been following along or want to start this example at the same point at which I’m starting, you can clone the github repository with my examples:

git clone https://github.com/buchuki/orkiv.git
git checkout end_part_six

Remember to create and activate a virtualenv populated with the appropriate dependencies, as discussed in part 1.

Table Of Contents

If you’ve never thought about it before, let me tell you: maintaing a table of contents on WordPress is a royal pain in the ass.

  1. Part 1: Introduction to Kivy
  2. Part 2: A basic KV Language interface
  3. Part 3: Handling events
  4. Part 4: Code and interface improvements
  5. Part 5: Rendering a buddy list
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Width based layout
  9. Part 9: Deploying your Kivy application

Receiving Messages

Receiving messages with SleekXMPP isn’t quite as simple as sending them, but it’s pretty damn simple! SleekXMPP has an event handler, not unlike the one Kivy provides, but don’t confuse them! SleekXMPP is running in a separate thread and doesn’t know or care that Kivy is running it’s own event loop. Further, they use different APIs, which basically means that they use different method names to accomplish similar things, like issuing an event or listening to receive one.

Thus, in order to not be confused with Kivy Events, we don’t call our handler on_message or something in that. Instead, we add a handle_xmpp_message method to the OrkivRoot class:

(commit)

def handle_xmpp_message(self, message):
    if message['type'] not in ['normal', 'chat']:
        return
    jabber_id = message['from'].bare

    if jabber_id not in self.chat_windows:
        self.chat_windows[jabber_id] = ChatWindow(jabber_id=jabber_id)
    self.chat_windows[jabber_id].chat_log_label.text += "(%s) %s: %s\n" % (
            datetime.datetime.now().strftime("%Y-%m-%d %H:%M"),
            jabber_id,
            message['body'])

When called by SleekXMPP, this method is passed a message object, which is some kind of stanza object. I didn’t bother to fully understand how this object is constructed; instead I stole some code from the SleekXMPP quickstart and then introspected it by printing stuff to the terminal when the handler is called. It can often be useful to print(dir(message)) to see what kind of attributes and methods an unknown object has. print(type(message)) can also be useful in that knowing the name of a class can help figure out where to look in the documentation or source code.

So after a wee bit of trial and error, I came up with the above method. It first checks the type, and gets the bare attribute off the jabber_id (the documentation helped here). Then, if the chat window does not exist, we create it, the exact same way we did when the user created the object. Even if the user is not currently looking at the chat, the message will show up in it. We then append the new message to the label using the same format we used when sending messages.

Now, hook up the event handler at the moment the xmpp object is constructed in the connect_to_jabber method of the Orkiv. It just takes one new line of code at the end of the method:

(commit)

    self.xmpp.add_event_handler('message', self.root.handle_xmpp_message)

We’ll make the label look a bit cleaner shortly, but first: test it! Start chatting with your jabber friends. Tell them how you’ve written your own jabber client using Python and Kivy. If they aren’t sufficiently impressed, find new friends!

Cleaning up duplicate code

There’s a couple pieces of code in our OrkivRoot and ChatWindow classes that look very similar to each other. First, the code for creating a new chat window only if one doesn’t exist is identical. And the code for formatting a new chat message and appending it to the chat log is also congruent.

It is a good idea to remove pieces of duplicate code as soon as you notice them. Duplicate code is a “code smell”; it indicates that your design is not good. From a practical point of view, it means maintenance can be difficult if you decide to change something (say the formatting of the date on a chat message) and forget to update multiple places at once. Of course right now you remember it’s in two places, but six months from now when these two classes have been refactored into separate files, you might not remember so easily.

There are a variety of patterns to remove duplicate code, and each situation can be different. The easiest thing to do is to refactor the things that stay the same into a function and pass the things that change in as arguments. Or you might be able to use object oriented programming and inheritance. Python’s decorator syntax or context managers can also be extremely useful for reducing duplicate code. The point is, as soon as you’ve hit the copy-paste key combination, you should start thinking about the most effective API that can be designed to remove redundancy.

In this case, we can just add a couple new methods, one to OrkivRoot and one to ChatWindow. Let’s start by refactoring the code that creates a chat window only if it doesn’t exist:

(commit)

    def get_chat_window(self, jabber_id):
        if jabber_id not in self.chat_windows:
            self.chat_windows[jabber_id] = ChatWindow(jabber_id=jabber_id)
        return self.chat_windows[jabber_id]

    def show_buddy_chat(self, jabber_id):
        self.remove_widget(self.buddy_list)
        self.add_widget(self.get_chat_window(jabber_id))

    def handle_xmpp_message(self, message):
        if message['type'] not in ['normal', 'chat']:
            return
        jabber_id = message['from'].bare

        chat_window = self.get_chat_window(jabber_id)
        chat_window.chat_log_label.text += "(%s) %s: %s\n" % (
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M"),
                jabber_id,
                message['body'])

All we did was add a new method that gets the chat window from the dictionary, creating one if it doesn’t exist. Then we simplify the two methods that were previously performing this task to simply call that method. Those methods are now easier to follow. The basic principle is to abstract the complex code into it’s own function so that calling code is more readable.

Now let’s do the same thing for message formatting by adding a append_chat_message method to the ChatWindow class:

(commit)

    def send_message(self):
        app = Orkiv.get_running_app()
        app.xmpp.send_message(
            mto=self.jabber_id,
            mbody=self.send_chat_textinput.text)
        self.append_chat_message("Me:", self.send_chat_textinput.text)
        self.send_chat_textinput.text = ''

    def append_chat_message(self, sender, message):
        self.chat_log_label.text += "(%s) %s: %s\n" % (
                sender,
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M"),
                message)

We also make a similar change to the code in OrkivRoot that handles incoming text events:

def handle_xmpp_message(self, message):
    if message['type'] not in ['normal', 'chat']:
        return
    jabber_id = message['from'].bare

    chat_window = self.get_chat_window(jabber_id)
    chat_window.append_chat_message(jabber_id, message['body'])

Interface Improvements

Let’s make some improvements to that chat window. Start by modifying the method we just added to include some color to help the reader identify who sent what. The Label class has a feature that allows you to include markup on the label. Unfortunately, it has it’s own microsyntax instead of using any standard markup languages. This is good because it allows the language to target the things that Kivy labels can do, but not so great in that you have yet another markup to learn. (My life would be much easier if I was not constantly switching between ReST, Markdown, Trac wiki syntax, Mediawiki syntax, and HTML! Except for HTML, I’ve never really been able to learn or distinguish them, so I have to look up the syntax every time I write it.)

The Kivy label syntax is reminiscent of bbcode popular on web forums, so if you’ve used that, you should be comfortable. I’m not going to go into detail of the syntax, since the documentation does a great job of that, but I will show you how to use colors and bold text to make our ChatWindow label much more eye-pleasing:

(commit)

    from kivy.utils import escape_markup  # At top of file

    def send_message(self):
        app = Orkiv.get_running_app()
        app.xmpp.send_message(
            mto=self.jabber_id,
            mbody=self.send_chat_textinput.text)
        self.append_chat_message("Me", self.send_chat_textinput.text, color="aaffbb")
        self.send_chat_textinput.text = ''

    def append_chat_message(self, sender, message, color):
        self.chat_log_label.text += "[b](%s) [color=%s]%s[/color][/b]: %s\n" % (
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
                color,
                escape_markup(sender),
                escape_markup(message))

We added a color parameter to append_chat_message. It needs to be a hextet like HTML colors. Then we used the markup syntax to make the usernames colored and the dates and usernames bold. We also escape the message and sender strings so that users don’t get confused if they accidentally include Kivy formatting tags.

Remember to add the color parameter when calling this function from the handle_xmpp_message method on OrkivRoot:

    chat_window.append_chat_message(jabber_id, message['body'], color="aaaaff")

Finally, this won’t actually render as markup until we tell the label that we want markup rendered. This can be done easily by setting the markup property on the label in the KV Language file:

    Label:
        markup: True
        id: chat_log_label

Now if you run the program and chat with someone, you can easily distinguish who said what! However, if your chat gets too long, it gets truncated, both horizontally and vertically. Let’s make it scroll using a Kivy ScrollView. This can be done entirely in orkiv.kv, though it’s not trivial to get it right:

(commit)

    ScrollView:
        Label:
            size_hint_y: None
            text_size: (root.width, None)
            size: self.texture_size
            markup: True
            id: chat_log_label

In addition to wrapping the label in a scroll view, we also had to set a few sizing properties on the label. The size_hint_y is set to None or else the label would always be exactly the same size as the ScrollView. We want it to be bigger than the scroll view, or else there wouldn’t be anything to scroll!. In fact, we want it to be exactly the same size as the texture_size, which gets updated whenever lines get added to the Label. Finally, to make the lines wrap (because we don’t want to do ugly horizontal scrolling), we set the text_size to have the same width as the parent.

This can be a bit confusing, so let me explain, hopefully correctly (I’m a bit confused, myself, to be honest). First, we have a ScrollView which does not have an explicit size, so it’s size is set for us by the parent BoxLayout. The ScrollView contains a Label, which contains a Texture. The height of the label is set to be the height of the texture, which is calculated from the number of lines being displayed, and updated dynamically. Further, the width of the text rendered on the texture is constrained to be the width of the root widget, which forces the text to wrap. When the text wraps, the texture size gets bigger, and the label gets bigger because it’s bound to the texture size. and the size of the label can be as big as it wants since it’s scrollable.

These changes have the side effect of left-aligning chat, which I was expecting to have to do manually. Go Kivy! One more thing I want to do is scroll the window to the bottom if there’s any new text. This is easily done by adding a single line to the append_chat_message method:

(commit)

    def append_chat_message(self, sender, message, color):
        self.chat_log_label.text += "[b](%s) [color=%s]%s[/color][/b]: %s\n" % (
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
                color,
                escape_markup(sender),
                escape_markup(message))
        self.chat_log_label.parent.scroll_y = 0.0

The scroll_y attribute on ScrollView is basically a percentage. it returns the current scroll position if you get it, and if you set it, it will scroll the window; 0 is the bottom, 1 is the top, and anything in between is… well, anything in between.

Now I’d like to have the form send a message on an enter keypress, not just when I click the button. This has been annoying me in testing this app and annoyances should be addressed. We can use the same technique we used in part 4. However, instead of hard-coding a call when the enter key is pressed, let’s create a new reusable widget that fires an event when enter is pressed. We’ve had quite a bit of experience binding functions to events, but we haven’t yet tried creating an event of our own.

We’ll call the new widget EnterTextInput. Binding and dispatching events is actually surprisingly simple, so our new class is pretty short:

(commit)

class EnterTextInput(TextInput):
    def __init__(self, **kwargs):
        self.register_event_type("on_enter_key")
        super(EnterTextInput, self).__init__(**kwargs)

    def _keyboard_on_key_down(self, window, keycode, text, modifiers):
        if keycode[0] == 13:  # 13 is the keycode for <enter>
            self.dispatch("on_enter_key")
        else:
            super(EnterTextInput, self)._keyboard_on_key_down(
                    window, keycode, text, modifiers)


    def on_enter_key(self):
        pass

We explicitly register the new event type with the EventDispatcher, which is a superclass of Widget (and therefore, a superclass of TextInput) and provides the methods associated with dispatching events. Then we use a super() call to do the parent initialization.

Next, we override _keyboard_on_key_down, just as we did in the AccountDetailsTextInput class. If the user pressed enter, we dispatch our custom event; otherwise we just let the superclass do it’s thing.

Now we have to hook up that event. I want to do that on the ChatWindow text entry, but first, let’s remove some more code duplication. You might not have noticed, but when I created the _keyboard_on_key_down method above, I copy-pasted some code from the AccountDetailsTextInput. It turns out we can make our AccountDetailsTextInput class extend the new code to reduce code duplication. That means if we make any improvements to the EnterTextInput class in the future, the subclass will get them for free. Here’s the new version of AccountDetailsTextInput:

(commit)

    class AccountDetailsTextInput(EnterTextInput):
        next = ObjectProperty()

        def _keyboard_on_key_down(self, window, keycode, text, modifiers):
            if keycode[0] == 9:  # 9 is the keycode for <tab>
                self.next.focus = True
            else:
                super(AccountDetailsTextInput, self)._keyboard_on_key_down(
                        window, keycode, text, modifiers)

        def on_enter_key(self):
            self.parent.parent.parent.login()  # this is not future friendly

Basically, we just removed the code for dealing with the enter key and add an event handler that does the same thing when the enter key is pressed. It’s a simple refactor, and arguably isn’t terribly more readable than the original version. However, it is more maintainable. For example, if in the future we discover that an enter key can generate different keycodes in different countries, we only have to fix it in one place.

Now let’s also hook up the new event on our ChatWindow in the KV language file. Change the TextInput to a EnterTextInput and hook up the new event:

(commit)

    EnterTextInput:
        id: send_chat_textinput
        on_enter_key: root.send_message()

That’s much better! However, there’s still a bit to be desired in terms of notifications. Let’s make Kivy play a sound whenever a message comes in! I grabbed a creative commons sound (attributed to User TwistedLemon) from Free sound and saved it as orkiv/sounds/in.wav. You can also steal a sound from your favourite IM program or record something yourself. Playing the sound is as easy as loading it in the OrkivRoot initializer:

(commit)

from kivy.core.audio import SoundLoader  # At top of file

    self.in_sound = SoundLoader.load("orkiv/sounds/in.wav")

And playing it in the handle_xmpp_message method:

    self.in_sound.play()

We’re running a bit short on space for this week, but there’s one more thing I’d like to do. Right now, when we receive a notification, it’s impossible to know who sent the message, unless we currently have that chat window open. Let’s make our BuddyList highlight those users who have new messages.

The plan I have for this is not the best plan, as it doesn’t clearly separate data from display. We’re adding a new piece of data for each jabber_id (whether or not it has new messages), but instead of creating a proper data model, I’m just going to store that data in a set. Remember, the args_converter is drawing data from the xmpp presence and other locations, and the attribute of whether something is selected is stored in the state of the BuddyListItem, so in some ways, the args_converter IS a data model.

It’s not elegant. However, making it elegant requires both that the ListView API be properly designed, and that we stop abusing it to take only the parts we want. So let’s go with simple for now!

Start by adding an attribute to the BuddyList.__init__ method. It will be a set object, and will contain only jabber_ids that have new messages:

(commit)

    def __init__(self):
        super(BuddyList, self).__init__()
        self.app = Orkiv.get_running_app()
        self.list_view.adapter.data = sorted(self.app.xmpp.client_roster.keys())
        self.new_messages = set()

We use a set instead of a list because sets are more efficient when we want to ask the question “is this object in the set”. We aren’t going to be doing that millions of times and the size of the set isn’t going to be tens of thousands of objects, so it’s not a really important distinction for this code. However, it’s good to get in the habit of using and knowing which data structures to use for which tasks. At some point in your programming career, you will want to study up on “Algorithms Analysis” and “Data Structures” to help you make these decisions. For now, just remember that any time you are keeping a collection of objects so you can repeatedly use the in keyword on it, use a set. Any time you need to know the order of objects, use a list.

Before we start adding messages to this set, let’s create a method on BuddyList that forces the listview to redraw itself. This method shouldn’t really be required if we were using the ListView API properly and it was working properly, but we aren’t and it’s not:

(commit)

    def force_list_view_update(self):
        self.list_view.adapter.update_for_new_data()
        self.list_view._trigger_reset_populate()

Now we need to add users to the set whenever we get a new message in handle_xmpp_message, telling the BuddyList to redraw itself as we do so:

(commit)

    if chat_window not in self.children:
        self.buddy_list.new_messages.add(jabber_id)
        self.buddy_list.force_list_view_update()

We only add the “new message” notification if the chat window is not the currently displayed one. If it is, then the new message shows up on the screen immediately, so it’s not unread.

Similarly, when the user selects a new buddy from the buddy list, we need to clear the new message status for that user:

    def show_buddy_chat(self, jabber_id):
        self.remove_widget(self.buddy_list)
        self.add_widget(self.get_chat_window(jabber_id))
        self.buddy_list.new_messages.discard(jabber_id)
        self.buddy_list.force_list_view_update()

The discard method, unlike remove does not raise an exception if the item is not in the set.

Of course, running this code won’t actually highlight anything. Knowing that the jabber_id has a new message doesn’t help us unless we do something with that knowledge. Let’s replace the background color if statement in BuddyList.roster_converter with the following:

(commit)

   if jabberid in self.new_messages:
       result['background_color'] = (0.6, 0.4, 0.6, 1)
   else:
       result['background_color'] = (0, 0, 0, 1)

   if index % 2:
       result['background_color'] = (x + .3 for x in result['background_color']) 

Basically, if it’s a new message, we give it a new color. But then, to maintain the alternating colors of list items, we add 0.3 to the colors in every other row. Overall, the interface is working pretty well.

Next time, we’ll see if we can get it to render the chat window beside the buddy list, if the window is wide enough! However, please note that it’s going to be a couple weeks before I can publish that article; I’m on vacation this weekend and the following weekend I’ll be at Pycon Canada (look me up if you’re there!).

Monetary feedback

While, in previous parts of this tutorial, I’ve been soliciting funds to support my continued involvement in writing and open source projects, this week, I’m going to ask you to fund Mathieu Virbel instead. Mathieu is the inventor and (utterly brilliant) lead developer of Kivy. The more time he can spend developing Kivy, the better it is for all of us (including me)!

Of course, you’re also welcome to tip me instead, or any of the other wonderful members of the Kivy team.

Or just promote Kivy and my books and articles on your favourite social media platforms. Spread the word, it’s always appreciated!

Creating an Application in Kivy: Part 6

Welcome back, or if you’ve just stumbled across this tutorial, welcome to the middle of something! You might want to jump back to the beginning if that’s the case. Don’t worry, we’ll wait for you to catch up.

We’re learning Kivy by developing a simple chat application together on the XMPP protocol. So far we’re able to display a Buddy List that doesn’t look too shabby, but we haven’t even tried chatting with anyone yet. Given the complexities of the sleekxmpp library, I’m only hoping that it won’t be too difficult!

If you want to start this part with a clean slate, in exactly the state that I’m starting it, you can clone my public repository:

git clone https://github.com/buchuki/orkiv.git
git checkout end_part_five

Remember to create and activate a virtualenv populated with the appropriate dependencies, as discussed in part 1.

Table Of Contents

Here’s some lovely navigation for you:

  1. Part 1: Introduction to Kivy
  2. Part 2: A basic KV Language interface
  3. Part 3: Handling events
  4. Part 4: Code and interface improvements
  5. Part 5: Rendering a buddy list
  6. Part 6: ListView Interaction
  7. Part 7: Receiving messages and interface fixes
  8. Part 8: Width based layout
  9. Part 9: Deploying your Kivy application

About The Current Kivy ListView API

I’m afraid this article’s going to be going up a little late because I’ve spent a great deal of time trying to understand the ListView API for interacting with the user. I’ve read through all the documentation, I’ve read through all the examples included with Kivy, and I’ve read through the source code for all the views, adapters, and widgets. I’ve come to the conclusion that, unlike the elegance in most of Kivy, the ListView API simply sucks. I think my favorite aspect of the documentation was the statement, “Warning: This code is still experimental, and its API is subject to change in a future version.”

I did discover that Kivy’s ListView API provides all the tools to do what we want, but it’s mixed in with a whole bunch of tools that don’t work at all, don’t work the way you would expect, or are far more complicated than necessary. Figuring out which of those tools to use and how to use them was a perfect chance to practice skills for dealing with frustration.

The good news is that when I complained about the ListView API on the #kivy IRC channel I was told that the developers are already actively working on improving it!

I have tried, in these tutorials, to explain not only how to do things, but also how to figure out how to do things. I’m not going to do that for this part for a few reasons. First, there is no clear explanation on the web for “how to effectively use Kivy’s ListView API”, so I’d like this tutorial to fill that niche. Second, with the API redesign in the works, I don’t want to spend a lot of time writing outdated content. Third, the approach I used to understand the API was basic brute force, and not educational in any way. Read all the documentation, read all the sources, and write tons of test code. By trial and error and a bit of good judgment, I think you’ll come to the same conclusions I did. Finally, I spent so much time doing that brute force work that I can’t easily summarize the steps I followed or all the paths I explored.

How to Ignore Most of The Bad Bits

I’m sure you’re aching to see some code, but let’s discuss exactly what we want the interface to look like, first. While our BuddyListItem widget contains several other widgets, we want it to highlight the entire widget with a different background color when it is touched. Essentially, even though multiple items are being displayed on the widget, we want the whole widget to behave like a button. Later, we’ll want to pop up a chat window when the selection event happens.

That “behave like a button” line is the key. The Button class inherits from label and adds some things for button state and touch events. Then Kivy extends that with a ListItemButton class that manages the state of multiple buttons in a ListView. As the Kivy examples show, using this class as the cls argument to a ListView works flawlessly. You touch a button, it gets highlighted. Unfortunately, adding widgets to a button doesn’t work so well because Button is not a layout. If only we could make our BuddyListItem a button AND a layout. Hmmm:

(commit)

from kivy.uix.listview import ListItemButton  # At top of file

class BuddyListItem(BoxLayout, ListItemButton):
    jabberid = StringProperty()
    full_name = StringProperty()
    status_message = StringProperty()
    online_status = StringProperty()
    background = ObjectProperty()

This is multiple inheritance; BuddyListItem is inheriting from both the BoxLayout and ListItemButton superclasses. I honestly did not expect this to work with widgets, since the inheritance graph is a weird diamond. However, the Kivy devs have been careful to support multiple inheritance (by always passing **kwargs to super calls). When I ran this code and saw selectable buttons in my ListView, I said to myself, “Holy shit! It works!” Then I giggled softly and promptly forgave the Kivy team for the confusing API faux pas.

Now we get to do one of my favorite activities: Deleting code. One of the nice things we inherit from the Button class is a background_color property. This means we can take the entire canvas.before section out of orkiv.kv:

(commit)

<BuddyListItem>:
    size_hint_y: None
    height: "75dp"

    BoxLayout:
        size_hint_x: 3
        orientation: 'vertical'
        Label:
            text: root.jabberid
            font_size: "25dp"
            size_hint_y: 0.7
        Label:
            text: root.status_message
            color: (0.5, 0.5, 0.5, 1.0)
            font_size: "15dp"
            size_hint_y: 0.3
    Label:
        text: root.full_name
    Image:
        source: "orkiv/icons/" + root.online_status + ".png"

Nothing new here at all, it’s just shorter! You can also remove the background property from __main__.py. Now, just change the arg_converter to set background_color instead of background and the button will look less buttony and more listitemy:

(commit)

if index % 2:
    result['background_color'] = (0, 0, 0, 1)
else:
    result['background_color'] = (0.05, 0.05, 0.07, 1)

Finally, I’m not too keen on the default red color for the selected item, so let’s change the selected_color (this is a property available on ListItemButton) in the Kivy file:

(commit)

<BuddyListItem>:
    size_hint_y: None
    height: "75dp"
    selected_color: (0.2, 0.1, 0.2, 1.0)

Run this code and you should be able to touch any list item and see it highlighted. Now that’s what I call an easy way to display selected items on a ListView! Don’t take these as gospel, since your needs may differ, but here’s some ideas that I came up with while puzzling through the documentation and code for this API:

  • Don’t use DictAdapter. As far as I can tell, you can more easily use ListAdapter on the keys and reference the underlying dictionary directly from args_coverter, rather than trying to maintain the sorted_keys parameter.
  • As I mentioned in Part 5 don’t use the template argument to ListAdapter, as that functionality has been deprecated.
  • Pretend that CompositeListItem does not exist, never did exist, and could not possibly exist.
  • ListItemLabel is not interactive and is for display only, not selection.
  • While the documentation suggests that you can create your own item class, if you need selection, use ListItemButton. You can either extend it and add widgets as we did here, or put individual buttons inside another view, depending on your requirements.
  • If you don't use ListItemButton and you need selection, you need to make sure your object provides a on_release event, since that is what ListView is hardcoded to trigger selection on.
  • propagate_selection_to_data and SelectableDataItem add a lot of complexity and are only worth it if you really know they are worth it. Don't "avoid them at all costs", but make sure you really need them before trying.

Designing a Chat window

I have an exercise for you that I don't think you'll have any trouble with if you've followed the previous parts of this series. Of course, I'll provide examples and explanations shortly if you get stuck.

Here's the exercise: Create an empty new ChatWindow class that extends BoxLayout. Style the class in the KV language so that it contains a vertical box layout holding a label identifying the user you are chatting with, a second label containing the text of the conversation, and a horizontal box layout containing a Button to return to the buddy list, a TextInput and a second Button to send chat text. Try to figure out which of these widgets need to have an id in the KV Language file hooked up to an ObjectProperty in the python file. Finally, ensure the widgets have appropriate size and size_hint properties to look reasonable.

Here's how my ChatWindow code turned out:

(commit)

class ChatWindow(BoxLayout):
    jabber_id = StringProperty()
    chat_log_label = ObjectProperty()
    send_chat_textinput = ObjectProperty()    

And here's my first attempt at styling. It's a compliment to Kivy's simplicity and elegance that when I actually hooked up the events to display this code, it looked exactly like I expected it to:

(commit)

    <ChatWindow>:
        orientation: "vertical"
        chat_log_label: chat_log_label
        send_chat_textinput: send_chat_textinput
        Label:
            text: "chatting with " + root.jabber_id
            size_hint_y: None
            height: "40dp"
        Label:
            id: chat_log_label
        BoxLayout:
            size_hint_y: None
            height: "50dp"
            Button:
                size_hint_x: None
                width: "70dp"
                text: "Buddies"
            TextInput:
                id: send_chat_textinput
            Button:
                size_hint_x: None
                width: "70dp"
                text: "Send"

Now we need an event to display a ChatWindow when a user makes a selection. Kivy's ListView API has the ability to propagate selection events to the adapter and even the model behind the adapter. However, it's rather involved and I suggest avoiding it unless you actually have a reason to store the advanced selection stuff. Unfortunately, this means you can't listen to the adapter's on_selection_change event.

However, for our use case, all we really want to do is show the chat window if the user touches an item in the list. Hence, we just need to listen for a touch event on the BuddyListItem (which extends Button and therefore has all of Button's events) and react to that.

The reaction will be to display a new ChatWindow object on the root. This means adding a new method to OrkivRoot:

(commit)

def show_buddy_chat(self, jabber_id):
    self.remove_widget(self.buddy_list)
    chat_window = ChatWindow(jabber_id=jabber_id)
    self.add_widget(chat_window)

When called, this code will simply clear the window and display a new, empty chat window in it's place. One thing I may not have mentioned before is how ChatWindow accepts jabber_id as a keyword argument, even though there is no __init__ method that sets the argument. This is because Kivy automatically maps keyword arguments to Property objects on the class or any parent classes, whether we defined those properties in our code or they were defined by the Kivy widget itself.

Now all we have to do is hook up the on_release event on the BuddyListItem in the KV Language file. Note that if you are doing more complicated selection, you may want to hook up an on_state handler instead. It will be called whenever the state property changes (for selection or deselection).

While we're at it, we can also add a similar event to render the buddy list again when the Buddies button is clicked inside the ChatWindow:

(commit)

<BuddyListItem>:
    size_hint_y: None
    height: "75dp"
    selected_color: (0.2, 0.1, 0.2, 1.0)
    on_release: app.root.show_buddy_chat(self.jabberid)

and

    Button:
        size_hint_x: None
        width: "70dp"
        text: "Buddies"
        on_release: app.root.show_buddy_list()

Now, if we run the app, we can switch between the buddy list and a chat view whenever we want. This will be useful for a phone-sized layout, and in a later tutorial we'll make it possible to switch to a side-by side view for tablets and desktops.

However, this isn't an optimal code design. First, it's wasteful of resources; each time, we go back to the Buddy List view, the list is constructed afresh, from scratch. Worse, new chat windows are also constructed each time we go back to a jabberid, meaning that any previous conversation would be lost. Let's fix both these things in the OrkivRoot class:

(commit)

class OrkivRoot(BoxLayout):
    def __init__(self):
        super(OrkivRoot, self).__init__()
        self.buddy_list = None
        self.chat_windows = {}

    def show_buddy_list(self):
        self.clear_widgets()
        if not self.buddy_list:
            self.buddy_list = BuddyList()
        for buddy_list_item in self.buddy_list.list_view.adapter.selection:
            buddy_list_item.deselect()
        self.add_widget(self.buddy_list)

    def show_buddy_chat(self, jabber_id):
        self.remove_widget(self.buddy_list)
        if jabber_id not in self.chat_windows:
            self.chat_windows[jabber_id] = ChatWindow(jabber_id=jabber_id)
        self.add_widget(self.chat_windows[jabber_id])

We added an initializer that sets a couple variables for us. In the show_buddy_list method, we now instantiate the BuddyList only if it has not been previously created. However, this means that selection is kept between calls, so we have to loop over the selected items and deselect them.

This might need some explaining. The ListView doesn't know anything about selection; that is kept track of in the ListAdapter. The selection property is a list containing all BuddyListItem objects that are currently selected. Because BuddyListItem is a ListItemButton, which provides the SelectableView, interface, it has a deselect method that we can use to disable selection of that item.

The chat_windows dictionary maps jabber_id keys to individual ChatWindow objects, any of which can be displayed when the associated user is touched in the buddy list.

Sending a Message

Ok, now that our chat text box is looking pretty sweet and we can switch between buddies at will, let's try actually sending a message! Add a new method, send_message to the ChatWindow widget in __main__.py:

(commit)

    import datetime  # At top of file

    def send_message(self):
        app = Orkiv.get_running_app()
        app.xmpp.send_message(
            mto=self.jabber_id,
            mbody=self.send_chat_textinput.text)
        self.chat_log_label.text += "(%s) Me: %s\n" % (
                datetime.datetime.now().strftime("%Y-%m-%d %H:%M"),
                self.send_chat_textinput.text)
        self.send_chat_textinput.text = ''

This is actually fairly simple code considering how exciting it is to send a message and have it show up on a different account when we run it. First we get the app and instruct the xmpp object to send a message. I got this code from the SleekXMPP documentation. Then, while that message is finding it's way to our other chat client, we update the chat_log_label text so that it contains a new line with the current date and the text we sent. Finally, we update the send_chat_textinput to be empty so we can send another value right away.

Now we just need to invoke this method whenever the send button is clicked. I'm sure you can figure out how to do that by yourself. It's just one line of code in the Kv Language file:

(commit)

    Button:
        size_hint_x: None
        width: "70dp"
        text: "Send"
        on_release: root.send_message()

Now for the fun part. If you have two different jabber accounts, connect your normal instant messenger to one of them, and then log into Orkiv with the other one. Send a message to the "normal" messenger and giggle manically when it arrives. This is why we program: the success is so tangible!

And I'll leave you with that success for today. In the next part, we'll hook up the message receiving side of things and make several interface improvements to make the app much more usable.

Monetary feedback

In previous parts of this tutorial, I've asked users to support me financially if they'd like to see more of it. I'm trying to gauge whether it is feasible for me to consider switching from full time work to being supported entirely by open source development, writing, and education in the (far) future.

Instead, today, I'd like to take some time to thank those of you who have answered my call. It's impossible to tell how many book sales come from readers if these essays, or how many Gittips were prompted by this work rather than my various other open source projects, but as both have increased in the weeks that I've been publishing this series, it's clear that some of you support my work. I would like to thank you from the bottom of my heart. I truly believe that an economy based on paying what the buyer, rather than the seller, thinks something is worth, will one day lead to a happier, more peaceful world.

I'd also like to thank all of you who have tweeted, commented, blogged, or otherwise spread the word about these articles. I think they are one of the best available resources for new Kivy developers outside of the excellent Kivy documentation itself. I am exhilarated to know that so many of you agree!