Pyjaco in a real app: Todos with local storage
I didn’t get the memo, but there appears to be a movement to demonstrate emerging web technologies with a simple todo list application, much as hello world is used to introduce programming languages.
In my last post, I introduced using jQuery with Pyjaco, the PYthon JAvascript COmpiler. Since then, I’ve made several contributions to the project and have been involved in diverse discussions with Pyjaco developers regarding the current and future status of the project. This post goes further by acting as a tutorial for writing a basic todos app using Pyjaco.
Pyjaco is alpha software. It is hard to write valid code, and harder to debug. I’ve managed to both lock up Firefox and hard crash it while using the Pyjaco library.
On the positive side, Pyjaco is under active, rapid development. The head developer, Christian Iversen is extremely responsive to both questions about Pyjaco, and to code contributions. This is a project with a lot of potential, and I see it as the current best bet for Python programmers hoping to avoid javascript one day in the future.
In spite of the hiccups, it is possible to generate a working javascript app using just Pyjaco. Here’s how.
Let’s start:
mkdir pyjados cd pyjados virtualenv2 venv --distribute --no-site-packages source venv/bin/activate pip install git+https://buchuki@github.com/buchuki/pyjaco.git@run_script
First we create a directory to work in and install a virtualenv. Pyjaco does not currently work with python 3, so in Arch Linux, I use the virtualenv2 command. We then activate the virtualenv and install the pyjaco package. Here I am installing from my personal fork, as it contains some changes for generating the built-in standard library that have not yet been merged upstream. You should normally install directly from chrivers’s git repository using pip install git+git://github.com/chrivers/pyjaco.git.
Now let’s create a basic HTML 5 page with jQuery loaded:
<!DOCTYPE html>
<html>
<head>
<title>PyJaco Todo List Example</title>
<script type="text/javascript" src="http://code.jquery.com/jquery-1.6.4.min.js"></script>
</head>
<body>
<h1>PyJaco Todo List Example</h1>
</body>
</html>
We can load this in our web browser using a file:// URL. This is the only HTML page in our app, and it can be refreshed to load our changes as we work.
Pyjaco doesn’t simply translate Python into Javascript. Rather, it creates a basic standard library of Python-like objects that are utilized in the compiled javascript code. I wasn’t too keen on this idea when I first heard it, as it introduces a dependency that currently weighs in at 65K before minification and compression. While this is not a terribly heavy library, there are efforts under way to shrink the builtins or to dynamically generate it to contain only those builtins that your code actually touches. At any rate, we need to ensure this library is available to our code. First we generate the library:
pyjs.py --builtins=generate --output=py-builtins.js
pyjs.py is the name of the pyjaco command. It is expected to be renamed to pyjaco in the future. The --builtins=generate option tells pyjaco to generate the standard library, while the --output flag provides the filename for the new library file:
$ ls
index.html py-builtins.js venvWe then need to load this library in the head of our html file. Let’s also load the future pyjados.js script at this time:
<head>
<title>PyJaco Todo List Example</title>
<script type="text/javascript" src="http://code.jquery.com/jquery-1.6.4.min.js"></script>
<script type="text/javascript" src="py-builtins.js"></script>
<script type="text/javascript" src="pyjados.js"></script>
</head>
Now, before we start coding the Python file that will be compiled to Javascript, I want to discuss what I consider to be the most confusing aspect of Pyjaco development. There are basically two types of variables in Pyjaco, Javascript variables, and Python variables. Javascript variables refer to “normal” variables that you would call in Javascript. These include alert, window, document and the like, as well as variables in third-party Javascript libraries, such as the ubiquitous jQuery. Further, any attributes on those objects are also Javascript variables, and the return value of any methods will also be Javascript variables.
Python variables, on the other hand, refer to any variables that you define in your Python source code. If you create a dict or a list, for example, it will be compiled to a list or dict object from the standard library we just generated. In the compiled script, of course these Python variables are represented by Javascript objects, but from the point of view of a Pyjaco coder, it is important to keep the two types of files separate. Almost all the bugs I have encountered in my Pyjaco code have been caused by confusing the two types of variables.
The distinction between python and javascript variables introduces a couple of complications to writing Pyjaco compatible python code. First we need to flag all of our Javascript variables using a decorator on methods that access them. Second, we need to explicitly convert our variables between Javascript and Python any time we access one from the other. I’m told that this conversion can — and one day will — be done automatically by the pyjaco multiplexer, but in the meantime, we need to make it explicit. We do this by using two javascript functions supplied with the standard library we just generated, appropriately named js() and py(). You will see examples of these shortly.
When I finally figured out the distinction, my first thought was, “ok, let’s prefer to always work with python variables.” Therefore, in my initialization code, I tried jQ=py(jQuery). Unfortunately, jQuery is a rather large object, and the py function apparently recursively converts all attributes from javascript to python. I ended up with a stack overflow.
Now, let’s create our first python code and watch it compile to Javascript. Name the file pyjados.py:
def setup(): print "Pyjados Hello World" jQuery(js(setup));
First we write a python function named setup. This function is a python object. jQuery is a javascript object that expects a javascript object as input. Therefore, we wrap setup in a js() call and pass the result into the jQuery function. jQuery will now run setup when document.ready is fired.
Now we compile the code using the following command inside our activated virtualenv:
pyjs.py --watch pyjados.py --output pyjados.js
You’ll notice the command doesn’t exit. That is the --watch option at work. If you now make a change to pyjados.py and save it, it will automatically recompile it. The output file pyjados.js is regenerated each time. This is the file we included in our html file. So now, open that html file in a web browser using a file:// url. Make sure the Javascript console is displayed and reload the page. You should see the words “Pyjados Hello World” printed on the console. Pyjaco automatically compiles print statements into console.log output.
Before we start implementing our Todo list, let’s look at an example of accessing a javascript variable inside a function. Change setup.py to utilize alert, as follows:
@JSVar("alert") def setup(): alert(js("Pyjados Hello Alert") jQuery(js(setup));
Did you look closely at that code? There is a missing close bracket on the alert line. You’ll see the syntax error in your console where pyjs.py is watching the compiled code. Add the bracket and let it automatically recompile itself:
@JSVar("alert") def setup(): alert(js("Pyjados Hello Alert")) jQuery(js(setup));
Let’s analyze this snippet. First, notice how we told the compiler that alert is a Javascript variable when used inside setup(). This is a bit odd, since the JSVar decorator is never actually imported into the namespace. This is a bit of magic in the Pyjaco compiler, just pretend it has been imported.
Second, notice that since alert has been flagged as a JSVar, it must accept a Javascript variable. However, the string “Pyjados Hello Alert” is a Python variable. Therefore, we convert it using js() as we pass it into the alert call.
Now let’s prepare to create some working todo-list code. Start by adding a form for submitting todos and a list to render the todos to the html body:
<body>
<h1>PyJaco Todo List Example</h1>
<form id="add_todo_form">
<input type="text" id="add_box" placeholder="Add Todo", autofocus="autofocus">
<button id="add_button">Add Todo</button>
</form>
<ul id="todo_items"></ul>
</body>
Nothing too exciting here. Note the ids on the elements, since we’ll be utilizing these from Pyjaco using jQuery selectors.
Now back into the python file. Let’s create a class to manage the various todo elements:
class TodosApp: @JSVar("jQuery", "js_add_form") def __init__(self): js_add_form = jQuery(js("#add_todo_form")) js_add_form.submit(js(self.add_todo)) def add_todo(self, event): print "form submitted" return js(False) def setup(): todo_app = TodosApp() jQuery(js(setup));
The __init__ function hooks up the form’s submit button to a method on the object. Notice that we need to flag not just jQuery, but also js_add_form as a javascript variable. Pyjaco does not (currently) know that a javascript variable is returned when calling a method on an existing javascript variable. I like to add the js_ prefix to variable names to help remind myself that this is a javascript variable.
In an ideal world, we could convert this variable to a Python variable using py(), but as noted earlier, calling py on a jQuery object results in a stack overflow or browser crash.
Also pay attention to the way we wrap the self.add_todo method name in a js() call when we pass it into the submit handler. The submit method is a javascript function expecting a javascript object.
The def add_todo method has its single parameter flagged as a @JSVar, since the method is being called internally by jQuery when the event occurs. We also wrap the False return value (to prevent event propogation on the submit handler) in a js() call so that jQuery recognizes it as a javascript false rather than a (true) object named False.
Try the code. Ensure the compiler recompiled it, and reload the html file. Enter some characters into the text box and use the Enter key or the Add Todo button to submit the form. The words form submitted should be displayed in the javascript console.
Now let’s actually store and render a newly added todo. The todos are stored in memory in a python dict object. Initialize this object by adding the following two lines of code to the end of __init__:
self.todos = {} self.next_id = 1
And rewrite add_todo as follows as well as a new method named render
@JSVar("event", "js_add_box") def add_todo(self, js_event): js_add_box = jQuery(js("#add_box")) self.todos[self.next_id] = py(js_add_box.val()) js_add_box.val('') js_add_box.focus() self.next_id += 1 self.render() return js(False) @JSVar("js_todo_items") def render(self): js_todo_items = jQuery(js("#todo_items")) js_todo_items.html("") for id, todo in sorted(self.todos.items()): js_todo_items.append(js('<li>%s</li>' % (id, todo)))
Note that the todos dict is a Python object, so when we insert the value of the js_add_box into it, we must convert it from a javascript object using py(). Also note how, because we are writing in a python function, manipulating the python value self.next_id requires no conversion, and calling the python function self.render is also clean.
In the render function itself, I think it’s pretty cool that string formatting using % is supported by pyjaco (as an aside, the str.format method introduced in python 2.6 is not yet available) and that the python sorted() function is available. Note also how we can loop over items() on the self.todos dictionary just as if we were using a normal python dictionary.
Now let’s add the ability to complete todos. Let’s start by adding a template string as a class variable, and use that string inside the render function. This illustrates that pyjaco supports class variables:
class TodosApp: list_item_template = """<li> %(text)s </li>"""
and we change the for loop in render to:
for id, todo in sorted(self.todos.items()): js_todo_items.append(js(TodosApp.list_item_template % { "id": id, "text": todo}))
Reload the page again and notice how checkboxes have been displayed beside each todo. The next step is to make clicking these boxes actually complete the todos. We add a couple lines to our __init__ method to connect a live click event to the checkbox items, which now looks like this:
@JSVar("jQuery", "js_add_form", "js_checkbox") def __init__(self): js_add_form = jQuery(js("#add_todo_form")) js_add_form.submit(js(self.add_todo)) js_checkbox = jQuery(js("input[type=checkbox]")) js_checkbox.live("click", js(self.complete_todo)) self.todos = {} self.next_id = 1
Don’t forget to add js_checkbox to the JSVar decorator.
The complete_todo method looks like this:
@JSVar("event", "jQuery", "todo_item") def complete_todo(self, event): todo_item = jQuery(event.target).parent() id = int(py(todo_item.attr("id"))[5:]) del self.todos[id] todo_item.delay(1500).fadeOut("slow")
The first line is using exclusively javascript arguments, and returns the <li> element containing the checkbox that was clicked. The id = line converts the javascript string id attribute of this element (which looks like “todo_5“, as defined in list_item_template) into the python integer id of the todo. The remaining lines simply remove that todo from the internal list and from the DOM, after a 1.5 second delay.
In fact, we now have a fully functional todo list that allows adding todos and checking them off. Now, as a bonus, let’s try hooking this up to the HTML 5 localStorage object so that the list is maintained across page reloads. We start by adding a store() method to our class:
@JSVar("localStorage", "JSON") def store(self): localStorage.setItem("todolist", JSON.stringify(js(self.todos)))
The main line of code is easiest to read from the inside out. First we convert the self.todos dict to a normal javascript object using the js() function. Then we call JSON.stringify on this object to create a string suitable for insertion into localStorage.
Now add this call to the end of the two methods that manipulate the todo list, add_todo and complete_todo:
self.store()
.
Refresh the page, add a couple todos, and inspect the localStorage object in your console. You should see the stringified dict in the todolist value.
Now all we have to do is ensure the self.todos dict is loaded from localStorage when the app is initialized. Add the following to the end of the __init__ method (make sure to add js_stored_todos to the JSVars decorator):
js_stored_todos = localStorage.getItem("todolist") if js_stored_todos: stored_dict = dict(py(JSON.parse(js_stored_todos))) self.todos = dict([(int(i), stored_dict[i]) for i in stored_dict.keys()]) self.next_id = max(self.todos.keys()) + 1 self.render()
Note that calling py() on the output of JSON.parse creates a python object, not a python dict. The code is therefore wrapped in a call to dict(), which converts the object to a dictionary.
Unfortunately, the resultant dict contains keys that are strings, whereas our original dict used integer keys. So a pure-python list comprehension is used to convert the dictionary to one with integer keys. This line is a bit hard to read, but I wanted to include it to demonstrate that Pyjaco can parse list comprehensions. Finally, we set self.next_id using the python max() call, which Pyjaco also automatically translates into javascript.
Try it out. Load the pyjados HTML file, add some todos, check a few of them off, then close and reload the web browser. Your todos will be stored!
I hope you’ve enjoyed this introduction to Pyjaco. It is a nice tool with a lot of potential. Currently, I find writing Pyjaco code to be approximately equally tedious to writing Javascript code. However, I feel that as I learn the ins and outs of Pyjaco, and as the developers continue to refine and improve the compiler, Pyjaco may one day be a perfectly viable alternative to writing pure Javascript or to the rather too Ruby-esque, but otherwise excellent Coffeescript.