Posts tagged ‘web development’

Google Wave Bugtracker… concept

Last night I received access to the Google Wave Developer Sandbox. My original request for access was made when I had a lot more free time than I do now (I’m measuring free time on a negative scale now: I don’t expect to have any until May). My plan at the time was to integrate Google Wave with bugtracking software (written from scratch). I haven’t actually looked at the wave APIs yet, but I had considered all three methods of extending Wave for use with a bug tracker:

  • Embedding: An embedded wave on an external bug tracking webapp would make discussion much simpler and more robust. Most bug trackers don’t even have threaded comments right now. The ability of a wave to allow synchronous or asynchronous discussion would make the discussion system much more useful (most discussion on bugs currently happens “out of band” on mailing lists or in chats). In addition, a “subscription” feature could be as simple as adding yourself to the wave. Finally, the ability to edit wave postings (wiki style) would allow the data in the bug report to be easily organized and classified. People wouldn’t have to read entire lists of comments to get an overview.
  • Robots: At the simplest form, adding a robot to an existing wave would create a new bug report based on that wave. It would be nice if you could also have the robot automatically set bug attributes such as project, version, milestone, architecture, etc (whatever the project demands).
  • Extensions: An extension to wave could allow bugs to be integrated into other waves, instead of just waves into bugs. Instead of being stuck on a fixed bug report server somewhere, the bugs could be sent to other wave users. That could even include upstream authors. This is the most interesting prospect, because it could lead to a “global bug tracking service” where bugs are not attached to a single reporting tool, but are easily routed to the people most able to fix them.

In Arch Linux, many, but not all, of our bugs are related to specific packages. It would be nice if bug reports could be attached to packages in our web interface. Then it would be easy for users to report bugs on specific packages, and for developers to find all bugs related to their packages.

Our team includes a few dedicated (and extremely under-appreciated) users who classify incoming bugs and assign them to the developers who ought to be responsible for fixing them. I imagine a wave scenario where this can be done as easily as forwarding all new bugs to these “bug wranglers” wave accounts, and allowing them to add other developers to the wave and have them automatically assigned.

Ideally, we would host the wave server ourselves, so as to be in complete control of our information. Of course, if someone else wants to add themselves to a wave from a different service, that should not pose a problem.

Unfortunately, I suspect that when I have time to complete this project, I’ll have been swept up in some new and exciting venture. But it’s certainly an interesting tool to think about.

Authenticated Threaded Comments in Django 1.1

I’m looking to include threaded comments in my latest project, such that only authenticated users can use them. The Django ThreadedComments project seems to have lost momentum; as the old site says that the new version will use the comment hooks provided in Django 1.1 and will be backwards incompatible, but very little fresh development has been done.

Turns out, almost all the required functionality is available for free in the standard Django comments system, and you only need one slightly messy hack.

Authenticated Comments

I want a system that doesn’t ask for name/email/url; you can only post if you’re logged in, and the only visible box is one for comments. Behind the scenes, django automatically fills in these details from the currently logged in user if they are not supplied. It also kindly attaches that user to the comment (so it can be retrieved as comment.user). All you have to do is restrict commenting to logged in users and render the template directly to ignore the fields you don’t need. Like this:

some_template.html

{% if user.is_authenticated %}
    <h4>Submit New Comment:</h4>
    {% get_comment_form for project as comment_form %}
    {% include 'comment_form.html' %}
{% endif %}

comment_form.html

    {% load comments %}
    <form action="{% comment_form_target %}" method="POST">
        {{comment_form.content_type.errors}}{{comment_form.content_type}}
        {{comment_form.object_pk.errors}}{{comment_form.object_pk}}
        {{comment_form.timestamp.errors}}{{comment_form.timestamp}}
        {{comment_form.security_hash.errors}}{{comment_form.security_hash}}
        {{comment_form.comment.errors}}{{comment_form.comment}}
        <span>{{comment_form.honeypot}}</span>
        <br />
    </form>

Most of the variables in comment_form.html are hidden fields to help keep spammers at bay. If you prefer, you can do this same customization a bit higher in the abstraction layer by using a Custom Comment Form that doesn’t contain the url, e-mail and name fields at all. The django.contrib.comments.forms.CommentSecurityForm holds everything you need except a honeypot.

Threaded Comments

Django comments can be attached to any object… including existing comments. That idea and some thoughtful template design provides threaded comments:

some_template.html

{% get_comment_list for project as comments %}
{% for comment in comments %}
    {% include 'comment.html' %}
{% endfor %}

Here I’m getting all the comments attached to a specific “project”. For non-threaded comments, comment.html would basically just contain {{comment}} to render the comment. But I want threaded comments:

comment.html

{% load comments %}
<div class="comment">
    {{comment.user}}
    {{comment.comment}}
    {% get_comment_list for comment as comments %}
    {% with 'comment.html' as comment_include %}
        {% for comment in comments %}
            {% include comment_include %}
        {% endfor %}
        {% if user.is_authenticated %}
        <br /><a href="#">Reply to This Comment</a>
            {% get_comment_form for comment as comment_form %}
            {% include 'comment_form.html' %}
        {% endif %}
    {% endwith %}
</div>

Here, for each comment, I get the list of comments attached to that comment, and render them, along with a box to reply to the comment. This box is hidden by default in my stylesheet and displayed with a bit of jquery. I recursively include the same template for each comment so you can have as many levels of comments as you like (that may be a bad thing).

The recursive include is the messy hack — by default, Django includes templates at parse time. This means the template is included regardless of whether any comments exist, and the Python stack quickly fills up with too many recursive calls. But if the template being included is a variable instead of a string, it has to know what that variable value is and can thus only be executed at render time. So I simply wrap the include with a with template tag. This forces django to use the IncludeNode instead of ConstantIncludeNode (see django/templates/loader_tags.py), which renders the template at render instead of parse time.

Some uncreative styling in my css makes the comments indent a bit and hides the reply box until the jquery link is clicked.

.comment {
    margin-left: 1em;
    border-style: solid;
    border-width: 1px;
    border-color: #aaa;
    border-top-width: 0px;
    border-right-width: 0px;
    padding: 1ex;
}
.comment_form textarea {
    height: 3em;
    width: 40em;
}
.comment .comment_form {
    display: none;
}

Obviously, it could do with a makeover from a talented web stylist, but here’s how the general idea looks:

Nearly-unstyled threaded comments.

Nearly-unstyled threaded comments.

Why I don't like reverse and {% url %} in Django

Django has a feature that allows you to look up an url based on a function name and arguments. Here’s how it works:

######
# urls.py
urlpatterns = patterns('',
    (r'^projects/(?P\d+)/$', 'views.view_project'),
)
 
######
# views.py
def view_project(request, project_id):
    # create the view
 
######
# something.py
some_url = reverse("views.view_project", 4)
# some_url = /projects/4/
 
######
# sometemplate.html
<a href="{% url 'views.view_project' 4 %}">My project</a>

The idea behind urls.py is to separate urls, which are not supposed to change (because someone may have bookmarked them, there might be links to the page, etc) from implementation (views.py), which can change at any time (due to refactoring, new features, and bugfixes). You can move functions around at will, and simply change urls.py to point at the right function without breaking people’s links. I love this decoupling.

The idea behind reverse and url is that you should not be hard-coding urls into your code, you should use the names of the functions being called instead. I’ve tried to do this because its the “proper thing to do” if you’re a django coder… but I don’t like it, and I’m going to quit using it in my personal projects.

Here’s why:
1) It’s harder to maintain. As stated above, I often change the name or location of a view function. When I do that, I have to go through all the files that have a url or reverse call and change the view there. If I had hardcoded the url, I’d only have to change it in urls.py. I’ve been stung by this much more often than by changing urls.

2) Its harder to read. When you see “/path/to/something” you know you’re looking at an url. When you see reverse(‘some.module.path’, some, arg), it takes longer for the brain to parse, even if you know what the reverse call does.

3) In the case of {% url %} it exposes implementation details to the template author. Template authors should not know or care what python functions are being called internally. But they know what an url is, and what it represents.

In short, it adds an extra layer of abstraction to url handling. I like abstraction layers to be thin and useful; in the case of reverse(), the added complexity does not, in my minimalist opinion, justify the supposed gain in simplicity.

Offline Enabled Webapps And Open Source

I’m currently working on a rather simple Todo list application intended to meet the requirements I outlined earlier. I’m developing this as an offline-enabled webapp and while I haven’t gotten very far (don’t try it, it barely satisfies the most elemental requirements), I have some interesting observations to make:

First, the app is currently 100% Javascript. Its wrapped in a Django project because I’m going to have to make it online enabled at some point, but I wrote this app from the ground up to run in offline mode. This is a huge departure from traditional web 2.0 development (as of now, web 2.0 is to be considered ‘traditional’), where logic is mostly stored on the server and an ugly mess of ajax calls run that business logic on different events. Instead, I have a Javascript app that is surprisingly elegant. MVC under this architecture is basically SQLite (model), XHTML/CSS (view), and javascript (controller). Then I will have a workerpool to sync up with the server in the background.

As a result, the entire code-base for this app is being served to the web browser. So if you personally want to hack this app, you can download the source files from the browser. Basically, I can’t release this app as closed source. I could put a license on it forbidding you to modify or redistribute it, but you can still read the code; the source is open.

Of course, that’s not a problem for me, as I release most of my code as open source and its already up on github. But it could kill corporate migration toward offline-enabled web 3.0. Because while its one thing to keep your trade secrets locked up on the server and provide a few incoherent javascript calls that interact with that API, it is a completely different beast to put your entire app available for download as a .js file.

I suspect the open source movement has gained enough momentum that any companies who have these fears will simply be put out of business by more modern outfits that will spring up to take their place. Otherwise, either web 3.0 will fail to grab market share, or somebody will come up with a way to ‘compile’ javascript into a standards compliant byte-code so they can pretend nobody can reverse engineer their app. This would be a damn shame as its a politically-motivated technically useless layer of complexity on a web based architecture that I am finally happy to be working with.

Offline-Enabled Web Apps: The Future

I was reluctant to join the world of web development. I started in high school with a few sites and realized several things: Javascript sucks, Internet Explorer sucks; therefore web development sucks.

Fast-forward through a couple academic degrees. Job hunting with one requirement: Python. Python jobs all require Django.

So I learned Django, assuming, incorrectly, that if I was developing python backends, I wouldn’t need to work with the horrors of Javascript or Internet Explorer. I earned money. I relearned Javascript and became a first rate web developer.

In the back of my mind I still felt that web development sucks. So a few weeks back when deciding on a platform for a personal project, I thought I’d try something new. The Android platform was in my hands and I gave it a whirl.

I didn’t enjoy it much and I am now rewriting the app as an offline enabled webapp using Google Gears.

Then Chrome OS was announced and I realized that I’ll probably be doing a lot of offline enabled webapps using Google gears and/or HTML 5. Like it or not, it’s the future. Me, I like it. There are a lot of advantages to this kind of setup: I can access the apps from my phone, my laptop, my parent’s desktop, or Phrakture’s hacked computer whenever and wherever I want. I don’t have to write a different client for each one. Its true ‘write once, run anywhere’. I can upgrade each of those clients automatically as long as there’s a network connection.

On that note, you don’t need a network connection to run HTML 5 or Google Gears based apps. They both provide a ‘localserver’ that caches pages and javascripts, and give you an SQLite database for data caching. Typically offline versions of apps are not as powerful as their networked counterparts, but they do not require network access to run. Further, because they are locally cached, they can be made to run as fast as a “standard” (old fashioned) desktop app. The apps run in the browser, but the browser is just a container, a window manager, to hold the application.

In traditional webapps, you code most of the logic on the server side. In this new model, you end up coding most of the logic in the client, because the app needs to run without a guaranteed server connection. For me, this has a massive, nearly show-stopping drawback: A large portion of the app must be written in Javascript. JQuery makes Javascript suck lest, but it still sucks. I’m a Python programmer.

For years, I’ve dreamed of browsers supporting tags that allow me to write my DOM manipulation scripts in Python rather than the ubiquitous and annoying Javascript. This wasn’t possible because python can’t be adequately sandboxed such that arbitrary scripts running on the web don’t have access to, say, your entire hard drive.

This is no longer true. The PyPy project finally has a complete Python 2.5 interpreter that can be safely sandboxed. Since discovering this at Pycon 2009, I’ve been thinking about interfacing it with a web browser.

I figured “somebody must have started this already”. Google didn’t help much, but when I logged into #pypy on freenode I was told “fijal started doing that with webkit yesterday”. I’ve been following up trying to get the project to build (I was warned that the build process is a mess and was invited to wait until it is cleaned up a bit). So far, no luck, but I am optimistic that python support is finally coming to the browser. Granted, it won’t be much use for public webapps (at first) since browsers won’t want to be distributing pypy, but a lot of my projects are personal, and satisfying the general public will be far lower on my priorities list than ‘developing in my preferred language’.

I’ll have to install a pypy interpreter into Chrome Lite under Android before this is useful to me. That may be tricky.