Posts tagged ‘validation’

Django and jquery.tmpl

Lately, I’ve been finding Django increasingly inappropriate for the web applications I develop. I have several complaints: the forms library doesn’t extend well to ajax requests, any extensive customization to the admin requires obtuse inspection of the admin source code, many admin customizations simply aren’t possible, the “reusable apps” philosophy has added a layer of complexity to a lot of things that really should not be there, there are no obvious best practices for ajax support.

In spite of all this Django is still better than other frameworks (Python or not) that I have investigated or tested. I’ve considered writing my own web framework, but I wouldn’t maintain interest in it long enough to get it off the ground. So I’m letting my complaints bounce around in the back of my mind with the hopes that I can improve Django so that it continues to work for me as a platform.

I’m currently trying to come up with a better system for ajax requests. I have nothing concrete in mind, but I’ve started with the premise that ajax requests should never return rendered html, but should return only json (hence my issue with the django forms library). With that in mind, I need a templating library for json data. JQuery is a must, and the officially supported JQuery templating library is jquery.tmpl http://api.jquery.com/category/plugins/templates/

The problem with jquery.tmpl is that it uses django-like syntax. The following is a valid block that may be rendered in a jquery.tmpl page:

<script id="project_tmpl" type="text/x-jquery-tmpl">
    {{each projects}}<li>${$value}</li>{{/each}}
</script>

If you try to include this in a django template, the {{ and }} tags will be replaced with (probably) empty variables. Django has {% templatetag %} to render these individual items, but what we really need is a way to tell the django templating system to leave complete sections of code alone. So I wrote the jqtmpl template tag. It allows us to wrap code in a block that tells Django not to render that block as template code. So the above would show up in a Django template as follows:

<script id="project_tmpl" type="text/x-jquery-tmpl">
{% jqtmpl %}
    {{each projects}}<li>${$value}</li>{{/each}}
{% endjqtmpl %}
</script>

Here’s the template tag:

from django.template import Library, TextNode, TOKEN_BLOCK, TOKEN_VAR
 
register = Library()
 
@register.tag
def jqtmpl(parser, token):
    nodes = []
    t = parser.next_token()
    while not (t.token_type == TOKEN_BLOCK and t.contents == "endjqtmpl"):
        if t.token_type == TOKEN_BLOCK:
            nodes.extend(["{%", t.contents, "%}"])
        elif t.token_type == TOKEN_VAR:
            nodes.extend(["{{", t.contents, "}}"])
        else:
            nodes.append(t.contents)
 
        t = parser.next_token()
 
    return TextNode(''.join(nodes))

This doesn’t handle Django’s {# comments #}, as token.contents doesn’t return a valid value for comment items. As far as I know, you wouldn’t use the comment construct inside a jquery.tmpl template anyway, so it’s still functional.

Next on my list is a better forms validation library to suit my theory that validation should be done client side. I’ve got a server-side system in mind that returns json responses, and does not return user-facing error messages, since those should have been caught client side. With these tools, and hopes for Grappelli to eventually create a decent admin, Django may continue to serve me.

HTML Form validation

I’ve been experimenting with pylons oven the past week. It’s a nice framework, but the recommended html form validation engine, formencode sucks.

I researched alternative form engines, and looked at the form engine I’m most familiar with, in Django with new eyes. It is also a pain to work with. I just never noticed it before. I can’t find anything that works as seamlessly as it should. I think Flatland may be a good solution for pylons, but I have decided that nobody is doing it right yet.

Why is this obviously common problem is so unsolved. What do html form validation frameworks do?

  • (optionally) generate the form in the first place.
  • Validate form data on the server side when form data is supplied
  • Convert the incoming string form data to Python objects or data structures
  • If the form data is not valid, issue a GET request that redisplays the form with error messages

The last step is the messy one. The two possible solutions I’ve seen are to either generate the response (ie: create an entire form with values and error messages) or to modify the response in some sort of middleware (the form is generated from a template, and then the middleware adds value=”" attributes and error messages to the result). Either way, we end up with controller code that is too tightly coupled to view code.

Yet, this isn’t how the web works anymore. The “POST to server and if it’s invalid, display the form with error messages” paradigm is what you might call “uncool.” In awesome apps, we validate the code on the client side, using Javascript, or, even better, HTML 5 form validation (I mean, we would use it if the browsers supported it consistently), preferably as the user types or as soon as the widget is blurred. This is a far better interaction experience.

Wouldn’t it be nice if we could do away with server-side validation altogether? Obviously, we can’t, because we need to respond appropriately if someone issues a malicious POST or tries to POST to our form from a script or scraper instead of the fancy validated web page we displayed to a web browser. The point I’d like to emphasize is that the appropriate response in these situations is NOT to redisplay the HTML form with values filled in and error messages. If it’s a malicious request, we just want to make sure we don’t do anything with it. If it is an invalid request from a script, the script programmer doesn’t want to see in the response, they will just want a succinct error message telling them how to fix the validation in their script.

I’d therefore like to propose properly decoupling the controller and view. The view (html+javascript) does validation, while the server side focuses on converting the incoming data to python objects. If it encounters an error while doing the conversion, it can bail with an appropriate error message (probably a 406 status code). Getting this status code means your webapp is broken –just like a 500 error would indicate– because your webapp should have validated the form input before submitting it. This would also be a suitable response in a REST based design. If a REST client sends invalid data, it should get an HTTP 406 with appropriate error message.

This simplifies the server side code a lot. It no longer has to worry about generating the html form, filling it with values, or returning error messages. The client side code becomes more complicated, but it’s the kind of more complicated that should be done anyway, to enhance the user experience. Ideally, it should be easier to write client-side validation than to default to server-side validation on form requests. It’s too easy to use django’s forms framework to create server-side validation without ever bothering with client-side validation.The coolest websites tell us our passwords don’t match *before* you submit the form. This should be the norm. It should be easier to write this kind of client-side validation than it is to default to server-side validation, so people default to using it.

Going forward, browsers need to support the new HTML 5 input types with proper validation and error messages. I also think there needs to be better support in the html spec (maybe html 5 has it and I don’t know about it?) for displaying validation errors. something like . For more complicated validation, we need a nice javascript framework. I haven’t used it yet, but I’m looking forward to experimenting with the jQuery Validation plugin.

With that in mind, I’m planning to continue using formencode for datatype conversion, but to ignore it’s buggy htmlfill stuff. Maybe eventually I’ll make a simpler stand-alone library that does this stuff and returns a 406 on invalid input.

Validate as you type in javascript

This is a common task, but has several facets that are easy to miss. I needed an input element that would validate as you type to ensure that an integer value was always entered. This was easily done by doing a replace against a regular expression, linked to the keyup event:

function validate_number(event) {
    this.value = this.value.replace([/[^0-9]/g,'');
}
 
$('jquery selector').keyup(validate_number);

This works fine provided the user is typing at the end of the box. The problem is that if the user has clicked in the middle of the number, aiming to insert a new digit, each time this.value is set, the cursor is moved to the end of the box, regardless of whether or not a valid character was entered. This happens on every keypress. That is bad.

The solution is to save and reset the selection range, as follows:

function validate_number(event) {
    if (this.value.match(/[^.0-9]/g)) {
        var start = this.selectionStart-1;
        var end = this.selectionEnd-1;
        this.value = this.value.replace(/[^.0-9]/g,'');
        this.setSelectionRange(start, end);
    }
}

I haven’t found any situations where this code doesn’t work, but there is one other caveat: pasting. If the user pastes a non-numeric value into the field without issuing a keypress (ie: using the right click–> paste menu), the value will not be validated. I haven’t found a way to stop this at the point when the value is pasted, but I’ve found it sufficient to additionally call the validation function when the textbox loses focus:

$('jquery selector').keyup(validate_number).blur(validate_number);

A variation of this code can be used to validate most inputs as the user types; a regular expression that matches whitespace can remove whitespace, phone numbers would look for digits and hyphens, decimal numbers would additionally search for a period, etc.