I’ve just had a very interesting conversation with my colleague Marco about different approaches to the organization of code inside a Django application.
As you might know (and if you don’t I’ll tell you anyway), Django’s views (somehow occupying the “Controller” level in an MVC architecture) must take (at least) an HttpRequest instance as a parameter and must return an HttpResponse instance. That’s how it goes in Django, this is the law. This means that you must be sure that the last instruction in your request processing code (in whichever way you’ve organized it) must return an HttpResponse instance, usually calling the HttpResponse() constructor (or of any of its useful subclasses), or by calling the django.shortcuts.render_to_response() function, or something similar.
This has, in my opinion, a major drawback: it might limit code reuse and it increases the coupling in the code. Everything’s not lost, however.
Before you start the flame wars, let me explain, using an example coming from the Django website; this represents a basic Django view function, returning some response containing data fetched from the database:
Let’s say now that I want to reuse that particular data (the ‘p’ variable) in another view: given that the return value is always an HttpResponse instance, you are screwed; sometimes you just need the data, to find something, or simply to render it in another format like JSON or XML (RESTful architectures, anyone?). This goes pretty much against the DRY principles, and if you don’t go deeper than the Django tutorials, your whole application might feature lots of repeated code.
Even worse, you have a direct reference to a template (”polls/detail.html”), and this kind of coupling does not scale well. It can become a real problem in big projects.
There are, however, strategies to avoid this: the first, the most common, is to refactor your code and to create a “layer” of data-specific functions, which will return instances (or arrays thereof) that you can reuse here and there. Doing this in a big project already started requires a good deal of unit testing first, to ensure that your refactoring is not breaking something elsewhere, but that’s another problem (because you DO unit test, right??). This approach might not scale well in complex projects, and thus you would like to organize your code in other ways.
I learnt about organizing views using callable objects instead of functions while studying the code in the Django REST Interface project. In this case, you create code like this:
The important bit here is the “__call__” method, which allows an instance to be called as a function, without specifying any particular method. This makes me remember of the dreadful VB default methods but in Python it’s not that bad, actually (VB is horrible by default anyway), and allows you to use a cool syntax to do complex tricks (”command pattern” way of doing things, without the method call overload). And of course, since you are using an object-oriented approach, you can use polymorphism and inheritance to organize and reuse code as much as you can (or want).
Finally, Marco told me that his team uses another cool approach: they avoid returning HttpResponse instances from the views, and instead use Python decorators to generate those. This way, you can achieve another neat separation of concerns, and you can reuse code simply and effectively.
I understand that the Python philosophy cares about explicitness, but the “easy” way of processing requests in Django leads to trouble in big applications: increased coupling, reduced DRY, more headaches. I think you should use some code-reuse strategy in your Django code, but this, of course, is more an architectural problem than a Django problem.
11 Comments
#1. Batiste Bieler 04.04.2008
Yes this obligatory HttpResponse is nasty a problem especially when you want to use a simple view into an aggregated one that use several simpler views.
The ressource object seems to be a powerfull idea for implementing REST.
But for complex view that render different objects on the same page is it possible to use these kind of ressource objects?
My solution was more in the decorator style of doing things, here is some example (look at only_context parameter):
http://www.djangosnippets.org/snippets/559/
#2. Adrian 04.04.2008
The “Resource” objects used in the Django REST project can have any number of methods in them, not only the __call__ method, so you can have some methods that return data and others that perform the serialization in whichever format you need.
Beware also of the fact that using objects means that you have an overhead in terms of instantiation and destruction, that you might not have using functions (however I don’t know that well the inner workings of the Python runtime, and the relative costs of the decorator vs. the OO approach). I should have mentioned this in the article, by the way.
#3. James Bennett 04.04.2008
So, the things that bug me in your arguments are twofold:
First, you rely to an extreme extent on the word “might”. Something “might” not scale well; another thing “might” cause repeated code. Relying on situations where something “might” happen might lead you rather far astray.
Second, you’re almost entirely arguing based off a single example from the Django tutorial, devoid of any further context; yeah, it doesn’t paramaterize return types or template names or refactor itself into a class to dynamically handle any feature you want out of it, but in a web framework oriented toward returning rendered HTML via HTTP, is it really so evil to have the tutorial use that as an example? You apparently managed to work out that if you want to do tons of other things, it’s easy enough to do so, so it can’t be that hard.
#4. Adrian 04.04.2008
@James thanks for your comments; again, as I’m not a native English speaker I might *grin* be using the word in non-conventional ways. Sorry for that. Secondly, I’m not entirely arguing about a single example; I simply think that the choice of writing the name of the template at the end of the view is simply flawed, and that the tutorials should *also* point out other ways, which is not currently the case. As I pointed out at the end of the entry, it’s an architectural problem, not specific to Django, by the way.
#5. David, biologeek 04.09.2008
@Adrian : I totally agree with you. I use the rest-interface project for all my apps now because that’s really more powerful when you think in terms of resources and representations.
@James : you react on details but the related issue is more interesting…
#6. Adrian 04.09.2008
Hi David!
Thanks for your comment! Now that’s a coincidence, I’ve stumbled on your site today for the first time :) I’m using an enhanced version of the (excellent) rest-interface project which I hope to release to the public soon, adding some new features here and there.
#7. David, biologeek 04.09.2008
Let’s call this coincidence greut ;)
I’m really interested in your enhancements, could you tell me more? Or maybe the rest-interface mailing-list is more appropriated? I hadn’t enough time yet to extract my own from my current project but it could be awesome to update the quite-dead rest-interface project.
@Batiste : your approach is really neat. It doesn’t handle different rendering, etc but it looks sufficient for some projects.
#8. Adrian 04.09.2008
Ahhhh I understand now :)
As for rest-interface:
- I’ve added support for the OPTIONS verb (and can be extended to other verbs like HEAD, too).
- I removed the use of Django’s own serialization of model objects, using native Python facilities instead, such as simplejson, yaml, XML, csv, etc, using a simple dictionary object as input for all of them, which includes not only the data, but also metadata (messages, timestamps, etc), and can then serialize any kind of data (arrays, typically). This way resources can return pretty much anything.
- I’ve added some configuration values, with separate security checks for valid HTTP headers (they can be activated and deactivated in the settings file).
- Most important: I created a special resource which returns the complete API description, which I then use to generate API wrappers in Python, Ruby, PHP, even Objective-C :) This means that each resource describes the input and output they require (kind of a poor man’s WSDL, if you want)
- I’ve also finished the implementation of Digest Authentication (what a mess, geez! Now I understand why only big companies use it!)
- and finally I’ve also extended the django.test.Client class (featured in a previous post in this blog) with support for PUT and DELETE and OPTIONS and Digest Authentication,
- and added plenty of testing to all of this.
Phew. :) I’m sure it can be useful to others, and I hope to publish it as soon as possible!
#9. Adrian 04.09.2008
@David, @Batiste: I’m sure that you can parameterize the decorators to make them return whatever you want…
I am using decorators in the modified rest-interface to secure some resource methods if needed (for example, GET does not require auth but POST yes, and so on) and from what I’ve seen, it’s just syntax sugar for encapsulating functions inside other functions, so you can parameterize plenty of stuff with it!
#10. David, biologeek 04.10.2008
Looks great!
You Client additions reminds me a ticket where I’d already done that (partly): http://code.djangoproject.com/ticket/5682
What’s the format of the returned to describe your API?
Decorators on verbs/methods for fine grained access is what we’d discussed recently on the rest-interface mailing-list, do not hesitate to post your enhancements.
This project really deserves a new team…
#11. Adrian 04.11.2008
The description is basically a tree, with a first level of leaves with the resources (one entry per URL described in the urls.py file) and, for each resource, each exposed method (GET, POST, etc) with its documentation, parameters, etc. You can have this in any format (JSON, XML, YAML) depending on the “Accept:” header MIME type, or the “format” parameter either in the GET or POST collection.
I agree that it’d be great to make this project go forward! I think having a good REST interface in Django would make it useful for many new projects. Just a question, are you a committer of Django?
Cheers!
Commenting