Dynamic form generation

Jacob Kaplan-Moss
February 28, 2010 I had the pleasure of being on a forms panel at PyCon 2010 chaired by Brandon Craig Rhodes. To get a stable baseline, Brandon asked each of us to provide code showing how each forms toolkit might tackle a problem: Imagine that someone has already written a form with your forms library. The form looks something like this:
New username: __________ Password: __________ Repeat password: __________ [Submit]

Now, someone from Marketing comes along and announces that the developers must add some additional questions to the form - and the number of extra questions is not determined until runtime! They give you a get_questions(request) function that looks up a profile they cook up for each person browsing the site, and returns a list of strings like one of these:
['Where did you hear about this site?'] ['What college did you attend?', 'What year did you graduate?'] ['What is the velocity of a swallow?', 'African or European?']

They explain that they cannot limit how many questions might be returned; for simple users who are bumpkins, it might just be one or two, but could be many if the user sounds like a very interesting one. Your form will, then, look something like this (in the case of the third datum above):
New username: __________ Password: __________ Repeat password: __________ What is the velocity of a swallow? _________ African or European? _________ [Submit]

When you get the answers, you can save each one of them simply by calling save_answer(request, question_text, answer_text). But each question must be answered; if the user fails to fill in one of the questions, then the form should be re-displayed with all of the data in place but with a note next to each unanswered question noting that it is a required field. (Yes: you can assume that get_questions(request) for a particular session returns the same list of questions over and over again, so thats not a value you have to stash away to make the form appear consistent from one page load to the next!)

Your form, then, will have to go from being static, and driven by a simple schema, to having a series of fields that are variable, and driven by runtime logic, rather than static and driven by a schema that the programmer can enter as a constant. How can your forms library best be used to present and validate the above form? I had fun with the problem, so heres my writeup on how Django would solve this problem. Before we start, lets talk real briefly about the parts you need to display and process a form in Django. Youll need: A form class. Thisll be a subclass of django.forms.Form, and will contain all the display and validation logic for the form. A view function. Thisll be responsible for connecting the form to the submitted data, rendering the form into a template, etc. (Usually) a template used to render the form and the rest of the page.

Part 1
The first part of this assignment username / password / repeat password is very easily solved: just install James Bennetts django-registration and go out for beer. Snark aside, this illustrates one of the best aspects of Django: theres an incredibly active community producing reusable apps, so when youre faced with a problem in Django theres a good chance that theres an app for that. In the real world, in fact, Id use django-registration for this entire round. However, to keep things simple, Ill stick to whats built into Django. The form Very simple:
from django.contrib.auth.forms import UserCreationForm

Thats right: Django ships with a form that handles this common user/password/confirm task perfectly. For pedagogical purposes, though, Ill show how youd actually define this form by hand if you cared to:
from django import forms class UserCreationForm(forms.Form): username = forms.CharField(max_length=30) password1 = forms.CharField(widget=forms.PasswordInput) password2 = forms.CharField(widget=forms.PasswordInput) def clean_password2(self): password1 = self.cleaned_data.get("password1", "") password2 = self.cleaned_data["password2"] if password1 != password2: raise forms.ValidationError("The two password fields didn't match.")) return password2

The salient features here: The form is a subclass of forms.Form.

 A form includes a number of fields. Djangos got a bunch of built-in fields for common types (characters, dates/times, file uploads, etc.), and its easy to create custom field types. Each field has an associated widget which defines how to render the form field. Djangos HTML-focused, and as such ships with built-in widgets for all the HTML input types. Again, its easy to write your own widgets, and since theres nothing HTML-specific until you reach the widget layer itd be easy to use the forms framework on top of something else like XForms. Here, all the fields use CharField, but the password fields get rendered as <input type=password> by using a PasswordInput. Fields generally know how to validate and clean (convert from strings to proper types) themselves. So an IntegerField will automatically coerece strings on the wire into integers. Here, we need some extra validation we want to check that password2 matches password1. For the purpose, we can define a custom clean method. We provide a clean method by defining a method on the form called clean_<fieldname>. By the time our clean_password method gets called, the fields clean methods have already been called, which has put cleaned and validated data into self.cleaned_data. However, we need to guard against password1 not having been provided: fields are required by default, but cleaned_data only contains data thats passed validation, so if password1 wasnt given, it wont be in cleaned_data. For similar reasons, clean_password2 will only be called if somethings been entered in the field. Clean functions do two things: Validate data, raising forms.ValidationError if the datas not valid. Clean coerce, modify, or otherwise munge the data. Theyll pull the old data out of self.cleaned_data, and return the new, cleaned and validated data. There are also various hooks for form-wide validation which arent needed here. The view Now that weve got a form, we need a view function to present that form. Views and how theyre wired up are out of scope for this discussion, so I wont cover them in detail. Instead, Ill just look at how they apply to form processing. So heres our view:
from django.shortcuts import redirect, render_to_response from myapp.forms import UserCreationForm def create_user(request): if request.method == 'POST': form = UserCreationForm(request.POST) if form.is_valid(): do_something_with(form.cleaned_data) return redirect("create_user_success") else: form = UserCreationForm() return render_to_response("signup/form.html", {'form': form})

Salient details: To be a good HTTP citizen, we only process the form if the request was submitted with POST. If the request was a POST, well create a bound form a form bound to some associated data by passing in request.POST. request.POST is essentially a dict, and forms will operate on any dict-like structure. I mention this to point out that though forms are usually used with HTML/HTTP, they can also be used with lots of other data sources. For example, Piston, a REST API framework, uses Djangos forms to validate data submitted via an API in JSON, YAML, XML, etc. Calling form.is_valid() kicks off all the data cleaning and validation. If it returns True then all the validation succeeded. At this point we can do something with our submitted data usually by reading form.cleaned_data. Again, to be a good Web citizen, well follow the common Post/Redirect/Get pattern upon success. If the form wasnt valid, well fall through returning a response. The form instance, however, will have an errors attribute containing all the errors that prevented it from being valid. If the request wasnt a POST, well construct an unbound form. This form will have no associated data, and hence will have no errors. Finally, well render a template signup/form.html and pass in the form instance to be rendered. Theres actually a slightly shorter way (one thats rapidly becoming a Django idiom) to write this view:
def create_user(request): form = UserCreationForm(request.POST or None) if form.is_valid(): do_something_with(form.cleaned_data) return redirect("create_user_success") return render_to_response("signup/form.html", {'form': form})

Exactly how and why this works is left as an exercise to the reader. Hint: recall that request.POST will only have POST-ed data in it The template Again, templates are out of scope, so Ill just look at how theyd touch on form rendering. At its simplest, a the signup/form.html template would be very simple indeed:
<form method="POST" action="."> {{ form }} <input type=submit> </form>

Forms know how to render themselves into HTML with relatively simple markup. This does quite a lot, though: itll render the form, filling in any pre-existing data if the form was bound, and will also render associated error messages.

Notice that the form doesnt render the surrounding <form> tag, nor does it render the <ipnut type=submit>. This is by design: you could, for example, compose multiple forms into a single <form>, or you could be rendering this form into something thats not HTML. There are a couple of other shortcuts to rendering forms in a few common ways: {{ form.as_p }}, {{ form.as_table }}, {{ form.as_ul }}. For maximum control, you can render each field separately, or even write the form completely by hand. For details, consult the form documentation

Part 2
Okay, lets kick it up a notch. The second part of this problem involves adding custom registration questions for each person. Well need to integrate these custom question into the form. Note that the template wouldnt need to be changed at all, so well skip it entirely in this section. The view Well actually start by looking at the view, since itll need a couple of minor tweaks to handle the new form:
def create_user(request): extra_questions = get_questions(request) form = UserCreationForm(request.POST or None, extra=extra_questions) if form.is_valid(): for (question, answer) in form.extra_answers(): save_answer(request, question, answer) return redirect("create_user_success") return render_to_response("signup/form.html", {'form': form})

Youll notice that the general structure of the form processing is intact. This is typical: form views almost always follow this idiomatic style. So whats different here? Were gathering a list of extra questions from this hypothetical get_questions(request) function, and then feeding them into the form class. If the form is valid, were calling form.extra_answers() and using the returned data to save those answers. Ill look at how to make the associated changes to the form in a second, but first a brief philosophical digression. In essence, weve pushed the handling of the questions and the validation of their answers down into the form class itself, but weve left the parts of the problem that deal with the request object in the view. This is the idiomatic way of tackling the problem: if we handed the request object to the form directly wed be cutting against the grain of Djangos form library, which doesnt want to be tied to HTTP directly. The form After writing the view, its clear we need to modify our form in a couple of ways: Well need to change the __init__ signature to accept this list of extra questions, and well need to modify the forms fields accordingly. Well need to add an extra extra_answers() function that returns (question, answer) pairs.

Lets look at __init__ first:

class UserCreationForm(forms.Form): username = forms.CharField(max_length=30) password1 = forms.CharField(widget=forms.PasswordInput) password2 = forms.CharField(widget=forms.PasswordInput) def __init__(self, *args, **kwargs): extra = kwargs.pop('extra') super(UserCreationForm, self).__init__(*args, **kwargs) for i, question in enumerate(extra): self.fields['custom_%s' % i] = forms.CharField(label=question)

So whatd we do here? We pulled the extra keyword argument out of **kwargs and then called the superclass initializer. In practice, wed probably want some error checking to make sure extra was given. We then looped over the extra questions and added them to self.fields. self.fields is a dict mapping field identifiers input names, essentially to Field instances. Normally, we define these declaratively (as we still do for username, password1, and password2). Since were dealing with dynamic data here, though we cant define them declaratively, so we can simple add new fields to self.fields. Were arbitrarily naming the fields custom_0, custom_1, etc. since weve been informed that get_questions is stable. If it wasnt, we might need to do something like hash the question text. By default, the label for the field will be the same as the fields identifier. Since Custom 1: _____ doesnt make for very good UX, weve passed the question text itself in as the fields label argument, which will render by default as an associated <label> tag. Fields are required by default, so we fulfill that part of the assignment without any custom validation. If they were to be optional we could pass required=False in to the fields constructor. That was the tricky part; the extra_answers() function is easy:
class UserCreationForm(forms.Form): ... def extra_answers(self): for name, value in self.cleaned_data.items(): if name.startswith('custom_'): yield (self.fields[name].label, value)

We simply loop over self.cleaned_data, yielding any pairs for fields starting with custom_. The only slightly tricky bit is that we need to pull the original question back out of the label value by accessing self.fields[name].label. Thanks for the challenge, Brandon; it was fun!