Django: Generators are your friend

Attribution: http://www.flickr.com/photos/dorkomatic/ / CC BY 2.0

Pro tip: if you need to shove a lot of data through a Django view, DO NOT attempt to create a big string-- use a generator.


Here's something that seems sensible on the surface:

body = ""
for region in regions:
  for zip in region.zipcode.iterator():
    body = body + "\t".join(
           [region.region, zip.zipcode]
           ) + "\n"
resp = HttpResponse(body, mimetype='application/ms-excel')
resp['Content-Disposition'] = 'attachment; filename=%s.xls' % (unicode("Regions"),)

Except when the set of regions and zipcodes gets large!  Let's consider the following replacement that uses a generator:

def dump_it():
    for region in regions:
       for zipcode in region.zipcode.iterator():
           yield "\t".join(
              [region.region, zipcode.zipcode]
           ) + "\n"

resp = HttpResponse(dump_it(), mimetype='application/ms-excel')

resp['Content-Disposition'] = 'attachment; filename=%s.xls' % (unicode("Regions"),)

There is a slight performance difference; the latter takes a few seconds (2 seconds for almost 70K resulting rows on my dog of a laptop).  However, I attempted to benchmark the former on the same dataset, and it took almost 13 MINUTES (775 seconds).  So, slight, meaning within 3 orders of magnitude.
Read more...

Django Forms: Alternate Date Handling

For usability on the "score" page at Sage Steps (free registration required if you want to check it out), we decided that the best method to present a date was a simple drop-down with, e.g. "February 2010" as the text.

I tried out several permutations with mixed luck, but then happened upon the following recipe:

def month_year():
    today = date.today()
    today = date(today.year, today.month, 1)
    dates = []
    for i in range(1,13):
        if (today.month > i):
            month = today.month - i
            year = today.year
        else:
            month = 12-(i-today.month)
            year = today.year - 1
        mon = date(year, month, 1)
        dates.append((mon, mon.strftime("%B %Y")))
    dates.reverse()
    return dates

This creates a set of tuples, e.g.: (date(2009,3,1), 'March 2009'),

month = DateField(widget=Select(choices=month_year()))

This will produce the drop-down as above, and

form.cleaned_data['month']

Will actually return a datetime.date object (trust me, that's a good thing). Read more...

Numeric Formatting in Django

You might have seen my previous post lamenting the weirdnesses around number formatting in python.  This spills over into Django; if you have DecimalFields in your models (especially if you are using the ModelForm object to create your forms.


The previous "format Decimal object as a string that any 10-year-old would expect" method is:

def dec_string(dec):
    if isinstance(dec, Decimal):
        if dec:
            if dec.normalize().as_tuple().exponent > 0:
                return "%d"%dec
            else:
                return "%s"%dec.normalize()
        else:
            return 0
    else:
        return dec

(I added the isinstance() bit to protect me from myself =)

Since we know how to format this, all that remains to be done is to create a widget that automatically formats DecimalFields correctly.

class DecimalInput(TextInput):
  def render(self, name, value, attrs=None):
    value = dec_string(value)
    return super(DecimalInput, self).render(name, value, attrs)

And then just use that in your form:

class EnergyForm(ModelForm):
    kwh = DecimalField(widget=DecimalInput(attrs={'size':'6'}))

There is more to the ModelForm, but that's what the docs are for, right? Read more...