A tale of two registrations

It was the best of registrations, it was the worst of registrations... just kidding.

Here are two alternates for the new Sage Steps registration page, and I'd like to solicit your feedback.

Which of these do you like better? The mad-libs version, or the "traditional" version? Read more...

I'm psychic!

I have developed a tremendous psychic ability to read people's minds.  This has taken years of training, and arduous study at a Shaolin temple, but I'm finally ready to "out myself".

So here it is.

I'm going to read your mind.


Here's the question, and then I'm going to tell you what you were thinking:


How many kilowatt hours did you use last month?


And the answer?


I'm getting something... vaguely... one of your relatives is trying to communicate from beyond the veil...


Here's what you're thinking: "I don't know."


Aren't I amazing?  Well, yes, but that has nothing to do with it: I've asked this question a lot of times.  I have had one person that knew (and he's in a "carbon measurement" startup).  The rest answered, "I don't know."


It is very difficult to improve something that you measure, and things that you measure improve virtually automatically.  It is time to begin to measure how much energy, water and fuel that you use.  Because we can all improve our environmental impact.


Note that this doesn't address any particular ideology: you should be measuring whether you want to save the planet, or if you want to save money every month on your power bill. Read more...

Design prototype for Sage Steps

This is something that I've been working on for a bit, and I'd really appreciate your feedback. Read more...

Django: Generators are your friend

Attribution: http://www.flickr.com/photos/dorkomatic/ / CC BY 2.0

Pro tip: if you need to shove a lot of data through a Django view, DO NOT attempt to create a big string-- use a generator.


Here's something that seems sensible on the surface:

body = ""
for region in regions:
  for zip in region.zipcode.iterator():
    body = body + "\t".join(
           [region.region, zip.zipcode]
           ) + "\n"
resp = HttpResponse(body, mimetype='application/ms-excel')
resp['Content-Disposition'] = 'attachment; filename=%s.xls' % (unicode("Regions"),)

Except when the set of regions and zipcodes gets large!  Let's consider the following replacement that uses a generator:

def dump_it():
    for region in regions:
       for zipcode in region.zipcode.iterator():
           yield "\t".join(
              [region.region, zipcode.zipcode]
           ) + "\n"

resp = HttpResponse(dump_it(), mimetype='application/ms-excel')

resp['Content-Disposition'] = 'attachment; filename=%s.xls' % (unicode("Regions"),)

There is a slight performance difference; the latter takes a few seconds (2 seconds for almost 70K resulting rows on my dog of a laptop).  However, I attempted to benchmark the former on the same dataset, and it took almost 13 MINUTES (775 seconds).  So, slight, meaning within 3 orders of magnitude.
Read more...

Django Forms: Alternate Date Handling

For usability on the "score" page at Sage Steps (free registration required if you want to check it out), we decided that the best method to present a date was a simple drop-down with, e.g. "February 2010" as the text.

I tried out several permutations with mixed luck, but then happened upon the following recipe:

def month_year():
    today = date.today()
    today = date(today.year, today.month, 1)
    dates = []
    for i in range(1,13):
        if (today.month > i):
            month = today.month - i
            year = today.year
        else:
            month = 12-(i-today.month)
            year = today.year - 1
        mon = date(year, month, 1)
        dates.append((mon, mon.strftime("%B %Y")))
    dates.reverse()
    return dates

This creates a set of tuples, e.g.: (date(2009,3,1), 'March 2009'),

month = DateField(widget=Select(choices=month_year()))

This will produce the drop-down as above, and

form.cleaned_data['month']

Will actually return a datetime.date object (trust me, that's a good thing). Read more...

Random bass

I suppose I have to do this sooner or later. Here's a track I recorded.

Gear:

Fender Squier 5 string Jazz Bass, defretted by yours truly.
Korg PXR4
Audacity for some post-recording cleanup.

I made this whole thing up on the fly; there were three tracks:

1) the "drum" part: basically a muted slap of the fingerboard
2) single notes
3) "7" chords (1,3,b7)

Random bass Read more...

Numeric Formatting in Django

You might have seen my previous post lamenting the weirdnesses around number formatting in python.  This spills over into Django; if you have DecimalFields in your models (especially if you are using the ModelForm object to create your forms.


The previous "format Decimal object as a string that any 10-year-old would expect" method is:

def dec_string(dec):
    if isinstance(dec, Decimal):
        if dec:
            if dec.normalize().as_tuple().exponent > 0:
                return "%d"%dec
            else:
                return "%s"%dec.normalize()
        else:
            return 0
    else:
        return dec

(I added the isinstance() bit to protect me from myself =)

Since we know how to format this, all that remains to be done is to create a widget that automatically formats DecimalFields correctly.

class DecimalInput(TextInput):
  def render(self, name, value, attrs=None):
    value = dec_string(value)
    return super(DecimalInput, self).render(name, value, attrs)

And then just use that in your form:

class EnergyForm(ModelForm):
    kwh = DecimalField(widget=DecimalInput(attrs={'size':'6'}))

There is more to the ModelForm, but that's what the docs are for, right? Read more...

Numeric Formatting in Python

Sometimes I hate Python. Here's the difference between Perl and Python: Perl accepts, even embraces the fact that language is messy. Python sometimes gets this wrong.

Here are some samples of dealing with formatting numbers:

from decimal import Decimal

>>> f= Decimal('1200.1')
>>> "%s"%f.normalize()
'1200.1'

Huzzah, it does the right thing!

>>> f= Decimal('1200')
>>> "%s"%f.normalize()
'1.2E+3'

 Doh!  But clearly, removing the normalize() will fix things, right?

>>> "%s"%f
'1200'

Yee-haw! Now we're cooking!

>>> f= Decimal('1200.100')
>>> "%s"%f
'1200.100'

Hm, users don't want the trailing zeroes.  The only thing that I can think to do is use a regular expression, and now I have two problems.

 But wait! If the number doesn't have anything after the decimal, then we do one, and if it does, we do the other!

def dec_string(dec):
    if dec:
        if dec.normalize().as_tuple().exponent > 0:
            return "%d"%dec
        else:
            return "%s"%dec.normalize()
    else:
        return 0

 And that seems to be working.  Note that the final version uses "%d"%dec, rather than %s, since we want to make sure and truncate past the decimal.

Update: check out my post about integrating it with Django forms to make it seamless. Read more...

Nginx for static content

At Sage Steps, we're using Nginx (http://wiki.nginx.org/) as a front-end web server.  It serves up our static content, and load balances the back-end Apache servers.

location ~* ^.+\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|html|htm)$ {
root /static;
}

All of the dynamic stuff is happening in Django running under Apache using ModWsgi. Since this is a fairly heavy stack (Apache is very robust and featureful, but is also somewhat romanesque).

So that we're not using a 10 lb sledgehammer to do finish nails (remind me to tell you about the time that I saw a full-bird Colonel drop a sledgehammer on a Lt. Colonel's foot!), we basically tell Nginx to serve up anything ending in "jpg", "xls", "pdf", etc. from the static directory, without talking to Apache. Read more...

Blacklist in CyanogenMod

If you are running CyanogenMod on your Android device, and are surprised that some people seem to not be able to call you (half a ring, then hangup), you probably accidentally blacklisted that user.

My hope was that there was some simple method to clear this blacklist (which essentially "killfile"s phone numbers).  Either I misunderstood the user interface, or there was something desperately broken, but I could not figure out a way to clear the blacklist.  So I went digging in the source (yeah for open source!).

Turns out there is a file called blacklist.dat that stores these numbers, so here's my brute force solution for clearing the phone black list:

1. open terminal
2. su
3. rm /data/data/com.android.phone/files/blacklist.dat
4. reboot

Reboot is required.  Do it now.

And now, your wife is able to call you again, contributing to marital harmony FTW.

PS- no, I didn't tell you that you need a rooted phone, since you don't have this issue unless your phone is rooted. Read more...