Setup Memcached for Django in a Development Environment

So I just started adding a caching layer for a large Django project and found that the initial setup was much less painful than I expected. Here are the steps so far:

1. install memcached system-wide.

sudo apt-get install memcached

2. install python bindings

pip install python-memcached

3. Add cache settings to settings.py (or local settings for specific configuration). Set the location to local ip address.

CACHES = {
    'default': {
        'BACKEND':'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION':'127.0.0.1:11211',
    }
}

4. Test that everything works.

from django.core.cache import cache
cache.get('foo')
cache.set('foo', 'bar')
cache.get('foo')
'bar'

From here the Django cache settings can be used to cache the whole site, per-view or template framgents: https://docs.djangoproject.com/en/dev/topics/cache/

Advertisements

Set up git with ssh keys on bitbucket

I found the bitbucket documentation on this a little verbose.

So here is my minimalist guide. I assume that there is already a repo created in bitbucket.

1.  Copy the ssh url from bitbucket. On your repo page there should be a section along the lines of Clone this repository.

Hit SSH and copy the url part (e.g. git@bitbucket.org:myproject/myproject.git).

2.  On the local machine:

    $ git remote set-url bitbucket

3.  Check the remote address:

    $ git remote -v

This should show something like:

    origin git@bitbucket.org:myproject/myproject.git (fetch) 
    origin git@bitbucket.org:myproject/myproject.git (push)

4.  Now let’s generate a new SSH key. If you already have an SSH key for the machine, then that can be used. These instructions assume this is the first. Go ahead an accept the default location for saving and if you don’t want to be asked for a password each time you push/pull (which is one of the big advantages of using ssh keys), then hit Enter for the passphrase.

    $ ssh-keygen

5.  Go to the public part of the saved key. It’s probably in /home/.ssh/id_rsa.pub. Copy it.

6.  Now, on the bitbucket site, go to user->account->ssh keys, and paste the key into the box. Label the key something logical.

7. Back on the local machine:

    $ git push origin master

This should push your repo as normal, but using the SSH protocol and without the need for a password.

End.

Git remove all .pyc files

I ran into this recently with a project which already had several git commits but was still tracking .pyc files. This was starting to cause problems so I had to find a way to remove them from the repo:

    $ find . -name "*.pyc" -exec git rm -f {} \;

Then *.pyc can be added to the .gitignore file.

Thanks to Yuji Tomita for this information which can be found here.

Using GDAL with GeoDjango

So, just a quick solution to a simple but possibly common problem. When running through the official GeoDjango tutorial I had some problems with the GDAL library being recognized.

I did this:

    $ ogrinfo world/data/TM_WORLD_BORDERS-0.3.shp

and go this:

    $ ogrinfo world/data/TM_WORLD_BORDERS-0.3.shp

ogrinfo: error while loading shared libraries: libgdal.so.1: cannot open shared object file: No such file or directory

The answer, which took me a little while to find, was to add the library path, /usr/local/lib to /etc/ld.so.conf.

Then activate the path:

    $ ldconfig

Now all is fine.

Filtered Menus in Django

For a recent project I needed to make some classic filtered menus. This is the typical cascading choice type where a selection in one field of a form filters the available options in a subsequent field.

I cam across several methods to get this done in Django, most notably the Dajax project. However, I was expecting that this would be a one-off ajax call and I had recently done me some ajax learning so I figured I would just write it myself.

First off I need to define my model.

models.py:

class Country(models.Model):
    name = models.CharField(unique=True, max_length=255,)
    capital = models.CharField(unique=True, max_length=255)

Now I need a form.

forms.py:

from django import forms

class SearchForm(forms.Form):
    country = forms.ModelChoiceField(
                queryset=Country.objects.values_list('name'), 
                empty_label='Not Specified', 
                widget=forms.Select(attrs={ 
                                   "onChange":'getCity()'})
                )

    city = forms.ModelChoiceField(
                queryset=Country.objects.values_list('city'), 
                empty_label='Not Specified'
                )

Not much of a form but sufficient for demonstration purposes. A couple of things to note here:

1. I set up the querysets as value_lists for each field. This means that our default is to display a full list of both country name and city in our unmodified select boxes. In case the javascript breaks or is disabled, I will still be able to make our selection. Just not so elegantly.

2. I use the Widget.attrs argument to tweak the html output of the capital select field. Specifying additional attributes in the form is a very powerful method for adding specific markup to the form fields. In this case I add an onchange event handler.

Then of course the template which holds the html output of the form.

index.html:

<div id="select_form">
    <form action="{% url search %}" method="post">{% csrf_token %}
    {% for field in form %}
    <div class="field_wrapper">
        {{ field.errors }}
        {{ field.label_tag }} {{ field }}
    </div>
    {% endfor %}
    <input type="submit" name="submit" value="Search"  />
    </form>
</div>

Now, in my urls.py I need to specify two url patterns. One for the template and one for the ajax call.

urls.py:

 
    url(r'^$', views.form, name='form'), 
    url(r'^find_cities/$', views.find_cities, name='find_cities'),   

And in the views I make two views which handle the template and the ajax call respectively.

views.py:

 
#   index
def index(request):
    # create context dictionary
    context = {}
    # variables...
    context['form'] = SearchForm()
    return render(request, 'index.html', context)


#   find_cities (ajax processor)   
def find_cities(request, qs=None):
    if qs is None:
        qs = Country.objects.values_list('city', flat=True).all()
    if request.GET.get('country_name'):
        country_name=request.GET.get('country_name')
    # create an empty list to hold the results
    results = []
    qs = Country.objects.values_list('city', flat=True).filter(name=country_name).order_by('city')
    # iterate over each city and append to results list 
    for city in qs:
        results.append(city)
    # if no results found then append a relevant message to results list
    if not results:
        # if no results then dispay empty message
        results.append(_("No cities found")) 
    # return JSON object
    return HttpResponse(simplejson.dumps(results))

The last piece of the puzzle is the ajax call itself. Note that this particular method relies on jQuery being installed/ linked to.

find_cities.js:

// set up a new XMLHttpRequest variable
var request = false;
try {
    request = new XMLHttpRequest();
} catch (trymicrosoft) {
    try {
        request = new ActiveXObject("Msxml2.XMLHTTP");
    } catch (othermicrosoft) {
        try {
            request = new ActiveXObject("Microsoft.XMLHTTP");
        } catch (failed) {
            request = false;
        }
    }
}

if (!request)
    alert("Error initializing XMLHttpRequest!");

function getCity() {
    var countryName = document.getElementById("country").value;
    var url = "http://localhost:8000/collections/find_cities?country_name=" + escape(countryName);
    request.open("GET", url, true);
    request.onreadystatechange = updatePage;
    request.send(null);
} 

// what to do when http ready state changes
function updatePage() { 
    if (request.readyState == 4) {
        if (request.status == 200) {
 
            // get response array
            var data = JSON.parse(request.responseText); 
            update_select($('select[name=city]'), data);
        }  
        // some error checking
        else if (request.status == 404) {
            alert("Request url does not exist");
        }
        else {
            alert("Error: status code is " + request.status);
        }
    }
}   

function update_select(select, data) {
    select.find('option').remove();
    for (var k in data) {
        select.append($('<option value="'+data[k]+'">'+data[k]+'</option>'));
    }
}

This is pretty straightforward AJAX.

1. An XMLHttpRequest() is instantiated, with a special method of instantiating for IE (oh the joy!).

2. Then the getCity() function gets the value selected in the Country field which it then passes along with the url for our ajax view to request.open(). The final parameter, when set to true, requests an asynchronous connection (thus making this Ajax). When the readystate changes the updatePage function is called (note the lack of parenthesis when calling the function.

3. The updatePage() function checks that the readystate has changed to 4 which means the response from the server is complete. It then checks that the request.status is 200 (as opposed to 404 or 500). If all is good then we can grab the JSON data from the request.responseText and pass it to the updateSelect function along with the name of the select target (in this case city).

4. Finally we reach our updateSelect() function which first removes any existing options form the city select field. As a default this holds only the string Not Specified which I defined in the forms.py as empty_label='Not Specified'. The function then iterates through the JSON data array and populates the select function appropriately. Of course if no results were found then there will be only one element in the dump, namely the ‘No cities found’ fallback string.

And that’s it. This clearly lays out the moving parts associated with this common task. Of course this code could be written more concisely. For example we could use only one view + url which handles both the template and the ajax call, where we just pass an argument to the view which makes it execute the ajax call in an if loop, or return the template as a default.

Django extensions

I just started using django-extensions and it is a really simple way to add some really useful features to django.

Installation is really straightforward:

    $ pip install django-extensions

or

    $ easy_install django-extensions

I did also needed to install pygraphviz:

    $ apt-get install python-pygraphviz

Then add django-extensions to the INSTALLED APPS:

INSTALLED_APPS = (
    ...
    'django_extensions',
    ...
)

The first command I wanted to use was the graph_models command which basically creates a graphical relational diagram of the applications in the project. To visualize the whole project with grouping by application:

    $ ./manage.py graph_models -a -g -o my_project.png

and for specific apps:

    $ ./manage.py graph_models my_app | dot -Tpng -o my_app.png

This is a really nice way to let yourself and others visualize the db schema at a glance.

Another insanely useful feature is django shell_plus. Among other things this feature autoloads you models into the shell:

    $ ./manage.py shell_plus

You should see all the models loaded before the prompt.

So no more

    >>> from myapp.models import *

There are plenty of other features described in the documentation, but even in the short time I have played with this package I can tell it will be an indispensable tool in my django toolbox.