Installing Python 3.4 and setting up a virtualenv

As an update to my snippet on installing Python 3, the installation and setup of virtualenvs with Python 3.4 is a little different, namely because python 3.4 ships with its own virtual environment manager, Pyvenv.

1. Download the tarball and extract

$ cd /tmp
$ wget https://www.python.org/ftp/python/3.4.0/Python-3.4.0.tgz
$ tar -xvf Python-3.4.0.tgz

2. Run configure using the –prefix flag to specify installation location

$ ./configure --prefix=/usr/bin/python3.4

3. Make and install

$ make && sudo make install

4. Use the `pyvenv` to make a virtualenv, supplying the location of your virtualenvs

$ /usr/bin/python3.4/bin/pyvenv3.4 ~/.envs/py34

5. Activate your virtualenv

$ . ~/.envs/py34/bin/activate

6. Check Python version

$ python -V
Python 3.4.0

You might want to also make a bash alias to facilitate the creation of virtual environments this way.

UPDATE

I have noticed that often when compiling Python 3.X from source it complains about not being able to find the Bz2 module.

Some very good instructions for solving this are here: http://justanyone.blogspot.ca/2012/07/compiling-python-with-bz2.html

Basically you just need to make Bz2 prior to Python.

Setup a virtualenv for Python3

Just a quick note on using Python 3 in virtualenvs without changing the system python version. Most of this is shamelessly and gratefully copied from here.

1. Check system python version

$ python -V 
Python 2.7.3

2.  Install python 3

$ apt-get install python3 python3-doc

3. Setup a new virtualenv with python3

$ mkvirtualenv --python=/usr/bin/python3 python3

4. Check virtualenv python version

$ python -V
Python 3.2.3

Setting up Django-allauth

So I have wanted a good solution to OpenID authentication in my Django projects for a while now. I had been hearing good things about django-alluth for a little while and it seems that this project had the most traction currently. This presentation was especially useful in making me opt for Django-alluth: https://speakerdeck.com/tedtieken/signing-up-and-signing-in-users-in-django-with-django-allauth

In hindsight I would say that setting up django-allauth for a new project is quite easy, though it did take me a couple of hours of trial and error to get it right. This post is intended to remind me how I did it and maybe help others.

One of the biggest head-scratchers came right at the beginning. Namely, how do I install this thing?? To be honest I’m still not sure I did this correctly.

So first I set my virtualenv, pip installed what I needed and started a new Django project. Aside from my ‘essentials’ (django-extensions, django-debug-toolbar etc.), I pip installed django-allauth, which has it’s own dependencies.

1. So, much like my previous go-to for authentication, django-registration, this package has it’s own templates which will inevitably require customization at some point. So my solution was to grab the django-allauth source code from github and copy just the templates directory into my_project/allauth/. The allauth/ directory is completely empty except for the templates/ directory. This seems to work, and means that I am free to customize those templates as part of my project.

2. Now with django-allauth installed I need to set it up in Django. From my point of view this comprises essentially two setup paths. One, in the Django code so that allauth can function, and the other via the Django admin to setup the various APIs. The second one may seem optional, but without any social app APIs set, there will be no social login, so I would deem it pretty much essential.

So the Django code configuration steps run like this:

Note: more expansive details on this section can be found here.

a) Add the following to settings.py:

TEMPLATE_CONTEXT_PROCESSORS = (
    ...
    "django.core.context_processors.request",
    "allauth.account.context_processors.account",
    "allauth.socialaccount.context_processors.socialaccount",
    ...
)

AUTHENTICATION_BACKENDS = (
    ...
    "django.contrib.auth.backends.ModelBackend",
    "allauth.account.auth_backends.AuthenticationBackend",
    ...
)

INSTALLED_APPS = (
    ...
    'allauth',
    'allauth.account',
    'allauth.socialaccount',
    # ... include the providers you want to enable:
    'allauth.socialaccount.providers.bitly',
    'allauth.socialaccount.providers.dropbox',
    'allauth.socialaccount.providers.facebook',
    'allauth.socialaccount.providers.github',
    'allauth.socialaccount.providers.google',
    'allauth.socialaccount.providers.linkedin',
    'allauth.socialaccount.providers.openid',
    'allauth.socialaccount.providers.persona',
    'allauth.socialaccount.providers.soundcloud',
    'allauth.socialaccount.providers.stackexchange',
    'allauth.socialaccount.providers.twitch',
    'allauth.socialaccount.providers.twitter',
    'allauth.socialaccount.providers.vimeo',
    'allauth.socialaccount.providers.vk',
    'allauth.socialaccount.providers.weibo',
    ...
)

b) Add the aullauth/templates directory to TEMPLATE_DIRS. I have recently started using Unipath, after reading the excellent Two Scoops of Django book.

from unipath import Path

# the number you use as an argument for ancestor may vary
# depending on where your settings.py is located. 
# I have mine in a settings module 
# (e.g. my_project/settings/base.py).

PROJECT_DIR = Path(__file__).ancestor(3)

TEMPLATE_DIRS = (
    ...
    PROJECT_DIR.child("allauth").child("templates"),
    ...
)

c) Add /accounts to the project urls.py

urlpatterns = patterns('',
    ...
    (r'^accounts/', include('allauth.urls')),
    ...
)

d) Now do an initial syncdb. Becasue I want to build all tables initially, I did:

./manage.py syncdb --all

3. So now Django is setup to play nice with django-alluth, I can jump into the django admin and start configuring my app APIs. This should mirror the providers in the INSTALLED_APPS tuple of the settings.

Personally I wanted Google and LinkedIn logins, so I set about obtaining credentials for each of these:

Google

https://code.google.com/apis/console/#:access

LinkedIn

https://www.linkedin.com/secure/developer?newapp=

Twitter

https://dev.twitter.com/apps

Though the interface for registering apps vary, for our purposes we need just a couple of variables:

Client ID: something along the lines of Client ID or API key

Secret Key: the secret key part

These details can be added at localhost:8000admin/socialaccount/socialapp/

I also had to swap out the default site (www.example.com) for localhost:8000 (or wherever the devserver is using), and add it to ‘chosen sites’.

Change social app   Django site admin

Like so.

Hey presto! I can now log in using my LinkedIn or Google credentials.

Image Preloader

This is a neat trick which I found (and subsequently used) here.

Basically this is a small jQuery snippet which goes through your stylesheets and looks for images and caches them, so that they are not loaded when the user interacts with a link, logo, overlay window etc.

Here it is:

function preLoadImages () {
    $.get($('link[rel="stylesheet"]')[0].href, function(data){
        r = /url\(['|"]?(\S+\.(gif|jpg|jpeg|png)[^'(]*)['|"]?\)/ig;
        while (match = r.exec(data)){
            var cacheImage = document.createElement('img');
            cacheImage.src = match[1];
        }
   });
}
preLoadImages();

Getting data into a Django application on Heroku

You can use Django to dump data from your db and git to push the dump to heroku. If the data is sensitive you should take care to remove the data by resetting the head after loading the data in heroku. A more in depth solution can be found here.

1. Dumpdata from dev build.

python manage.py dumpdata --natural --indent 2 > data.json

2. Add data to repo and commit.

git add data.json

git commit -m "Dump data"

3. Push to Heroku.

git push heroku master

4. Load data into Heroku build.

heroku run python manage.py loaddata data.json

Setting up Postgres for Django on Debian Linux

Assumptions: Debian-based linux distro

1. Install postgres

sudo apt-get install postgresql

2. Create a password for the default postgres user

sudo passwd postgres

3. Login as PostGres user

su - postgres

4. Connect to postgres

psql

5. Start Database as root (automatically runs initdb)

service postgresql start

6. Create a role (the Django user in the DATABASES dictionary in settings.py is actually a role)

CREATE ROLE somebody WITH LOGIN PASSWORD 'secret123';

7. Create  a database

create database mydatabase

8. Now configure settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'mydatabase',
        'USER': 'somebody',
        'PASSWORD': 'secret123',
        'HOST': ''  #localhost
    }
}

Done!

Set Up New Relic for a Django project

1. Setup a New Relic account:

http://newrelic.com/signup

2. Install the newrelic app

pip install newrelic

2. Copy your license key

4. Create the new relic config file:

newrelic-admin generate-config LICENSE-KEY newrelic.ini

5. Edit the newrelic.ini file to give your application a name:

app_name = My Python Application

6. Update wsgi.py:

import newrelic.agent
import os

newrelic.agent.initialize('/home/username/path/to/myproject/newrelic.ini')

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings") 

from django.core.wsgi import get_wsgi_application 

application = get_wsgi_application()
application = newrelic.agent.wsgi_application()(application)

6. Verify it’s working:

newrelic-admin validate-config newrelic.ini

7. Start making request to the dev server and log into your New Relic account to see the data!