Reverse proxying HTTP and WebSockets with virtual hosts using nginx and tcp_proxy_module

I spent today trying to work out how we could get PythonAnywhere to support WebSockets in our users' web applications. This is a brief summary of what I found, I'll put it in a proper post on the PythonAnywhere blog sometime soon...

We use nginx, and it can happily route HTTP requests through to uwsgi applications (which is the way we use it) and can even more happily route them through to other socket-based servers running on specific ports (which we don't use but will in the future so that we can support Twisted, Tornado, and so on -- once we've got network namespacing sorted).

But by default, nginx does not support reverse proxying WebSockets requests. There are various solutions to this posted around the net, but they don't explain how to get it working with virtual hosts. I think that this is because they're all a bit old, because it's actually quite easy once you know how.

(It's worth mentioning that there are lots of cool non-nginx solutions using excellent stuff like haproxy and hipache. I'd really like to upgrade our infrastructure to use one of those two. But not now, we all too recently moved from Apache to nginx and I'm scared of big infrastructure changes in the short term. Lots of small ones, that's the way forward...)

Anyway, let's cut to the chase. This excellent blog post by Johnathan Leppert explains how to configure nginx to do TCP proxying. TCP proxying is enough to get WebSockets working if you don't care about virtual hosts -- but because arbitrary TCP connections don't necessarily have a Host: header, it can't work if you do care about them.

However, since the post was written, the nginx plugin module Johnathan uses has been improved so that it now supports WebSocket proxying with virtual hosts.

To get nginx to successfully reverse-proxy WebSockets with virtual host support, compile Nginx with tcp_proxy_module as per Johnathan's instructions (I've bumped the version to the latest stable as of today):

export NGINX_VERSION=1.2.4
curl -O http://nginx.org/download/nginx-$NGINX_VERSION.tar.gz
git clone https://github.com/yaoweibin/nginx_tcp_proxy_module.git
tar -xvzf nginx-$NGINX_VERSION.tar.gz
cd nginx-$NGINX_VERSION
patch -p1 < ../nginx_tcp_proxy_module/tcp.patch
./configure --add-module=../nginx_tcp_proxy_module/
sudo make && make install

Then, to use the new WebSockets support in tcpproxymodule, put something like this in your nginx config:

worker_processes  1;

events {
    worker_connections  1024;
}

tcp {
    upstream site1 {
        server 127.0.0.1:1001;

        check interval=3000 rise=2 fall=5 timeout=1000;
    }

    server {
        listen 0.0.0.0:80;
        server_name site1.com;

        tcp_nodelay on;
        websocket_pass site1;
    }
}


tcp {
    upstream site2 {
        server 127.0.0.1:1002;

        check interval=3000 rise=2 fall=5 timeout=1000;
    }

    server {
        listen 0.0.0.0:80;
        server_name site2.com;

        tcp_nodelay on;
        websocket_pass site2;
    }
}

Hopefully that's enough to help a few people googling around for help like I was this morning. Leave a comment if you have any questions!

Running Django unit tests on PythonAnywhere

I was working on a side project today, a Django app hosted at PythonAnywhere. While writing some initial unit tests, I discovered a confusing bug. When you try to run the tests for your app, you get an error message creating the database (for the avoidance of doubt, USERNAME was my PA username):

18:57 ~/somewhere (master)$ ./manage.py test
Creating test database for alias 'default'...
Got an error creating the test database: (1044, "Access denied for user 'USERNAME'@'%' to database 'test_USERNAME$default'")
Type 'yes' if you would like to try deleting the test database 'test_USERNAME$default', or 'no' to cancel: no
Tests cancelled.

The problem is that PythonAnywhere users don't have the privileges to create the database test_USERNAME$default (whose name Django's unit testing framework has auto-generated from the USERNAME$default that is the DB name in settings.py). PA only allows you to create new databases from its web interface, and also only allows you to create databases that are prefixed with your-username$

After a bit of thought, I realised that you can work around the problem by setting TEST_NAME in settings.py to point to a specific new database (say, USERNAME$unittest) and then creating a DB of that name from the MySQL tab. Once you've done that, you run the tests again; you get an error like this:

19:02 ~/somewhere (master)$ ./manage.py test
Creating test database for alias 'default'...
Got an error creating the test database: (1007, "Can't create database 'USERNAME$unittest'; database exists")
Type 'yes' if you would like to try deleting the test database 'USERNAME$unittest', or 'no' to cancel:

You just enter "yes", and it will drop then recreate the database. This works, because when you created it from the MySQL page, the settings were set up correctly for you to be able to create and drop it again in the future. Once this has been done once, tests run just fine in the future, with no DB errors.

Obviously we'll be fixing this behaviour in the future (though I can't offhand see how...). But there's the workaround, anyway.

PythonAnywhereAnywhere

We recently added something cool to PythonAnywhere, our Python online IDE and web hosting environment -- if you're writing a tutorial, or anything else where you'd find a Python console useful in a web page, you can use one of ours! Check it out:

What's particularly cool about these consoles (apart from the fact that they advertise the world's best Python IDE-in-a-browser) is that they keep the session data on a per-client basis -- so, if you put one on multiple pages of your tutorial, the user's previous state is kept as they navigate from page to page! The downside (or is it an upside?) is that this state is also kept from site to site, so if they go from your page to someone else's, they'll have the state they had when they were trying out yours.

Bug or feature? Let me know what you think in the comments...

Teaching programming

One thing we wanted to do with PythonAnywhere, our Python online IDE and web hosting environment, was put together a short introduction to Python for non-programmers. I wrote the first cut the other day.

I've always been a fan of the Pimsleur language lessons. Unlike very traditional ways of teaching foreign languages, they don't make you learn vocabulary lists and grammatical rules. Unlike more modern systems, they don't try to teach you phrases. There's no written textbook, just a bunch of CDs (or these days MP3s). And they throw you right in at the deep end.

At the start of each Pimsleur course, a soothing voice says something like "Welcome to Pimsleur Klingon part one, lesson one. Listen to the following conversation." And then you hear something that sounds like this. The soothing voice comes back and says "in 30 minutes you'll hear that again, and you'll understand it." And then you're introduced to the different sentences, and told to repeat stuff. "Repeat this: tlhab 'oS 'Iw HoHwI' So' batlh" they'll say, and you'll stumble your way through it. Then they'll get you to repeat tlhab on its own a few times. For complicated words they'll break it down, and get you to repeat the last bit first, gradually building up. I. wI. oHwI. HoHwI. After 30 minutes of this, your brain is aching, and then they play the conversation you heard at the start -- and it makes perfect sense!

Quite incredible.

So what's this all got to do with teaching programming? Well, we were looking for a decent "Python for non-programmers" course that we could use, and while we found a couple, they all seemed to have the same problem -- they'd work by gradually introducing simple concepts, and after twenty pages you might have learned enough to write a trivial program.

That's not how any of us learned programming. Instead, we tended to pick it up bit by bit by inspection of complete working bits of code. For me, it was typing in listings from the home computing magazines that I bought. (Note for younger readers: listings were printed programs in magazines in the days when disks were too expensive to give away with magazines. (Note for even younger readers: disks were primitive USB-stick-like things that people used to use to store data, and were often attached to magazines as a way of passing data from magazine to reader in the days when you couldn't just put a URL in the article text. (Note for yet younger readers: magazines were collections of very thin slices of wood on which text and images were "printed" using chemical dyes, which could be purchased in places called shops. They were popular in the Dark Ages before the Facebook.)))

That way of learning -- which must be even easier now that any aspiring programmer can look at the source of tons of open source software, or just view the source of any web page -- was quick, efficient, and got you straight to a position where you felt you'd achieved something. You'd written your first program! You could change the bits you understood, and leave the rest for a later day when you knew more. A great way to learn, as well as being excellent training for later days when you find yourself maintaining some horrific codebase written by someone who thinks that operator overloading is the best thing since sliced bread.

So, my theory for the first part of this tutorial -- and, if it works out, for any following ones -- is that each should start with a program listing that the reader won't understand yet, but should be able to understand with a little explication. Core concepts should be drilled in by relentless repetition. No baby-talk -- start using the technical terms straight away, and then repeat them enough that if the reader forgets what "variable" means then they can work it out from context. And aim to get them from knowing nothing to simple OO and functional programming in five half-hour lessons.

What do you think?

Resolver is hiring

We’re looking for a Software Developer to work on PythonAnywhere, Resolver One, and our other products, based in our offices in Clerkenwell, London, UK. More information on the Resolver jobs page.

Busy, busy, busy

A couple of weeks back we were brainstorming about other ways we could make use of the code infrastructure we'd put together for Dirigible. We had loads of stuff for running functional tests, determining dependencies between spreadsheet cells, executing untrusted user code safely on our servers, and so on. Any of those could potentially make an interesting product, so we put together some basic landing pages, one for each idea, and put a bit of money into Google AdWords to see if any of them got any interest.

One of them took off immediately, and even started getting traction on Twitter: PythonAnywhere, an online Python IDE and web application environment -- basically, Dirigible without the spreadsheet grid. This fits in with what we suspected -- lots of people were interested in Dirigible, but it wasn't the spreadsheet side of it that excited them, it was the easy Python grid computing.

What's been particularly cool with this idea is not only that most of it is done and "just" needs breaking out of Dirigible and putting into a new product, but that people are keen to engage with us about it. When people signed up on our landing page, we sent them an email with a few questions -- "What would you use it for? Which features excite you? What would you pay for it? Any suggestions for other features?" About 25% of people have replied, with lots of great feedback, and we've changed our plans (and altered the relative priorities of features) based on their input. All very Lean Startup...

Anyway, all good clean fun. If you'd like a look at it when it goes into beta, you can sign up on the site, or just leave a comment below.