Writing a reverse proxy/loadbalancer from the ground up in C, part 1: a trivial single-threaded proxy

This is the first step along my road to building a simple C-based reverse proxy/loadbalancer so that I can understand how nginx / OpenResty works -- more explanation here. It's called rsp, for Really Simple Proxy. This version listens for connections on a particular port, specified on the command line; when one is made it sends the request down to a backend -- another server with an associated port, also specified on the command line -- and sends whatever comes back from the backend back to the person who made the original connection. It can only handle one connection at a time -- while it's handling one, it just queues up others, and it handles them in turn. This will, of course, change later.

I'm posting this in the hope that it might help people who know Python, and some basic C, but want to learn more about how the OS-level networking stuff works. I'm also vaguely hoping that any readers who code in C day to day might take a look and tell me what I'm doing wrong :-)

[ Read more ]

Writing a reverse proxy/loadbalancer from the ground up in C, part 0: introduction

We're spending a lot of time on nginx configuration at PythonAnywhere. We're a platform-as-a-service, and a lot of people host their websites with us, so it's important that we have a reliable load-balancer to receive all of the incoming web traffic and appropriately distribute it around backend web-server nodes.

nginx is a fantastic, possibly unbeatable tool for this. It's fast, reliable, and lightweight in terms of CPU resources. We're using the OpenResty variant of it, which adds a number of useful modules -- most importantly for us, one for Lua scripting, which means that we can dynamically work out where to send traffic as the hits come in.

It's also quite simple to configure at a basic level. You want all incoming requests for site X to go to backend Y? Just write something like this:

    server {
        server_name X
        listen 80;

        location / {
            proxy_set_header Host $host;
            proxy_pass Y;
        }
    }

Simple enough. Lua scripting is pretty easy to add -- you just put an extra directive before the proxy_pass that provides some Lua code to run, and then variables you set in the code can be accessed from the proxy_pass.

But there are many more complicated options. worker_connections, tcp_nopush, sendfile, types_hash_max_size... Some are reasonably easy to understand with a certain amount of reading, some are harder.

I'm a big believer that the best way to understand something complex is to try to build your own simple version of it. So, in my copious free time, I'm going to start putting together a simple loadbalancer in C. The aim isn't to rewrite nginx or OpenResty; it's to write enough equivalent functionality that I can better understand what they are really doing under the hood, in the same way as writing a compiler for a toy language gives you a better understanding of how proper compilers work. I'll get a good grasp on some underlying OS concepts that I have only a vague appreciation of now. It's also going to be quite fun coding in C again. I've not really written any since 1997.

Anyway, I'll document the steps I take here on this blog; partly because there's a faint chance that it might be interesting to other experienced Python programmers whose C is rusty or nonexistent and want to get a view under the hood, but mostly because the best way to be sure you really understand it is to try to explain it to other people.

I hope it'll be interesting!

Here's a link to the first post in the series: Writing a reverse proxy/loadbalancer from the ground up in C, part 1: a trivial one-shot proxy.

New gadget: Nokia N900

I got an early Christmas present today! Robert was able to find a Nokia N900 at a very reasonable price, and Lola decided to get it for me as combined Christmas and birthday presents...

The is a fascinating device. Nokia bill it as a "mobile computer"; it has about the CPU power of a 10-15-year-old PC (say, a Pentium II), but also has a modern graphics processor, and it runs Maemo, Nokia's rebranded version of Debian Linux. Earlier this month, Robert showed me his one running his own port of Frotz, and demonstrated that it had a full version of Linux; I saw the video of it running WebGL later on the same evening, and fell in love :-)

One thing I've noticed, though, is that although you can get a full Debian install running, it takes a little work and there's no good step-by-step documentation. I'll post about that next.

Dear lazyweb: what is is about Linux and WPA?

I don't think I really have enough of a readership for this blog to get an answer to this, but... perhaps someone passing by will know. Why is WPA support invariably so bad in Linux-based OSes?

Three examples:

1. Despite having been a Linux user since 1992, I held off on switching my primary laptop, an aging Fujitsu-Siemens machine, from Windows for many years because no distribution I could find supported WPA out of the box with any kind of user interface I was willing to spend the time learning. This is not because I'm lazy -- I'm willing to put in weeks to learn web server configuration details, and years to learn programming languages. These things are inherently difficult and cannot be reasonably simplified. But connecting to an encrypted wireless network? Sure, the underlying tech is as complex as it gets, but if Microsoft can make it easy to use then why did it take so long for anything usable to get into any Linux distro? Sigh. Well, anyway, I eventually tried out Ubuntu 7.10, Gutsy Gibbon, and was delighted to discover that its support was OK. Not as good as Windows XP's, but good enough for me to switch the old lappy over, which I did sometime last autumn, and have never regretted. So, standard defined in 2004 or so, decent support in mid-2007. Hmm.

2. So along came my new Eee, just before Christmas, and for some reason I have difficulties connecting to my work or home networks. Why? Well, I banged my head against it for a while, and eventually discovered that there was a bug in the scripts used for WPA connections that meant it could not connect if you had a key with a space in it. A bit of digging around on the Eee Wiki helped me sort that one out, but seriously...

3. Anyhow, I posted earlier on today that I'd not been able to connect my OLPC XO to my work or home networks. No prizes for guessing what the problem was... from the Laptop.org wiki (found via):

   WPA is not working from the User Interface in Build 650! Some OLPCs (including in the G1G1 program) cannot connect to wireless routers protected with the WPA protocol. Although the networks are visible from the OLPC, the stall when user enters a password.

   The first solution they suggest is to switch off WPA. Right. Sorry, but I'm not Bruce Schneier, and there's no way I'm switching to WEP. It's almost enough to send me into some kind of Fake Steve Jobsesque rant.

   Luckily, some kind soul has worked out a hack to work around the problem, so I'll try that. [UPDATE, posted from XO: it works!]

But seriously -- what is the problem? Linux is a great OS, but WPA support seems to be seriously messed up. It can't just be a driver problem, because both the XO and the Eee have OSes built for them by the hardware manufacturers. Does anyone out there know?