love Django and I love Python. However, I also love javascript, Node.js, and I even like playing with Ruby and Ruby on Rails. I love playing around with cool web technology in general. But the the one thing I've always hated was building apache correctly to handle all of these different apps and languages only to have to sit Nginx In front of it. What I really want is a single light weight server that can deal with any application written in any language with out having to build modules, plugins or even recompile the server to get everything to work right. What would be even better is if all of these apps running on a single server could share data freely between each other. Well, my friends - Mongrel2 to the rescue.

You may be familiar with, Mongrel, the ruby web server used in most ruby on rails applications. It was great in concept but took a lot of heat for it's performance and scalability problems. Mongrel2, however, is written in C and uses ZeroMQ as its communication backend. The makes the server, by nature, asyncronous, event driven and language agnostic. You don't configure the server to deal with different languages, they just talk to each other over TCP sockets. If there is a language binding for whatever language your preferred web app is written in, you can run it from Mongrel2! On top of that, because all of your apps with be using ZeroMQ, you can use the same bindings to push data between apps any way you want! But first things first - Lets get Mongrel2 up and running.

Installing Mongrel2

By default mongrel2 wants to install itself into /usr/bin and looks for the ZeroMQ libs in /usr/include. All fine and good for local development. However, if you don't have root permissions, or running in a shared environment, this is less than ideal. We can get around this problem, but it just requires a little set up of the environment. We basically need to tell the C compliler where to look for shared libraries. Assuming you have folders called tmp, bin, lib and include in your home directory, you can do this:

export TMPDIR=$HOME/tmp
export PATH="$HOME/bin:$PATH"
export C_INCLUDE_PATH="$HOME/include:$C_INCLUDE_PATH"
export LIBRARY_PATH="$HOME/lib:$LIBRARY_PATH"
export LD_LIBRARY_PATH="$HOME/lib:$LD_LIBRARY_PATH"

Now we need to install the dependencies for Mongrel - ZeroMQ, SQLite, and we will also need the python bindings for zeromq, pyzmq.

mkdir $HOME/src $HOME/tmp

# install zeromq
cd $HOME/src
wget http://download.zeromq.org/zeromq-2.2.0.tar.gz
tar -xzf zeromq-2.2.0.tar.gz
cd zeromq-2.2.0
./configure --prefix=$HOME
make
make install

Setup Python bindings for ZeroMQ. If you have multiple versions of python, make sure to run setup.py with the appropriate version.

# install pyzmq
cd $HOME/src
wget https://github.com/zeromq/pyzmq/downloads/pyzmq-2.1.10.tar.gz
tar -xzf pyzmq-2.1.10.tar.gz
cd pyzmq-2.1.10
python setup.py install --zmq=$HOME

Setup a local Sqlite ( optional if already installed )

# install sqlite
cd $HOME/src
wget http://www.sqlite.org/sqlite-autoconf-3071300.tar.gz
tar -xzf sqlite-autoconf-3071300.tar.gz
cd sqlite-autoconf-3071300
./configure --prefix=$HOME
make
make install

Install Mongrel2

# install mongrel2
cd $HOME/src
git clone https://github.com/zedshaw/mongrel2.git
PREFIX=$HOME make all install

This will install two executable files, mongrel2 & m2sh. The mongrel is it would sound, the server binary itself. m2sh is the command line tool for configuring and working with the environment mongrel runs in.

Install WSGID

So the last piece to the setup is to install WSGID. By default, the install script for wsgid will try to pull its dependencies from PyPi and run their default installation which will look for things under /usr/bin and /usr/include. This will, of course, fail as they aren't installed there. So we need to do a custom install of wsgid which will look like so:

export CPPFLAGS="-I$HOME/include $CPPFLAGS"
export LDFLAGS="-L$HOME/lib $LDFLAGS"

git clone https://github.com/daltonmatos/wsgid.git
cd wsgid
export DESTINATION=$HOME
export PYTHONVER=python2.7export PYTHONPATH=$DESTINATION/lib/$PYTHONVER 
$PYTHONVER setup.py install --root=$DESTINATION --install-data=lib/$PYTHONVER --install-lib=lib/$PYTHONVER --install-scripts=bin

Validate Install

We just installed a bunch of stuff in a rather manual way. Lets make sure that everything wound up where we wanted it. From the python shell you should be able to now do this with out any errors

import wsgid
import zmq
print zmq
print wsgid

To test out Mongrel2, try the following from your terminal

#test mongrel install
mkdir $HOME/mongrel_server
cd $HOME/mongrel_server
mkdir run logs tmp
cp /path/to/mongrel2/source/tests/config.sqlite .
m2sh servers

This should out put something like the following:

SERVERS:
------
mongrel2.org localhost 5dc1fbe7-d9db-4602-8d19-80c7ef2b1b11

Remember again, to use the appropriate version of python you want to run Your application under. An important note about WSGID, after having tried to setup Mongrel2 & WSGID with a couple of applications, it became painfully obvious that wsgid doesn't support python2.5. Mainly because it uses the new string formatting syntax that was introduced in python2.6 & the included json module over the external simplejson package. Simplejson is easy to install, but the string formatting is not such an easy fix. It would require patching quite a sizable chunk of the library. Bummer! I don't think a string formatting syntax is a reason to cut out an entire version of python.

Configure Mongrel2

Alright, your computer didn't melt! We are in good shape. Lets get this thing running. To do that we are going to need to configure a new server instance for mongrel2. Mongrel2 uses a sqlite database to store all of the configurations you may use, and all of the configurations as basically python files that describe how the mongrel2 instance should serve your applications. The simple example provided with the mongrel2 source code( sample.conf ) looks like this:

main = Server(
    uuid="f400bf85-4538-4f7a-8908-67e313d515c2",
    access_log="/logs/access.log",
    error_log="/logs/error.log",
    chroot="./",
    default_host="localhost",
    name="test",
    pid_file="/run/mongrel2.pid",
    port=6767,
    hosts = [
        Host(name="localhost", routes={
            '/tests/': Dir(base='tests/', index_file='index.html',
                             default_ctype='text/plain')
        })
    ]
)

Pretty easy! You basically tell mongrel where to dump its log files, stash a .pid file and a port to bind to. The hosts array & Host is the bit that actually tells  the server how to run your application. In this example it is just a Dir ( basically a file server ) that is mounted to /tests/. So anything incoming requests to /tests/* will just get returned the requested file. ( Note - The UUIDs, don't actually need to be UUIDs, they just need to be a unique string as far as the mongrel is concerned ) What we are going to need to do is define a Host that knows how to get and send information through ZeroMQ. For that we will need to modify our server config. Assuming we are still in the mongrel_server directory we created earlier:

#define a handler for Django & ZeroMQ
django_server = Handler(send_spec='tcp://127.0.0.1:9997',
                       send_ident='34f9ceee-cd52-4b7f-b197-88bf2f0ec378',
                       recv_spec='tcp://127.0.0.1:9996', recv_ident='')

#define server routes
routes = {
    '/': django_server
}

#server definition
main = Server(
    uuid="f400bf85-4538-4f7a-8908-67e313d515c2",
    access_log="/logs/access.log",
    error_log="/logs/error.log",
    chroot=".",
    pid_file="/run/mongrel2.pid",
    default_host="codedependant",
    name="main",
    port=9000,
    hosts=[
        Host(name="codedependant", routes=routes)
    ]
)

#tell mongrel what to run
servers = [main]

That is it! Mongrel2 is ready. Now just reload the new configuration into the config DB.

m2sh load -config server.conf

The really cool thing about this is that the server, Mongrel2, doesn't really care anything about the application you want to run. When you want to add a different application in a different language, you just define a new Host on a different set of ports. Done! As long as there is a ZeroMQ binding for the language you need, you can run it.

Running Django

We have mongrel configured correctly. Lets set up a django application to run on WSGID and Mongrel2. The first thing we need to do is let wsgid set up the folder structure it wants to run with

cd $HOME/mongrel_server
wsgid init --app-path=./djangoproject

this will create something like this:

|-- ~/
|-- mongrel_server/
|   |-- djangoproject/
|   |   |-- apps/
|   |   |-- logs/
|   |   `-- pid/

wsgid expects your applications to live in the app folder. so you can either create a new django project in that folder or move an existing project in to the app folder.

cd djangoproject/app
pip install Django==1.3
python django-admin.py startproject myapp

Now we need to start the mongrel server with the configuration we just created

cd $HOME/mongrel_server
m2sh start -name main

This will start mongrel using the "main" server Simple enough. However, this alone won't run our django application, we need to start up wsgid to convert the Messages from ZeroMQ into proper requests/responses to Django and vice versa. To start  wsigd we initialize wsgid with the path to the app folder and tcp URI that map in the opposite directions of our django_server handler defined above

wsgid --app-path=./junkbone --recv=tcp://127.0.0.1:9997 --send=tcp://127.0.0.1:9996 --workers=4

Notice that in the server definition, it sends to tcp://127.0.0.1:9997 and our wsgid later listens to tcp://127.0.0.1:9997. The same logic applies in to opposing direction. You just want to make sure that the server sends/receives to URI your application is receives/sense.

And that is it! Your django application is running, direct your browser to http:/127.0.0.1:9000 and you should be greeted with the It Works Django start page. Their are a couple of down side from what I gather in my limited use of wsgid.  Obviously, this should be more of a production set up.

The Django development server still provides a lot of conveniences to the development process that you don't get with this set up. However, in a production set up, Mongrel2 blows apache out of the water in terms of performance, flexibility, and it's ability to scale. Once you have it running, adding applications and language support is only as difficult as getting the application running it self. Mongrel2 is for all intensive purposes plug-n-play.