Printing to the cloud

A movie is worth 10,000 words...

Using AWS Elastic Load Balancing with a password-protected site

Scenario: you have a password-protected site running in EC2 that you want handled via Amazon Elastic Load Balancing. The problem with that is that the HTTP healthchecks from the ELB to the instance hosting your site will fail because they will get a 401 HTTP status code instead of 200. Hence the instance will be marked as 'out of service' by the ELB.

My solution was to serve one static file (I called it 'check.html' containing the text 'it works!') without password protection.

In my case, I have nginx handling both the dynamic app (which is a Django app running on port 8000) and the static files. Here are the relevant excerpts from nginx.conf (check.html is in /usr/local/nginx/static-content):

http {
    include       mime.types;
    default_type  application/octet-stream;

    upstream django {
        server 127.0.0.1:8000;
    }

    server {
        listen       80;

        location / {
            proxy_pass http://django/;
            auth_basic            "Restricted";
            auth_basic_user_file  /usr/local/nginx/conf/.htpasswd;
        }

        location ~* ^.+check\.html$
        {
            root   /usr/local/nginx/static-content;
        }
    }
}

Passing user data to EC2 Ubuntu instances with libcloud

While I'm on the topic of libcloud, I've been trying to pass user data to newly created EC2 instances running Ubuntu. The libcloud EC2 driver has an extra parameter called ex_userdata for the create_node method, and that's what I've been trying to use.

However, the gotcha here is that the value of that argument needs to be the contents of the user data file, and not the path to the file.

So...here's what worked for me:

1) Created a test user data file with following contents:

#!/bin/bash

apt-get update
apt-get install -y munin-node python2.6-dev
hostname coolstuff

2) Used the following script to create the node (I also created a keypair which I passed to create_node as the ex_keypair argument):

#!/usr/bin/env python

import os, sys
from libcloud.types import Provider 
from libcloud.providers import get_driver 
from libcloud.base import NodeImage, NodeSize, NodeLocation
 
EC2_ACCESS_ID     = 'MyAccessID'
EC2_SECRET_KEY    = 'MySecretKey'
 
EC2Driver = get_driver(Provider.EC2) 
conn = EC2Driver(EC2_ACCESS_ID, EC2_SECRET_KEY)

keyname = sys.argv[1]
resp = conn.ex_create_keypair(name=keyname)
key_material = resp.get('keyMaterial')
if not key_material:
    sys.exit(1)
private_key = '/root/.ssh/%s.pem' % keyname
f = open(private_key, 'w')
f.write(key_material + '\n')
f.close()
os.chmod(private_key, 0600)

ami = "ami-88f504e1" # Ubuntu 10.04 32-bit
i = NodeImage(id=ami, name="", driver="")
s = NodeSize(id="m1.small", name="", ram=None, disk=None, bandwidth=None, price=None, driver="")
locations = conn.list_locations()
for location in locations:
    if location.availability_zone.name == 'us-east-1b':
        break

userdata_file = "/root/proj/test_libcloud/userdata.sh"
userdata_contents = "\n".join(open(userdata_file).readlines())

node = conn.create_node(name='tst', image=i, size=s, location=location, ex_keyname=keyname, ex_userdata=userdata_contents)
print node.__dict__

3) Waited for the newly created node to get to the Running state, then ssh-ed into the node using the key I created and verified that munin-node and python2.6-dev were installed, and also that the hostname was changed to 'coolstuf'.

# ssh -i ~/.ssh/lc1.pem ubuntu@domU-12-31-38-00-2C-3B.compute-1.internal

ubuntu@coolstuff:~$ dpkg -l | grep munin
ii  munin-common                      1.4.4-1ubuntu1                    network-wide graphing framework (common)
ii  munin-node                        1.4.4-1ubuntu1                    network-wide graphing framework (node)

ubuntu@coolstuff:~$ dpkg -l | grep python2.6-dev
ii  python2.6-dev                     2.6.5-1ubuntu6                    Header files and a static library for Python

ubuntu@coolstuff:~$ hostname
coolstuff

Anyway....hope this will be useful to somebody one day, even if that somebody is myself ;-)

libcloud 0.4.2 and SSL

Libcloud 0.4.2 was released yesterday. Among its new features is an important one: SSL certificate validation is now supported when opening a connection to a cloud provider. However, for this to work, you have to jump through a couple of hoops.

1) Python 2.5 doesn't have the ssl module installed (2.6 does) -- so you need to install it from PyPI. The current version for ssl is 1.15.

2) By default, SSL cert validation is disabled in libcloud.

If you open a connection to a provider you get:

/usr/lib/python2.5/site-packages/libcloud/httplib_ssl.py:55:
UserWarning: SSL certificate verification is disabled, this can pose a
security risk. For more information how to enable the SSL certificate
verification, please visit the libcloud documentation.
 warnings.warn(libcloud.security.VERIFY_SSL_DISABLED_MSG)

To get past the warning, you need to enable SSL cert validation and also provide a path to a file containing common CA certificates (if you don't have that file, you can download cacert.pem from http://curl.haxx.se/docs/caextract.html for example). Add these lines before opening a connection:

import libcloud.security
libcloud.security.VERIFY_SSL_CERT = True
libcloud.security.CA_CERTS_PATH.append("/path/to/cacert.pem")

As an aside, the libcloud wiki page on SSL is very helpful and I used it to figure out what to do.