Managing infrastructures in the cloud, with lessons learned the hard way

Here is a collection of blog posts I wrote over the last 3 years or so. Some of them are practical step-by-step tutorials on using various tools for managing cloud instances, while others talk about lessons learned the hard way, by deploying large-scale infrastructures in the cloud. I am aggregating them here for ease of future reference:

Lessons learned

• Experiences deploying a large-scale infrastructure in Amazon EC2 (April 2009)
• Lessons learned from deploying a production database in EC2 (April 2011)
• Dark launching and other lessons from Facebook on massive deployments (July 2009)
• You're not a cloud provider if you don't provide an API (February 2009)

Working with EC2-specific tools

• Experiences with Amazon EC2 and EBS (September 2008)
• Update on EC2 and EBS (October 2008)
• Deploying EC2 instances from the command line (December 2008)
• Working with Amazon EC2 regions (December 2008)

Load balancing (ELB and HAProxy)

• Using AWS Elastic Load Balancing with a password-protected site (January 2011)
• Load balancing in Amazon EC2 with HAProxy (February 2009)
• HAProxy, X-Forwarded-For, GeoIP, KeepAlive (March 2009)
• HAProxy and Apache performance tuning tips (March 2009)

Working in the multi-cloud with libcloud

• Using libcloud to manage instances across multiple cloud providers (December 2010)
• libcloud 0.4.2 and SSL (January 2011)
• Passing user data to EC2 Ubuntu instances with libcloud (January 2011)
• Slides for 'Working in the multi-cloud with libcloud' presentation (March 2011)

Backups to S3

• Incremental encrypted backup to Amazon S3 using duplicity (May 2008)

Mail server setup in EC2

• Setting up a mail server in EC2 with postfix and Postini (May 2010)

Rackspace CloudFiles

• Managing Rackspace CloudFiles with python-cloudfiles (September 2010)