Unhelpful AWS Tip 1 - Compute Instances, Not Computers

I've been hacking around on AWS for five years now, and am starting to compile a list of tips and tricks to help make the most of things. I'm hoping to use this series of posts to help me clarify my own thinking, and if you find any of this helpful, even better.

When working in a bare metal environment, I'd likely provision a new box and hook it up to a Chef Server or Puppet Master. I'd spend most of my time thinking about my on-instance configuration management, as that matters quite a bit since these boxes are going to hang around for a long time.

This isn't the right way to approach EC2. In the AWS model, you want your EC2 instances to be stateless and thrown away often. You should be pushing state onto dedicated services such as Heroku Postgres or Amazon RDS / DynamoDB. Your computer instances should be as thin as possible, focusing as much of their compute power on the task at hand.

It's useful to think of EC2 instances as nothing more than dynamic compute containers awaiting instruction. For example, take a look at this small ruby script:

#!/usr/bin/env ruby

require 'aws-sdk'

metadata = {
  image_id:      'ami-9aaa1cf2',
  instance_type: 't2.micro',
  subnet:        'subnet-a55587fc',
  key_name:      'ops',
  slug_url:      'https://s3-us-west-1.amazonaws.com/example-slugs/demo.tar.gz',
  cmd:           'bin/web'
}

script=<<ENDSCRIPT
#!/bin/bash

set -exo pipefail

apt-get update
apt-get install -y runit ruby

# our hero
useradd -r app
mkdir -p /srv/app
chown app:app -R /srv/app

# fetch the app
su - app -c 'curl -s #{metadata[:slug_url]} | tar xvz -C /srv/app'

# write our runit script
mkdir -p /etc/sv/app
cat >> /etc/sv/app/run <<EOF
#!/bin/bash

cd /srv/app
exec chpst -u app #{metadata[:cmd]}
EOF
chmod +x /etc/sv/app/run

# write our runit logger script
mkdir -p /etc/sv/app/log
mkdir -p /var/log/app
cat >> /etc/sv/app/log/run <<EOF
#!/bin/bash
exec svlogd -t /var/log/app
EOF
chmod +x /etc/sv/app/log/run

# symlink to turn our app on
ln -s /etc/sv/app /etc/service/app
ENDSCRIPT

ec2 = AWS::EC2.new
i = ec2.instances.create({
  image_id:                             metadata[:image_id],
  instance_type:                        metadata[:instance_type],
  subnet:                               metadata[:subnet],
  associate_public_ip_address:          true,
  key_name:                             metadata[:key_name],
  instance_initiated_shutdown_behavior: 'terminate',
  user_data:                            script
})

puts "launched #{i.id}..."

Let's walk through this:

And that's about it. If you were to visit port 5000 with a browswer, you'd find a 'Hello World' web app running.

The Good Parts

Room for Improvement

Things that immediately jump to mind:

In Summary

The more you treat EC2 instances like bare metal computers, the more you'll hate your life. This is a proven ratio, embedded deep within the universe. I'm pretty sure it was featured in a recent Dan Brown novel. Treat them like single-task compute instances, and you'll find yourself working with something more like Lego Blocks. Start treating complexity as a smell (as God intended), and watch your life improve.

Future posts will likely dig into improvements and higher level concerns. I'll probably also go all hipster on you and demonstrate how docker could be applied here.