Auto Tag EC2 Spot Instances and Volumes with Boto

Tags are a great way to organize Amazon Web Services (AWS) resources. For example, you can use tags to itemize your AWS bill into different projects. Unfortunately, with spot requests, there is no automatic way to tag the EC2 spot instances and EBS volumes once the spot request has been fulfilled. Here is one way to do that using boto, the Python SDK for AWS.

One of the challenges of using EC2 spot instances is that there is no easy way to tag them and their attached volumes. Using the AWS Management Console, you can tag the spot request, but the tags do not propagate to the related instances and volumes. You have to sit by your computer, click the refresh button every few seconds to wait for the spot instances to be fulfilled before manually tagging them and their volumes.

You can also use EC2 Auto Scaling Groups to propagate the tags from your spot requests to the fulfilled instances. However, the tags do not propagate to the attached volumes. You will still need to tag the volumes manually.

These approaches are not ideal, since I use spot instances quite often for development and testing, and it is a hassle to have to manually tag spot instances and volumes every time I use them. After searching for a while but not finding a good solution, I wrote my own Python script to automate this process, using boto.

Getting Started

You can get the script from my Github gist AWS EC2 Spot Requests with Auto Tagging using Boto.

To use the script, you will need:

  • Python 2.6 or 2.7 (boto does not fully support Python 3 yet)
  • boto (install using pip install boto
  • AWS credentials configured according to the boto configuration guide
  • A JSON configuration file for the spot instance that you want to set up (see below)

This script allows you to create one instance at a time, and use the same set of tags for the spot request, the fulfilled instance and any volumes attached to the instance. You can modify it, however, to create more instances or use different tags for different instances and volumes.

JSON Configuration File

The JSON configuration file that you need to create supplies the tags to create, the configuration of the spot request (arguments to boto.ec2.connection.EC2Connection.request_spot_instances()), and the configuration of any block devices to be attached to the spot instance (arguments to boto.ec2.blockdevicemapping.BlockDeviceType).

The format of the file is shown below. This example shows the full list of configuration values supported by boto, but you can omit most of them to use boto’s default values. In this case, we are requesting an m3.medium instance using the 64-bit, HVM, EBS-backed Amazon Linux AMI (ami-d13845e1). We are assigning it the security key “my-key” and security group “Demo”. We are also requesting two EBS volumes: an 8 GB SSD volume attached to /dev/xvda with the Amazon Linux snapshot and a 2 GB standard (magnetic) volume attached to /dev/xvdb with no snapshot. Finally, we are tagging the instance and both volumes with “Spot Test” as the name, and “Demo” as the project.

{
    "tags": {
        "Name": "Spot Test",
        "Project": "Demo"
    },

    "spot-request": {
        "price": 0.1,
        "image_id": "ami-d13845e1",
        "count": 1,
        "type": "one-time",
        "valid_from": null,
        "valid_until": null,
        "launch_group": null,
        "availability_zone_group": null,
        "key_name": "my-key",
        "security_groups": ["Demo"],
        "user_data": null,
        "addressing_type": null,
        "instance_type": "m3.medium",
        "placement": null,
        "kernel_id": null,
        "ramdisk_id": null,
        "monitoring_enabled": false,
        "subnet_id": null,
        "placement_group": null,
        "instance_profile_arn": null,
        "instance_profile_name": null,
        "security_group_ids": null,
        "ebs_optimized": false,
        "network_interfaces": null,
        "dry_run": false
    },

    "block-device-mapping": {
        "/dev/xvda": {
            "ephemeral_name": null,
            "no_device": false,
            "volume_id": null,
            "snapshot_id": "snap-d15cde24",
            "status": null,
            "attach_time": null,
            "delete_on_termination": true,
            "size": 8,
            "volume_type": "gp2",
            "iops": null
        },
        "/dev/xvdb": {
            "ephemeral_name": null,
            "no_device": false,
            "volume_id": null,
            "snapshot_id": null,
            "status": null,
            "attach_time": null,
            "delete_on_termination": true,
            "size": 2,
            "volume_type": "standard",
            "iops": null
        }
    }
}

Running the Script

If we save this as config.json, we can run the script like this, specifying the EC2 region and the name of the JSON file:

python spot-request.py us-west-2 config.json

When you run the script, it creates the spot request, then waits for it to be fulfilled. Once fulfilled, it immediately tags the instance and any attached volumes with the specified tags. Voila! All done automatically without having to monitor the spot request fulfillment status or manually tag anything.

Run python spot-request.py -h to see other script options.

Advertisements

One thought on “Auto Tag EC2 Spot Instances and Volumes with Boto

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s