Friday, January 7, 2011

Using euca2ools rather than ec2-api-tools with EC2

The Ubuntu UEC Images that Ubuntu produces on EC2 are in every way fully supported, "Official Ubuntu". As with other official releases, access to source code for security and maintenance reasons affects our decisions on what is included.

In the UEC images, the most notable packages left out are 'ec2-api-tools' and 'ec2-ami-tools'. I personally use the ec2-api-tools and ec2-ami-tools quite frequently and Amazon has done a great job with them. However, the license and lack of source code prevents them from being in Ubuntu 'main'.

Fortunately
a.) There are packages made available in the Ubuntu 'multiverse' component.
b.) The euca2ools package is installed by default and provides an almost drop in replacement for the ec2-api-tools and ec2-ami-tools.

I think that many users of EC2 aren't aware of the euca2ools, so I'd like to give some information on how to use them here.

The ec2-api-tools use the SOAP interface and thus use the "EC2_CERT" and "EC2_PRIVATE_KEY". The euca2ools sit on top of the excellent boto project. Boto uses the AWS REST api, which means authentication is done with your "Access Key" and "Secret Key". As a result, configuration is a little different. (Note, bundling images, you still need the EC2_CERT and EC2_PRIVATE_KEY for encryption/signing).

Configuration for euca2ools can be done via environment variables (EC2_URL, EC2_ACCESS_KEY, EC2_SECRET_KEY, EC2_CERT, EC2_PRIVATE_KEY, S3_URL, EUCALYPTUS_CERT) or via config file. I personally prefer the configuration file approach.

Here is my ~/.eucarc that is configured to operate with the EC2 us-east-1 region.

CRED_D=${HOME}/creds/aws-smoser
EC2_REGION="${EC2_REGION:-us-east-1}"
EC2_CERT=${CRED_D}/cert.pem
EC2_PRIVATE_KEY=${CRED_D}/pk.pem
EC2_ACCESS_KEY=ABCDEFGHIJKLMNOPQRST
EC2_SECRET_KEY=UVWXYZ0123456789abcdefghijklmnopqrstuvwx
EC2_USER_ID=950047163771
EUCALYPTUS_CERT=/etc/ec2/amitools/cert-ec2.pem
EC2_URL=https://ec2.${EC2_REGION}.amazonaws.com
S3_URL=https://s3.amazonaws.com:443


Things to note above:
  • euca2ools sources the ~/.eucarc file with bash, and then reads out the values of EC2_REGION, EC2_CERT, EC2_PRIVATE_KEY, EC2_ACCESS_KEY, EC2_USER_ID, EC2_URL, S3_URL. This means that you use other bash functionality in the config file as I've done above with 'EC2_REGION'. This allows me to do something like:

    EC2_REGION=us-west-1 euca-describe-images

  • If there is no configuration file specified with '--config', then those values will be read from environment variables

  • Amazon's public certificate from the ami tools is included with euca2ools in ubuntu, and located in /etc/ec2/amitools/cert-ec2.pem

  • Many of the euca2ools commands will run significantly faster than the ec2-api-tools. The reason for slowness of the ec2-api-tools is their man java dependencies (please correct me if I'm wrong).
  • Your ~/.eucarc file contains credentials and therefore it should be protected with filesystem permissions (ie 'chmod go-r ~/.eucarc').
Hopefully this will make it easier for you to use euca2ools with EC2 on Ubuntu.

3 comments:

  1. Have you ever tried to bundle an AMI intended to be used with Amazon EC2 using euca2ools? I ask because some months ago I filed a bug[1] in Debian about that use case.

    1. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=599464

    ReplyDelete
  2. I sourced eucarc file and when i tried to run the query"euca-describe-instances" I got an error as follows"

    EC2_ACCESS_KEY enviroment variable not set
    Connection failed

    But the environment variable was set when I checked the list by using "env" command.
    Can you suggest a solution for this

    ReplyDelete
  3. @sabvina,
    If you want to source the eucarc file (and put those files into your environment), then you will have to export each of the values above. Ie, you would need to append:

    export EC2_ACCESS_KEY EC2_CERT EC2_PRIVATE_KEY EC2_SECRET_KEY EC2_URL EC2_USER_ID EUCALYPTUS_CERT S3_URL

    However, I recommend instead that you do simply put that in a file in ~/.eucarc and it will be read by the euca2ools, and then those credentials are not in your environment.

    ReplyDelete