Introducing cloud-init's cloud-config syntax

Since this is my first post on a brand new blog, *and* I'm fairly new to the Ubuntu community, I figure I should introduce myself. My name is Scott Moser, I've been a member of the Ubuntu Server team for the past 9 months or so. The majority of my time has been focused on Ubuntu's cloud efforts, both on ec2 and on the Ubuntu Enterprise Cloud (UEC).

Thats enough personal introduction, now on to the content.

The cloud-init package provides "first boot" functionality for the Ubuntu UEC images. It is in charge of taking the generic filesystem image that is booting and customizing it for this particular instance. That includes things like:

• setting the hostname
• putting the provided ssh public keys into ~ubuntu/.ssh/authorized_keys
• running a user provided script or otherwise modifying the image

If you have used the Official Ubuntu Images for Hardy or Karmic, you may be aware that the above functionality was previously provided by ec2-init. The cloud-init is package is largely a "cloud agnostic" replacement for ec2-init. The AWS specific portion of the name didn't fit with UEC at the moment, and seemed limiting for the future. We hope to have it working on other cloud offerings as well.

Setting hostname and configuring a system so the person who launched it can actually log into it are not terribly interesting. The interesting things that can be done with cloud-init are made possible by data provided at launch time called user-data.

ec2-init, cloud-init, and the Alestic images support customization through user-data in one very simple yet effective manner. If the user-data starts with '#!', then it will be stored and executed as root late in the boot process of the instance's first boot (similar to a traditional 'rc.local' script). Output from the script is directed to the console. For example:

$ cat ud.txt
#!/bin/sh
echo ========== Hello World: $(date) ==========
echo "I have been up for $(cut -d\  -f 1 < /proc/uptime) sec"

$ ec2-run-instances ami-a908e7c0 --key mykey.us-east-1 \
   --user-data-file=ud.txt
# wait now for the system to come up and console to be available

$ ec2-get-console-output i-97fc7afc | grep --after-context=1 Hello
========== Hello World: Mon Mar 29 18:05:05 UTC 2010 ==========
I have been up for 28.26 sec

The simple approach shown above gives a great deal of power. The user-data can contain a script in any language where an interpreter already exists in the image (#!/bin/sh, #!/usr/bin/python, #!/usr/bin/perl, #!/usr/bin/awk ... ).

For many cases, the user may not be interested in writing a program. For this case, cloud-init provides "cloud-config", a configuration based approach towards customization. To utilize the cloud-config syntax, the supplied user-data must start with a '#cloud-config'. For example:

$ cat cloud-config.txt
#cloud-config
apt_upgrade: true
apt_sources:
- source: "ppa:smoser/ppa"

packages:
- build-essential
- pastebinit

runcmd:
- echo ======= Hello World =====
- echo "I have been up for $(cut -d\  -f 1 < /proc/uptime) sec"

$ ec2-run-instances ami-a908e7c0 --key mykey.us-east-1 \
   --user-data-file=cloud-config.txt

Now, when the above system is booted, it will have:

• added my personal ppa
• run an upgrade to get all updates available
• installed the 'build-essential' and 'pastebinit' packages
• printed a similar message to the script above

The 'runcmd' commands are run at the same point in boot that the '#!' script would run in the previous example. It is present to allow you to get the full power of a scripting language if you need it without abandoning cloud-config.

Note, that in this case the fairly large amount of output to the console from 'apt-get upgrade' ended up scrolling our 'Hello World' message off the ec2-console buffer, so it didn't appear there. That is something that will need to be addressed in lucid+1.

For more information on what kinds of things can be done with cloud-config, see doc/examples in the source.

cloud-init supports a couple other formats of user-data which provide more customization possibilities. I hope to write another blog entry covering those other formats soon.