Monday, February 11, 2013

Using Ubuntu cloud-images without a cloud

Since sometime in early 2009, we've put effort into building the Ubuntu cloud images and making them useful as "cloud images". From the beginning, they supported use as an instance on a cloud platform. Initially that was limited to EC2 and Eucalyptus, but over time, we've extended the "Data Sources" that the images support.

A "Data Source" to cloud-init provides 2 essential bits of information that turn a generic cloud-image into a cloud instance that is actually usable to its creator. Those are:
  • public ssh key
  • user-data
Without these, the cloud image cannot even be logged into.

Very early on it felt like we should have a way to use these images outside of a cloud. They were essentially ready-to-use installations of Ubuntu Server that allow you to bypass installation. In 11.04 we added the OVF as a data source and a tool in cloud-init's source tree for creating a OVF ISO Transport that cloud-init would read data from. It wasn't until 12.04 that we improved the "NoCloud" data source to make this even easier.

Available in cloud-utils, and packaged in Ubuntu 12.10 is a utility named 'cloud-localds'. This makes it trivial to create a "local datasource" that the cloud-images will then use to get the ssh key and/or user-data described above.

After boot, you should see a login prompt that you can log into with 'ubuntu' and 'passw0rd' as specified by the user-data provided.

Some notes about the above:
  •  None of the commands other than 'apt-get install' require root.
  •  The 2 qemu-img commands are not strictly necessary. 
    • The 'convert' converts the compressed qcow2 disk image as downloaded to an uncompressed version.  If you don't do this the image will still boot, but reads will go decompression.
    • The 'create' creates a new qcow2 delta image backed by 'disk1.img.orig'. It is not necessary, but useful to keep the '.orig' file pristine. All writes in the kvm instance will go to the disk.img file.
  • libvirt, different kvm networking or disk could have been used. The kvm command above is just the simplest for demonstration. (I'm a big fan of the '-curses' option to kvm.)
  • In the kvm command above, you'll need to hit 'ctrl-alt-3' to see kernel boot messages and boot progress. That is because the cloud images by default send console output to the first serial device, that a cloud provider is likely to log.
  • There is no default password in the Ubuntu images. The password was set by the user-data provided.
The content of 'my-user-data' can actually be anything that cloud-init supports as user-data.  So any custom user-data you have can be used (or developed) in this way.

19 comments:

  1. The cloud-utils link takes you to landscape.net. I think you mean launchpad.net...

    ReplyDelete
  2. Hey Scott, thanks for the great work. Can this be used to seed ubuntu launching on ESXi ? I'm currently launching Ubuntu from template on ESXi and using the powershell cmdlet (InvokeVMScript to customize) but was wondering what are your thoughts.

    ReplyDelete
  3. @Ahmed,
    I'm embarrassingly ignorant on all things vmware. this definitely should work, if you can manage to get the resultant ISO file attached as a CD-ROM or as a disk.

    ReplyDelete
  4. I ran the above script and received the following error when I ran the kvm command:

    Could not initialize SDL(No available video device) - exiting

    ReplyDelete
  5. @dtlewis16,
    If you ran the kvm command above on a remote system (or anywhere without X), then the error you saw is expected. By default, kvm will try to open up a X11 window.

    Try using '-display vnc=:5900' or '-display curses'.

    I'm a big fan of the curses display, but it does take some getting used to. Hit alt-1 to see the tty/vga console, alt-2 to see the qemu monitor, and alt-3 to see the serial console. To exit type 'quit' at the qemu monitor.

    ReplyDelete
  6. I was able to get the cloud image to boot up but the userid "ubuntu" and password "passw0rd" will not log me into the system.

    ReplyDelete
  7. Could we please get these, and any other up-to-date instructions and supplemental information, put in the Ubuntu Wiki 'cloud' image instructions for local use?

    I've wasted a morning trying to figure out how to log-in to an image locally based on the Wiki instructions ( https://help.ubuntu.com/community/UEC/Images ) which, under the title "Natty onwards" describe using "./make-iso ovf-env.xml.tmpl user-data" and so on.

    ReplyDelete
  8. @tj,
    Thanks for pointing out that this content should be on https://help.ubuntu.com/community/UEC/Images . I've updated that page to include it. In the future, please do take the "Its a wiki" philosophy and fix things like that yourself.

    Also, the ovf preseed path "should [still] work". If it didn't work for you feel free to open a bug against cloud-init.

    Thanks for reading and commenting.

    ReplyDelete
  9. does not accept user ubuntu pasword passw0rd

    ReplyDelete
  10. Great Article! Worked as a charm

    Thanks

    ReplyDelete
  11. Fatso here,

    I hope that this note finds you and your family doing well. I have recently started looking into linux again. It has obviously been years since my last dabbling in it. I recently installed linux mint 15 olivia on an old laptop that had been left for dead. Needless to say, it has been resurrected. I am looking to install a new OS on a few newer laptops. What would you suggest for a neophyte like myself?

    Take care,
    Fatso

    ReplyDelete
  12. TrevolutionNow,

    Good to hear from you. Life is wonderful.

    Regarding distribution choice, I sense a trap :). I'm very happily employed by Canonical, the company behind Ubuntu. I'm personally very happy with Ubuntu and would recommend it. Inside of Ubuntu, you basically have 2 choices, 12.04 LTS Precise Pangolin ("Long Term Support") and 13.04 ("Raring Ringtail"). 13.04 is newer and flashier but has security patch support only for 9 months, while 12.04 is supported on desktop packages for 3 years and 5 years on server packages.

    I'd recommend downloading each ISO, putting them on a USB key and giving them a try with that in mind.

    ReplyDelete
  13. Is there a way to create local img in 12.04. Looks like 12.04 does not have cloud-localds; and the ubuntu wiki for 12.04 points to cloud-utils in 12.10(https://help.ubuntu.com/community/UEC/Images)

    ReplyDelete
  14. Ravi,
    If you want to create a local datasource image with on 12.04, I would suggest just doing:

    bzr branch lp:cloud-utils
    sudo cp cloud-utils/bin/cloud-localds /usr/local/bin
    sudo apt-get --assume-yes install genisoimage qemu-utils coreutils

    And then making sure you have the 'genisoimage', 'qemu-utils' and 'coreutils' packages installed. cloud-localds is otherwise pretty stand-alone and written in bash.

    ReplyDelete
  15. This is very interesting. I'm trying to create a local VM that matches our cloud images, but I'm on mac and as far as I know can't use KVM. Currently I'm trying to use veewee with virtualbox. Any suggestions?

    I posted my problems in find that right recipe for all this in the ec2ubuntu google group.

    ReplyDelete
  16. Great, thanks! is there a way to change the default 'ubuntu' user to something else?

    ReplyDelete
  17. @Curvian,
    In 12.04 cloud-init does not create users, the 'ubuntu' user already exists in the images.
    in 12.10 and later, there is no user inside the image, and cloud-init creates it on boot. There, its as easy as specifying a different default user. http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-user-groups.txt

    ReplyDelete
  18. My query is a bit of a deviation from the article, but I wasn't immediately able to find somewhere more apropos.

    A co-worker of mine indicated a desire to either a) provide a pull request for additions/changes to Ubuntu cloud images, or b) a way to provide feedback to request specific additions/changes.

    What would you recommend in this case?

    ReplyDelete