Wednesday, August 14, 2013

lxc with fast cloning via overlayfs and userdata via clone hooks

Serge Hallyn and Stéphane Graber made some really nice improvements to LXC in the last few months.  These include:
  • user namespaces, which will bring us secure containers in 14.04 and the ability to safely run containers without root. 
  • a library with bindings for C, go and python3.
  • cloning with overlayfs
  • hooks executed at clone time.
I had previously worked with Ben Howard on the 'ubuntu cloud' template, and I just finished some updates to it that take advantage of overlayfs and clone hooks to provide a great environment to use or test cloud-init.

Previously the ubuntu cloud template (which downloads a cloud image to create a container) allowed the user to specify userdata or public keys at container creation time.  The change was really just to move the container customization code to a clone hook.

Thanks to the daily build ppa, you can do this on any release from 12.04 to 13.10.

Hopefully the example below explains this better.  The times reported are from my Thinkpad X120e, which is a netbook class cpu and slow disk.  Times clearly will vary, and these are not meant to be scientific results.
If you do not see the embedded file below, please read the remainder of this post in the gist on github.
###
### Fast cloning with overlayfs, and specifying user-data on clone
### blog post:
### http://ubuntu-smoser.blogspot.com/2013/08/lxc-with-fast-cloning-via-overlayfs-and.html
###
### Eventually, this should make it into 13.10 and the stable lxc ppa
### https://launchpad.net/~ubuntu-lxc/+archive/stable
### But right now you'll have to use the daily ppa.
$ sudo apt-add-repository -y ppa:ubuntu-lxc/daily
$ sudo apt-get update --quiet
$ sudo apt-get install --assume-yes --quiet lxc
### Now, create a pristine 'source' container which we'll clone repeatedly
### The create will download a tarball of the root filesystem from
### http://cloud-images.ubuntu.com and extract it, so this will take some
### time. Subsequent create's will use the cached download.
$ sudo lxc-create -n source-precise-amd64 -t ubuntu-cloud -- \
--release=precise --arch=amd64
### Compare to clone and delete a container with and without overlayfs
###
$ TIMEFORMAT=$'real: %3R user: %3U system: %3S'
### First the old method.
$ time sudo lxc-clone -o source-precise-amd64 -n test1
Created container test1 as copy of source-precise-amd64
real: 29.842 user: 17.392 system: 21.616
$ time sudo lxc-destroy -n test1
real: 2.766 user: 0.184 system: 2.528
## Now using overlayfs snapshot
$ time sudo lxc-clone --snapshot -B overlayfs -o source-precise-amd64 -n test1
Created container test1 as snapshot of source-precise-amd64
real: 0.143 user: 0.024 system: 0.044
$ time sudo lxc-destroy -n test1
real: 0.044 user: 0.008 system: 0.028
###
### Its clear that the clone and destroy were more than a little bit faster.
### 29 seconds to 0.14 seconds, and 2.8 to 0.04 seconds respectively.
###
### Now, lets see about performance of booting a system, and demonstrate
### passing user-data.
###
### You can see options you can pass to 'clone' for the ubuntu-cloud
### clone with '/usr/share/lxc/hooks/ubuntu-cloud-prep --help'
### The most useful are probably '--userdata' and '--auth-key'
### use a user-data script that just powers off to time boot to shutdown
$ printf "%s\n%s\n" '#!/bin/sh' '/sbin/poweroff' > my-userdata
### clone then start without overlayfs
$ sudo lxc-clone -o source-precise-amd64 -n p1 \
-- --userdata=my-userdata
$ time sudo lxc-start -n p1
<4>init: hwclock main process (6) terminated with status 77
real: 14.137 user: 10.804 system: 1.468
### clone then start with overlayfs
$ sudo lxc-clone -o source-precise-amd64 --snapshot -B overlayfs -n p2 \
-- --userdata=my-userdata
$ time sudo lxc-start -n p2
<4>init: hwclock main process (6) terminated with status 77
...
* Will now halt
real: 12.489 user: 10.944 system: 1.672
### So, we see above that overlayfs start/stop was a bit faster.
### I think those differences are inside the realm of noise, but
### they do demonstrate that at least for this test there was not
### a huge cost for the benefit of overlayfs.