The other day, I was sitting with some of the awesome folks at Docker in the hopes of getting my feet wet with docker internals. Anil Madhavapeddy suggested that I start by building my own containerd and successfully running a container there, the idea being to determine if the README (of containerd) was good enough. I ran this all in AWS on a clean install of the latest Ubuntu AMI. All of the below commands were run as the default user (ubuntu).
To begin with, I figured I would need Go and Git. I downloaded the latest version of Go from the website to ensure I had the latest version from 1.6 (which was the best choice at the time of writing this, YMMV).
wget https://storage.googleapis.com/golang/go1.6.2.linux-amd64.tar.gz
Then I installed make, gcc, git:
sudo apt-get install make git
Now I was ready to check out the sources.
Important: If you are working on a forked version of containerd, ensure that in $GOPATH your code is still in github.com/docker/containerd, as the Makefile assumes the source will be found there.
mkdir -p $GOPATH/src/github.com cd $GOPATH/src/github.com git clone https://github.com/docker/containerd cd containerd make sudo bin/containerd
If things work, you should see something like this (but with ansi colors!):
ubuntu@ip-172-30-0-65:~/go/github.com/docker/containerd$ sudo bin/containerd WARN[0000] containerd: low RLIMIT_NOFILE changing to max current=1024 max=4096
The next step is to run an OCI bundle (which is the same concept as a docker image, just using the new open standard). I used redis, as recommended in the containerd README. I needed to prepare the filesystem (that will be used inside of the container) as follows:
sudo mkdir -p /containers/redis/rootfs sudo docker pull redis sudo docker create --name tempredis redis sudo docker export tempredis | tar -C redis/rootfs -xf - sudo docker rm tempredis
To generate the config used for the OCI bundle, I used the runc binary compiled from the runc github repository. It is also possible to download a binary directly, but given that I’ve just built containerd from scratch and have the toolchain on hand, I built from source:
mkdir -p $GOPATH/src/github.com/opencontainers cd $GOPATH/src/github.com/opencontainers git clone https://github.com/opencontainers/runc cd runc
At this point I discovered that I also needed the libseccomp C headers for the new seccomp stuff:
apt-get install libseccomp-dev
And now I could finally build runc:
make
If all goes well (and thankfully it did), I’ll have an executable runc. At this point, I went back to /containers to prepare the OCI bundle config:
cd /containers $GOPATH/src/github.com/opencontainers/runc/runc spec
That will give a spec, which is the config needed to run the container. However, if I try to run this with ctr, I’ll still get an error:
ubuntu@ip-172-30-0-65:/containers/redis$ sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers start redis /containers/redis [ctr] rpc error: code = 2 desc = containerd-shim not installed on system
At this point I discovered that if containerd-shim and runc (good thing we got runc to make the spec file!) aren’t present in $PATH, I need to tell containerd where they are. So kill containerd, and relaunch as follows:
sudo ./containerd --shim $GOPATH/src/github.com/docker/containerd/bin/containerd-shim --runtime /home/ubuntu/godev/src/github.com/opencontainers/runc/runc
And then try again:
sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers start redis /containers/redis sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers list ID PATH STATUS PROCESSES redis /containers/redis running init
Success!
Since it’s actually running sh (as I didn’t change the config yet), I can’t actually do anything with it yet (since there’s currently no ctr attach functionality).
To actually interact, I could relaunch and attach, or create a second process and attach:
sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers exec --id redis -a -t --pid 1 --cwd / -u 0 -g 0 # id uid=0(root) gid=0(root) groups=0(root) # exit
To see that it really works, I ask to become the “redis” user:
sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers exec --id redis -a -t --pid 1 --cwd / -u 999 -g 999 /bin/sh $ id uid=999(redis) gid=999(redis) groups=999(redis) $ exit
To kill it, I have to (rather forcefully) terminate init:
sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers kill redis --pid init --signal 9 ubuntu@ip-172-30-0-65:/containers/redis$ sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers ID PATH STATUS PROCESSES
To actually run redis inside, I want to change some information in the config.json:
Specifically, process.args needs to change from ["sh"] to ["redis-server", "--bind", "0.0.0.0"] and process.user from {} to {"uid":999,"gid":999}
Then launch again:
sudo $GOPATH/src/github.com/docker/containerd/bin/ctr containers start redis /containers/redis
This successfully gives us a container running redis. I still didn’t touch any networking that may be needed to expose redis, nor did I play with the new seccomp functionality (which is in the latest version), but this was a good starting point (and to be honest, everyone was leaving and the cleaning crew kicked me out of the room).