Docker can now run within Docker

Sep 05 2013


One of the (many!) features of Docker 0.6 is the new “privileged” mode for containers. It allows you to run some containers with (almost) all the capabilities of their host machine, regarding kernel features and device access.

Among the (many!) possibilities of the “privileged” mode, you can now run Docker within Docker itself. First, we will see how to make that happen; next, we will explain what is involved under the hood, and finally, we will show something even more powerful than Docker in Docker!

See Docker-in-Docker in action

If you have Docker 0.6, all you have to do is:

docker run -privileged -t -i jpetazzo/dind

This will download my special Docker image (we will see later why it is special), and execute it in the new privileged mode. By default, it will run a local docker daemon, and drop you into a shell. In that shell, let’s try a classical “Docker 101” command:

docker run -t -i ubuntu bash

Note how the container ID changes as you transition from the container running Docker, to the innermost container!

What’s special in my dind image?

Almost nothing! It is built with a regular Dockerfile. Let’s see what is in that Dockerfile.

First, it installs a few packages: lxc and iptables (because Docker needs them), and ca-certificates (because when communicating with the Docker index and registry, Docker needs to validate their SSL certificates).

The Dockerfile also indicates that /var/lib/docker should be a volume. This is important, because the filesystem of a container is an AUFS mountpoint, composed of multiple branches; and those branches have to be “normal” filesystems (i.e. not AUFS mountpoints). In other words, /var/lib/docker, the place where Docker stores its containers, cannot be an AUFS filesystem. Therefore, we instruct Docker that this path should be a volume. Volumes have many purposes, but in this scenario, we use them as a pass-through to the “normal” filesystem of the host machine. The /var/lib/docker directory of the nested Docker will live somewhere in /var/lib/docker/volumes on the host system.

And of course, the Dockerfile injects the Docker binary in the image, as well as a helper script. The helper script deals with three things.

  1. It ensures that the cgroup pseudo-filesystems are properly mounted, because Docker (or, more accurately, lxc-start) needs them.
  2. It closes extraneous file descriptors which might have been leaked from the parent process. This is not strictly necessary, but you might notice weird side effects if you don’t do it; so I took care of it for you.
  3. It checks if you specified a PORT environment variable through the -e PORT=... command-line option. If you did, the Docker daemon starts in the foreground, and listens for API requests on the specified TCP port. If you did not specify a PORT variable, it will start Docker in the background, and give you an interactive shell.

In the next section, I’ll tell you why I think that this PORT environment variable can be very useful.


If you just want to experiment with Docker-in-Docker, just start the image interactively, as shown above. Now, let’s pretend that you want to provide Docker-as-a-Service. I’m not speaking about Containers-as-a-Service here, but whole Docker instances. Well, each time someone wants their own private Docker instance, just run this:

docker run -privileged -d -p 1234 -e PORT=1234 jpetazzo/dind

Then use docker inspect to retrieve the public port allocated to that container, and give it to your user. They will be able to create containers on this “private Docker” by pointing their Docker client to the IP address and port that you gave them. (See Memcached-as-a-Service for a similar example.)

Note, however, that there are serious security implications there: since the private Docker instances run in privileged mode, they can easily escalate to the host, and you probably don’t want this! If you really want to run something like this and expose it to the public, you will have to fine-tune the LXC template file, to restrict the capabilities and devices available to the Docker instances. In the future, Docker will allow fine-grained permission management; but for now, we think that the ability to switch between “locked down” and “privileged” is a great first step.



Can I Run Docker-in-Docker-in-Docker? Yes. When you are inside a privileged container, you can always nest one more level:

docker run -t -i -privileged jpetazzo/dind

And in the resulting container, you can repeat the process, ad lib.

Also, as you exit nested Docker containers, this will happen (note the root prompts):

root@975423921ac5:/# exit
root@6b2ae8bf2f10:/# exit
root@419a67dfdf27:/# exit
root@bc9f450caf22:/# exit

At that point, you should blast Hans Zimmer’s Dream Is Collapsing on your loudspeakers while twirling a spinning top 😀

It doesn’t work!

While testing Docker-in-Docker in various environments, I found two possible problems.

It looks like the LXC tools cannot start nested containers if the devices control group is not in its own hierarchy. Check the content of /proc/1/cgroup: if devices is standing on a line on its own, you’re good. If you see that another control group is on the same line, Docker-in-Docker won’t work. The wrapper script will detect this situation and issue a warning. To work around the issue, you should stop all running containers, unmount all the control groups, and remount them one by one, each in its own hierarchy.

Also, if you use AppArmor, you need a special policy to support nested containers. If Docker-in-Docker doesn’t work, check your kernel log (with dmesg); if you see messages related to AppArmor, you can start Docker in unconfined mode, like this:

docker run -privileged -lxc-conf="aa_profile=unconfined" -t -i dind

Take Me To Your Repo

The Dockerfile, the wrapper, and some extra documentation is available on my github repository:

About Jérôme Petazzoni


Jérôme is a senior engineer at dotCloud, where he rotates between Ops, Support and Evangelist duties and has earned the nickname of “master Yoda”. In a previous life he built and operated large scale Xen hosting back when EC2 was just the name of a plane, supervized the deployment of fiber interconnects through the French subway, built a specialized GIS to visualize fiber infrastructure, specialized in commando deployments of large-scale computer systems in bandwidth-constrained environments such as conference centers, and various other feats of technical wizardry. He cares for the servers powering dotCloud, helps our users feel at home on the platform, and documents the many ways to use dotCloud in articles, tutorials and sample applications. He’s also an avid dotCloud power user who has deployed just about anything on dotCloud – look for one of his many custom services on our Github repository.

Connect with Jérôme on Twitter! @jpetazzo


21 thoughts on “Docker can now run within Docker

  1. a recursive docker xD

  2. Avatar for Jérôme Petazzoni

    Jérôme Schneider

    Great work dotCloud and Jérôme ! Docker inception !

  3. Where can I buy those “russian” penguins from?

  4. we have to go deeper

  5. Dockerception

  6. Hmm. /proc/filesystems contains aufs inside the container, and /proc/mounts shows that /var/lib/docker is getting special handling, but…

    root@eaa042241959:/# docker run -t -i ubuntu bash
    2013/09/16 20:27:26 POST /v1.4/containers/create
    Unable to find image ‘ubuntu’ (tag: latest) locally
    2013/09/16 20:27:26 POST /v1.4/images/create?fromImage=ubuntu&tag=
    Pulling repository ubuntu
    8dbd9e392a96: Download complete
    b750fe79269d: Download complete
    27cf78414709: Download complete
    2013/09/16 20:28:00 POST /v1.4/containers/create
    2013/09/16 20:28:00 POST /v1.4/containers/719891e9f926/start
    2013/09/16 20:28:00 Kernel does not support AUFS, trying to load the AUFS module with modprobe…
    2013/09/16 20:28:00 Error: Error starting container 719891e9f926: Unable to load the AUFS module

  7. Ahh. Found it — looks like the github README had been updated (lxc.aa_profile rather than just aa_profile) but the blog post hadn’t. Mea culpa.

  8. What is an alternative to using -privileged ? It seems like this would introduce security vulnerabilities if you wanted to create a pseudo-public-facing container

  9. Not that we would want this long term, but as a transition I’d like to run a docker container inside an LXC. Could the same helper script solve this problem? I’m at:
    write /sys/fs/cgroup/devices/lxc/…/docker/…/devices.allow: operation not permitted …. tnx

  10. This sounds really useful, does it require a specific version of Docker?
    I must admit that during a bit of testing I seemed to have delete the files in /sys/fs/cgroup on my host… Which was a bit unpleasant. 🙂

    • OK, my bad, the fact that I messed up my system had no relation to your script or Docker. I messed up the installation of util-linux for nsenter. (note, only do configure and make, *not* make install…)

  11. I have started Tomcat on port 8080 in a container started in a container. Is there a way I can see the app on my host system?

    • Yeah, but you’ll have to map the port of your app-container (-p option) to your host container and map the port of your host container to your “root” host.

      But it supposes you know the port that will be used at each level.

  12. regards,

    First I want to thank me for the information has been useful.

    Today upgrade to 1.6.0 docker, now I have an error when running my container base “FATA [0000] Error response from daemon: Can not use –lxc-conf With execdriver: native-0.2”

    if I remove –lxc-conf and start the container, the container that I run into errors shows that did not have before.

    Please could tell me what would be the solution to the new version.


  13. With latest Docker you’d have to use
    docker run –privileged=true -it jpetazzo/dind

  14. This is a good article as to setup DID.
    I have come up with implementation detail document making docker as dynamic slave for jenkins.

    Appreciate Feedback on same.

  15. Avatar for Jérôme Petazzoni

    Vaibhav Gawali

    Is it possible ro run Windows Docker containers inside another Windows Docker container?

  16. Avatar for Jérôme Petazzoni

    Erik van de Ven

    Suppose this article is worth mentioning! Before using docker-in-docker, think twice if you really want to use it or need it:

  17. Will it worked on Derived images as well ?

    #docker run –privileged -t -i jpetazzo/dind
    #docker commit <ID>
    #docker tag 12bf5b57ac3e abc:99
    #docker run –privileged -t -i abc:99 bash
    (On this new container, i am not able to start a container. Get the error docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Leave a Reply