By Jeff Nickoloff

Docker Machine can create and tear down whole fleets of Docker enabled hosts in a matter of seconds. In this article, excerpted from Docker in Action, we discuss the Docker Machine.

The first step in learning about and solving distributed systems problems is building a distributed system. Docker Machine can create and tear down whole fleets of Docker enabled hosts in a matter of seconds. Learning how to use this tool is essential for anyone that wants to learn how to use Docker in distributed cloud or local virtual environments.


Your choice of driver

Docker Machine ships with a number of drivers out of the box. Each driver integrates Docker Machine with a different virtual machine technology or cloud-based virtual computing provider. Every cloud platform has its advantages and disadvantages. There is no difference between a local and remote host from the perspective of a Docker client.

Using a local virtual machine driver like VirtualBox will minimize the cost of running the examples in this article. However, you should consider choosing a driver for your preferred cloud provider instead. There is something powerful about knowing that the commands you’ll issue here are actually managing real world resources, and that the examples you will deploy are going to be running on the Internet. At that point you’re only a few domain specific steps away from building real products.

If you do decide to use a cloud provider for these examples, you will need to configure your environment with the provider specific information (like access key and secret key) as well as substitute driver specific flags in any commands in this article.

You can find detailed information about the driver specific flags by running the, “docker-machine help create” command or consulting online documentation

Building and Managing Docker Machines

This section introduces commands for the Docker-Machine command line program. Since all of these tools are all so similar in form and function, this section will make the introduction through a small set of examples. If you want to learn more about the docker-machine command line you can always use the help command:

docker-machine help

The first and most important thing to know how to do with Docker Machine is to create Docker hosts. The next three commands will create three hosts using the VirtualBox driver. Each command will create a new virtual machine on your computer.

docker-machine create --driver virtualbox host1
docker-machine create --driver virtualbox host2
docker-machine create --driver virtualbox host3

After you run these three commands (they can take a few minutes) you will have three Docker hosts managed by Docker Machine. Docker Machine tracks these machines with a set of files in your home directory (under ~/.docker/machine/). They describe the hosts you have created, the certificate authority certificates used to establish secure communications with the hosts, and a disk image used for VirtualBox based hosts.

Docker Machine can be used to list, inspect, and upgrade your fleet as well. Use the ls subcommand to get a list of managed machines:

docker-machine ls
# Outputs something like:
# host1         virtualbox      Running tcp://
# host2         virtualbox      Running tcp://
# host3         virtualbox      Running tcp://

This command will list each machine, the driver it was created with, its state, and the URL where the Docker daemon can be reached. If you are using Docker Machine to run Docker locally you will have another entry in this list, and that entry will likely be marked as “active.” The active machine is the one that your environment is currently configured to communicate with. Any commands issued with the docker or docker-compose command line interfaces will connect with the daemon on the active machine.

If you want to know more about a specific machine or lookup a specific part of its configuration you can use the inspect subcommand.

docker-machine inspect host1
# Outputs a JSON document describing the machine

docker-machine inspect --format "{{.Driver.IPAddress}}" host1
# Outputs:

The inspect subcommand for docker-machine is very similar to the docker inspect command. You can even use the same Go template syntax to transform the raw JSON document that describes the machine. This example used a template to retrieve the IP address of the machine. If you needed that in practice you would use the ip subcommand.

docker-machine ip host1
# Outputs:

Docker Machine lets you build a fleet with relative ease, and it is important that you can maintain that fleet with the same ease. Any managed machine can be upgraded with the upgrade subcommand.

docker-machine upgrade host3
# Outputs:
# Stopping machine to do the upgrade...
# Upgrading machine host3...
# Downloading ...

# Starting machine back up...
# Starting VM...

The upgrade procedure stops the machine, downloads an updated version of the software, and restarts the machine. With this command you can perform rolling upgrades to your fleet with minimal effort.

You will occasionally have a need to manipulate files on one of your machines or access the terminal on a machine directly. It could be that you need to retrieve or prepare the contents of a bind-mount volume. Other times you might need to test the network from the host, or customize the host configuration. In those cases you can use the ssh and scp subcommands.

When you create or register a machine with Docker Machine, it creates or imports an SSH private key file. That private key can be used to authenticate as a privileged user on the machine over the SSH protocol. The docker-machine ssh command will authenticate with the target machine and bind your terminal to a shell on the machine.

For example, if you wanted to create a file on the machine named, “host1” then you could issue the following commands:

docker-machine ssh host1
# Your terminal is now bound to the shell on host1 touch dog.file
# Exit the remote shell and stop the command exit

It seems a bit silly to use a fully bound terminal to run a single command. If you don’t need a fully interactive terminal you can alternatively specify the command to run as an additional argument to the ssh subcommand. Run the following command to write the name of a dog to the file you just created:

docker-machine ssh host1 "echo spot > dog.file"

If you’ve got files on one machine that you need to copy elsewhere you can use the scp subcommand to do so securely. The scp subcommand takes two arguments, a source host and file, and a destination host and file. Try it for yourself and copy the file you just created from host1 to host2, and then use the ssh subcommand to view it on host2.

docker-machine scp host1:dog.file
host2:dog.file docker-machine ssh host2 "cat
# Outputs: spot

The SSH related commands are critical for customizing configuration, retrieving volume contents, and performing other host related maintenance. The rest of the commands that you need to build and maintains fleets are predictable.

Starting, stopping (or killing), and removing a machine are just like equivalent commands for working with containers. The docker-machine offers four subcommands: start, stop, kill, and rm.

# stop a host docker-machine stop host2

# stop a host immediately docker-machine kill host3

# start a stopped host docker-machine start host2

# stop and remove all three hosts docker-machine rm host1 host2 host3

We just covered the bulk of the basic mechanics for building and maintaining a fleet with Docker Machine. The next section will demonstrate how you can use Docker Machine to configure your client environment to work with those machines and how to access the machines directly.

Configuring Docker Clients to Work with Remote Daemons

Docker Machine accounts for and tracks the state of the machines that it manages. You can use Docker Machine to upgrade Docker on remote hosts, open SSH connections, and securely copy files between hosts. However Docker clients like the docker command line interface or docker-compose are designed to connect to a single Docker host at a time. For that reason one of the most important functions of Docker Machine is producing environment configuration for an, “active” Docker host.

The relationship between Docker Machine, Docker clients, and the environment is illustrated in Figure 1.


Figure 1: The relationship between docker, docker-machine, and relevant state sources.

Get started learning how to manage your Docker environment by creating a couple of new machines and activating one. Start by running create:

docker-machine create --driver virtualbox machine1 docker-machine create --driver virtualbox machine2

In order to activate this new machine, you must update your environment. Docker Machine includes a subcommand named, “env.” The env subcommand attempts to automatically detect the user’s shell and print commands to configure the environment to connect to to a specific machine. In the case that it cannot automatically detect the user’s shell, the specific shell can be set with the --shell flag.

# Let env autodetect your shell docker-machine env machine1

# Get Powershell configuration
docker-machine env --shell powershell machine1

# Get CMD configuration
docker-machine env --shell cmd machine1

# Get fish configuration
docker-machine env --shell fish machine1
# Get the default (POSIX) configuration docker-machine env --shell bash machine1

Each of these commands will print out the list of shell specific commands that need to be run and a comment on how to invoke docker-machine so that these are executed automatically. For example, to set machine1 as the active machine you can execute the docker-machine env command in a POSIX shell:

eval "$(docker-machine env machine1)"

If you use Windows and run Powershell you would run a command like:

docker-machine env --shell=powershell machine1 | Invoke-Expression

You can validate that you’ve activated machine1 by running the active subcommand.

Alternatively, you can check the, “ACTIVE” column on the output from the ls subcommand.

docker-machine active
# Outputs:
# machine1

docker-machine ls
# Note the asterisk in the ACTIVE column:
# machine1 *    virtualbox      Running tcp://
# machine2      virtualbox      Running tcp://

Any client that observes the environment configuration on your local computer will use the Docker Remote API provided at the specified URL for the active machine. When the active machine is changed, so will the targets of any Docker client commands. The state of this environment is illustrated in Figure 2.


Figure 2: One of two machines created with Docker Machine has been activated in the local environment. Docker clients will use the Docker API provided by that machine.

Create a few containers and experience for yourself how simple and subtle it is to work with multiple machines. Start by pulling an image onto the active machine.

docker pull dockerinaction/ch12_painted

Images pulled onto an active machine will only be pulled onto that machine. This means that if there is some common image that you’ll be using across your fleet, you will need to pull that image on each of those machines. This is important to understand if minimizing container startup time is important. In those cases, you will want to pull on as many machines in parallel as possible and before container creation time. Before you start a container with this image change the active machine to machine2 and list the images.

# Replace this with the equivallent command appropriate for your shell eval "$(docker-machine env machine2)"
docker images

The output from the images subcommand should be empty. This example helps illustrate the need to pull images on each machine independently. This machine, machine2, has never had any images installed.

Next, pull the image and run a container for dockerinaction/ch12_painted on machine2.

docker run -t dockerinaction/ch12_painted \

Tiny turtles tenderly tounge tough turnips.

Now switch the active machine back to machine1 and list all containers:

# Replace this with the equivallent command appropriate for your shell eval "$(docker-machine env machine2)"
docker ps -a

Again, the list is empty because no containers have been created on machine1. This simple example should help illustrate how simple it is to work with multiple machines. It should also illustrate how confusing it can be to manually connect to, orchestrate, and schedule work across a sizable fleet of machines. Unless you specifically query for the active host, it is difficult to keep track of where your Docker clients are directed as the number of machines you use increases.

Orchestration can be simplified by taking advantage of Docker Compose, but Compose is like any other Docker client and will only use the Docker daemon that your environment has been configured to use. If you were to launch an environment with Compose and your current configuration where machine1 is active, all of the services described by that environment would be created on machine1. Before moving on clean up your environment by removing both machine1 and machine2.

docker-machine rm machine1 machine2

Docker Machine is a great tool for building and managing machines in a Docker based fleet.