Today I would like to demonstrate the use of Consul to help achieve loose coupling and discovery of services, two important principles of service-oriented architecture (SOA) present in modern, elastic infrastructures.
If you have ever designed an architecture to be deployed in a cloud computing environment, then you are probably already familiar with the idea that you should design for failure. Having the expectation that one or more components of your infrastructure may fail at any given time forces you to design your services in such a way that they are loosely coupled from one another. This usually means running multiple instances of web, application, and database servers, and making use of load balancers, message queues and / or APIs for communication between them.
The environment is scaled horizontally, by adding more instances to the pool. Because of this, instances need to be able to advertise the service they provide, and discover providers of other services.
Where Does Consul Fit In?
Consul’s introduction page does an excellent job of describing the features it provides and what its basic architecture looks like, but I’ll provide a brief summary here.
Consul is a tool for discovering and configuring services in your infrastructure. At its core, it provides service discovery, health checking, and a key/value store. It supports multiple data centers without having to add a layer of abstraction. Nodes that provide services run a Consul agent which talks to one or more Consul servers. Services or nodes can be discovered by querying any of the Consul servers or agents. Data is exposed through DNS and / or HTTP interfaces.
Its capabilities overlap with a number of different types of software including: service discovery systems such as ZooKeeper and etcd; configuration management systems such as Puppet or Chef; and monitoring systems such as Nagios or Sensu. Again, there is an excellent page on the Consul website that describes how it compares with these tools.
The Demo Environment
For my Consul demonstration, I elected to use a combination of Docker and Vagrant. Docker because it makes it easy for me to run multiple lightweight containers on the same machine, and Vagrant because it gives me a great deal of flexibility in building and controlling the containers I will be creating. If you are unfamiliar with either of these tools, allow me to provide a brief overview.
Docker
Per Wikipedia, Docker “.. is an open-source project that automates the deployment of applications inside software containers”. It consists of: Docker Engine, a container that runs on the host operating system; and Docker Hub, a cloud service similar to GitHub which allows users to share containers and automate workflows.
Docker makes use of the following features of the Linux kernel: cgroups (or control groups), which make it possible to limit and isolate resource usage (CPU, memory, etc) of process groups; and namespace isolation, where process groups’ view of operating environment resources such as process trees, network, and file systems are isolated from one another.
Vagrant
Vagrant is a tool for building complete, reproducible development environments, with a focus on automation. It started out as essentially a ‘wrapper’ for VirtualBox, but now supports VMware, Docker, kvm/libvirt and Amazon EC2 as providers, as well as a number of provisioners including Puppet, Chef, and Docker.
Vagrant was created by Mitchell Hashimoto, who initially maintained it in his free time. After it went on to become wildly successful, he formed HashiCorp so he could work on the product full time. HashiCorp has since released other products including Packer, Serf, Consul, and most recently Terraform.
Installing The Tools
Docker
Ubuntu 14.04 LTS includes Docker in its universe repository under the name docker.io and can therefore be installed this way:
$ sudo apt-get update $ sudo apt-get install docker.io
Keep in mind that this may not be the latest version of Docker. If you would like to try the latest version, you will need to add the Docker repository key and add the repository to apt’s sources list. A script has been made available to automate this process for you, so all that is necessary is to run the following:
$ curl -sSL https://get.docker.io/ubuntu/ | sudo sh
On Fedora, Docker is provided by the package named docker-io. If you are running Fedora 19 and have the (unrelated) docker package installed, you will need to remove it before continuing.
$ sudo yum -y remove docker
With Fedora 21 and later, the docker package’s functionality is provided by another package named wmdocker, and it does not conflict with docker-io. Use the following command to install Docker.
$ sudo yum -y install docker-io
Whichever operating system you are running Docker on, you will likely want to be able to use the commands as your regular, non-privileged user, without having to elevate privileges to root. Therefore, you will probably want to make yourself a member of the docker group so you have access to the socket file used by the various Docker commands.
$ sudo usermod -a -G docker your_name
Vagrant
For Vagrant, you will need at least version 1.6.0 as that is when the Docker provider was introduced. This demonstration was tested with version 1.6.3. To install Vagrant, visit its download page and obtain the appropriate package for your operating system. You can install the package on Ubuntu using the following command:
$ sudo dpkg -i vagrant_1.6.3_x86_64.deb
That’s it. In the next section, we will install Consul and continue with setting up our cluster.
Setting Up The Consul Cluster
Let’s begin by establishing the Consul cluster, which will be used by nodes to register themselves as providers of services, and can be queried to discover which services are being provided.
The first step is to download and install Consul on the host which will be running the Docker containers.
$ wget https://dl.bintray.com/mitchellh/consul/0.3.1_linux_amd64.zip $ unzip -d /usr/local/bin/ 0.3.1_linux_amd64.zip
Now we can start a Consul server that will bind to the IP address of the docker0 interface on your host.
$ consul agent -server -bootstrap -bind=172.17.42.1 -client=172.17.42.1 -data-dir /tmp/consul
This server will elect itself leader of the cluster (since it is currently the only member), and is what the rest of the Consul servers will connect to when joining. Ultimately, the architecture will look like this:
Before we can bring up the rest of the Consul servers, however, we’ll need to do some prep work. As mentioned above, the intent is to use Vagrant with the Docker provider to create this environment. Therefore, let’s begin by creating a working directory to contain our Vagrantfile.
$ mkdir -p vagrant/consul && cd vagrant/consul $ vagrant init
The contents of Vagrantfile should look like this:
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
JOIN_IP = ENV['JOIN_IP']
# A hash of containers to define.
# These will be the Consul cluster members.
consul_members = [ "consul1", "consul2", "consul3" ]
consul_members.each do |member|
config.vm.define member do |consul_config|
# Use Docker provider
consul_config.vm.provider "docker" do |docker|
docker.name = member
docker.image = 'progrium/consul'
docker.cmd = [ "-server", "-node=#{member}", "-join=#{JOIN_IP}" ]
end
end
end
end
What should be self evident is that Vagrant is assigning the value of an environment variable called JOIN_IP to a local variable with the same name, and then enumerating through an array of three Consul members which it will create using the Docker provider.
What might not be so obvious, however, are the docker.image and docker.cmd lines, so I will explain them in greater detail.
docker.image = 'progrium/consul'
This line tells Docker to launch (or ‘run’) an instance of the progrium/consul image as found on the Docker Hub Registry. It is a small container based on BusyBox used to run Consul. The project’s source page can be found on GitHub if you’re interested in learning more. The next line:
docker.cmd = [ "-server", "-node=#{member}", "-join=#{JOIN_IP}" ]
is an array of strings which is used to build a custom command to run on the container. Vagrant will perform string interpolation on the member and JOIN_IP variables, replacing them with the current member’s name and the IP address that was provided via the JOIN_IP environment variable. The end result is that the container runs a command such as this:
/bin/consul agent -config-dir=/config -server -node=consul1 -join=172.17.42.1
Let’s see it in action by telling Vagrant to create our containers.
$ JOIN_IP=172.17.42.1 vagrant up --provider=docker
Bringing machine 'consul1' up with 'docker' provider...
Bringing machine 'consul2' up with 'docker' provider...
Bringing machine 'consul3' up with 'docker' provider...
==> consul2: Creating the container...
consul2: Name: consul2
consul2: Image: progrium/consul
consul2: Cmd: -server -node=consul2 -join=172.17.42.1
consul2: Volume: /home/bfraser/vagrant/consul:/vagrant
consul2:
consul2: Container created: d85fbfacdb45cabc
==> consul2: Starting container...
==> consul2: Provisioners will not be run since container doesn't support SSH.
==> consul1: Creating the container...
consul1: Name: consul1
consul1: Image: progrium/consul
consul1: Cmd: -server -node=consul1 -join=172.17.42.1
consul1: Volume: /home/bfraser/vagrant/consul:/vagrant
==> consul3: Fixed port collision for 22 => 2222. Now on port 2200.
==> consul3: Creating the container...
consul3: Name: consul3
consul3: Image: progrium/consul
consul3: Cmd: -server -node=consul3 -join=172.17.42.1
consul3: Volume: /home/bfraser/vagrant/consul:/vagrant
consul1:
consul1: Container created: 413dfa1a63c94bcc
==> consul1: Starting container...
==> consul1: Provisioners will not be run since container doesn't support SSH.
consul3:
consul3: Container creaited: fb54d80e8ce58a46
==> consul3: Starting container...
==> consul3: Provisioners will not be run since container doesn't support SSH.
This created three containers and substituted the node name and join address as expected. Now let’s see what Docker reports.
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES fb54d80e8ce5 progrium/consul:latest /bin/start -server - About a minute ago Up About a minute 53/udp, 8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 8400/tcp, 8500/tcp consul3 413dfa1a63c9 progrium/consul:latest /bin/start -server - About a minute ago Up About a minute 53/udp, 8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 8400/tcp, 8500/tcp consul1 d85fbfacdb45 progrium/consul:latest /bin/start -server - About a minute ago Up About a minute 53/udp, 8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 8400/tcp, 8500/tcp consul2
There are in fact three containers running. Other details are provided but they aren’t relevant at this point, I just wanted to show you how to view the status of your newly created Docker containers.
Now if we check Consul, we should see each of the containers listed as members of the cluster (note: we can’t just run consul members as we need to tell it which IP address to use as the RPC address).
$ consul members -rpc-addr=172.17.42.1:8400 Node Address Status Type Build Protocol laptop 172.17.42.1:8301 alive server 0.3.0 2 consul2 172.17.0.5:8301 alive server 0.3.0 2 consul1 172.17.0.6:8301 alive server 0.3.0 2 consul3 172.17.0.7:8301 alive server 0.3.0 2
As you can see, we have successfully created a cluster of Consul servers, comprised of an initial server running on the host and three servers running as Docker containers created by Vagrant. In my next post, we will see how to add Consul clients providing services (‘providers’), register the services, and query available services from clients (‘consumers’).
1 Comment. Leave new
it require to comment the line “#config.vm.box = “base” in Vagrantfile to successfully execute the command “JOIN_IP=172.17.42.1 vagrant up –provider=docker”
Very good blog. Thanks for sharing.