Introduction

For the reasons exposed in the previous post, I’ve begun exploring Hashicorp’s Nomad.

I am mostly interested in long running services (microservices), and the most useful article I have found out about Nomad dealing with web apps is Introducing cluster schedulers. Unfortunately, this article does not consider any load balancer in front of them, which is mandatory in order to serve several instances of the same app / service. I was not then able to find out any article adding a load balancer in front of a Nomad cluster, but several combining Docker, Consul, Consul Template, Registrator, and Nginx or HAProxy, listed below:

• Using HAProxy and Consul for dynamic service discovery on Docker
• Automatic container registration with Consul and Registrator
• Where are my containers? Dockerized service discovery with Consul
• Scalable Architecture DR CoN: Docker, Registrator, Consul, Consul Template and Nginx
• Service discovery with consul and consul-template
• Load-balancing Docker containers with Nginx and Consul-Template

Among them, I’ve chosen Consul Template and Registrator, as its environment is configured and run by means of Vagrant, like the environment used in the target article on Nomad; it does not matter it’s written in Japanese, as we are currently interested just in a couple commands exposed there. WARNING: Google Translator capitalizes the first letter of every command…

Basically, I have added the VM configured in the latter article, with an HAProxy load balancer, into the set configured in the former one.

Source code available at repo

Explanation

I will depict those configurations, but just providing the raw commands, as the whole explanation on them is supplied in those two excellent articles. Hence, I’d suggest taking a look at both of them before running the following tasks. Commands to run are shown below grouped in sections, each after which of these two articles they were taken from:

Introducing cluster schedulers

cd nomad-intro
vagrant up
vagrant status
vagrant ssh nomad
nomad run /vagrant/microbot.nomad

Consul Template and Registrator

vagrant ssh haproxy1
cat /etc/haproxy/haproxy.cfg

HAProxy config file will then lack any entry at its very bottom for any Web app.

sudo consul-template\
 -consul=10.7.0.15:8500\
 -template=/vagrant/haproxy.ctmpl:/etc/haproxy/haproxy.cfg:"/etc/init.d/haproxy reload"

cat /etc/haproxy/haproxy.cfg

Now, HAProxy config file includes an entry for each of the nine Web apps started by Nomad in the three “Docker” nodes. And invoking curl succesively, it will show the different Docker container IDs; for example:

curl 10.7.0.31

Introducing cluster schedulers

Open the microbot.nomad file with your favorite text editor and change the instance value from 9 to 50. Once you’re done, tell Nomad to about our changes by running:

vagrant ssh nomad
nomad run /vagrant/microbot.nomad

Consul Template and Registrator

Hence, HAProxy config file includes an entry for each of these fifty Web apps started by Nomad in the three “Docker” nodes. And invoking curl succesively, it will show the different Docker container IDs, such as:

curl 10.7.0.31

You can check running services and their corresponding instances by means of Consul:

List running services:

curl -s http://10.7.0.15:8500/v1/catalog/services

List information about our microbot service:

curl -s http://10.7.0.15:8500/v1/catalog/service/microbot-web-microbot

Or straight with the browser at:

http://10.7.0.15:8500/ui/

And that’s it !

Next Steps

From this base config, I plan to explore in the near future the following paths, sharing the experience here with the corresponding articles:

• Load balance with Nginx
• Deploy in AWS with Amazonica
• Monitor the cluster with Riemann
• Provision with Puppet

I hope this information to be useful for you ! And till next post !