Getting Started

Create a Local Consul Datacenter

Now that you have practiced using Consul, it's time to learn a bit more about how Consul operates. In this guide, you'll create your first datacenter with multiple members.

When a new Consul agent starts, it doesn't know about other agents; it is essentially a datacenter with one member. Agents learn about each other in two ways. To add a new agent to an existing datacenter you give it the IP address of any other agent in the datacenter (either a client or a server), which causes the new agent to join the datacenter. Once the agent is a member of the new datacenter, it automatically learns about the other agents via gossip.

In this guide, you will join two agents together to create a two-member Consul datacenter.

»Set up the environment

Consul is a distributed application that is designed to have one agent per machine. To run two agents on the same computer you will need to install VirtualBox, and Vagrant, which will run virtual machines to simulate a distributed environment.

Make a directory to store Vagrant's configuration for this guide.

$ mkdir consul-getting-started-join

Create a new file in the directory called Vagrantfile and paste the content of Consul's demo Vagrant file into it. Then save the file. This file will tell Vagrant to create two virtual machines on your computer with the Consul binary preinstalled.

Boot your two virtual machines. This may take a moment to download everything needed for the environment to spin up.

$ vagrant up
Bringing machine 'n1' up with 'virtualbox' provider...
Bringing machine 'n2' up with 'virtualbox' provider...
...

Once the environment is up, ssh into node 1 to begin configuring of your datacenter.

$ vagrant ssh n1
Linux n1 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have new mail.
vagrant@n1:~$

»Start the agents

In previous guides, you used a single agent in development mode to test Consul's functionality. However, you should never run development agents in production. In this guide you will configure your first Consul agent to run in server mode instead, via the following command line flags. (In production you would provide these settings to consul in a configuration file.)

  • server switch - Providing this flag specifies that you want the agent to start in server mode.

  • -bootstrap-expect flag - This tells the Consul server how many servers the datacenter should have in total. All the servers will wait for this number to join before bootstrapping the replicated log, which keeps data consistent across all the servers. Because you are setting up a one-server datacenter, you'll set this value to 1. You can read more about this process in the bootstrapping guide.

  • -node name - Each node in a datacenter must have a unique name. By default, Consul uses the hostname of the machine, but you'll manually override it, and set it to agent-one.

  • -bind address - This is the address that this agent will listen on for communication from other Consul members. It must be accessible by all other nodes in the datacenter. If you don't set a bind address Consul will try to listen on all IPv4 interfaces and will fail to start if it finds multiple private IPs. Since production servers often have multiple interfaces, you should always provide a bind address. In this case it is 172.20.20.10, which you specified as the address of the first VM in your Vagrantfile.

  • data-dir flag - This flag tells Consul agents where they should store their state, which can include sensitive data like ACL tokens for both servers and clients. In production deployments you should be careful about the permissions for this directory. Find more information in the documentation. You will set the data directory to a standard location: /tmp/consul.

  • config-dir flag - This flag tells consul where to look for its configuration. You will set it to a standard location: /etc/consul.d.

Start your first Consul agent by running the following command. Consul will start up in the foreground of your terminal window.

# vagrant@n1:~
$ consul agent \
  -server \
  -bootstrap-expect=1 \
  -node=agent-one \
  -bind=172.20.20.10 \
  -data-dir=/tmp/consul \
  -config-dir=/etc/consul.d

...

Open a new terminal window and change directories into consul-getting-started-join. Then ssh into your second virtual machine.

$ vagrant ssh n2

Now start up your second Consul agent in client mode. You'll set the bind address to the IP address of the second VM (172.20.20.11, specified in the Vagrantfile) and the name to agent-two. Don't include the -server flag and the agent will start in client mode. Consul will run in the foreground of your terminal.

# vagrant@n2:~
$ consul agent \
  -node=agent-two \
  -bind=172.20.20.11 \
  -enable-script-checks=true \
  -data-dir=/tmp/consul \
  -config-dir=/etc/consul.d
...

Now you have two Consul agents running: one server and one client. The two agents still don't know about each other and each comprise their own single-node datacenters.

Verify this by ssh-ing into each VM and checking each agent's membership information. You'll need to open a new terminal window and change directories into consul-getting-started-join.

Check the membership of agent-two.

$ vagrant ssh n2
Linux n2 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have new mail.
Last login: Fri Aug  2 23:42:33 2019 from 10.0.2.2
# vagrant@n2:~
$ consul members
Node       Address            Status  Type    Build  Protocol  DC   Segment
agent-two  172.20.20.11:8301  alive   client  1.5.3  2         dc1  <default>

Open a new terminal window and change directories into consul-getting-started-join.

Check the membership of agent-one.

$ vagrant ssh n1
Linux n1 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u2 (2019-05-13) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have new mail.
Last login: Fri Aug  2 20:37:46 2019 from 10.0.2.2
# vagrant@n1:~
$ consul members
Node       Address            Status  Type    Build  Protocol  DC   Segment
agent-one  172.20.20.10:8301  alive   server  1.5.3  2         dc1  <all>

»Join the agents

You're now ready to create your multi-agent datacenter. Stay in the terminal window where you are ssh-ed into the first VM, and run the consul join command on the Consul server, giving it the bind address of the Consul client.

# vagrant@n1:~
$ consul join 172.20.20.11
Successfully joined cluster by contacting 1 nodes.

In the same window, run consul members again and you will see both agents listed.

# vagrant@n1:~
$ consul members
Node       Address            Status  Type    Build  Protocol  DC   Segment
agent-one  172.20.20.10:8301  alive   server  1.5.3  2         dc1  <all>
agent-two  172.20.20.11:8301  alive   client  1.5.3  2         dc1  <default>

Switch to the terminal window where your Consul server is running on the first VM, and you'll notice some log output indicating that agent two joined it.

Now switch to the terminal where your client is running on the second VM. You'll notice that the client had been throwing warnings and errors indicating that no servers were available. When the client learned about the server, it stopped throwing errors and synced its node information.

    2019/08/03 00:09:25 [WARN] manager: No servers available
    2019/08/03 00:09:25 [ERR] agent: failed to sync remote state: No known Consul servers
    2019/08/03 00:09:54 [WARN] manager: No servers available
    2019/08/03 00:09:54 [ERR] agent: failed to sync remote state: No known Consul servers
    2019/08/03 00:10:10 [INFO] serf: EventMemberJoin: agent-one 172.20.20.10
    2019/08/03 00:10:10 [INFO] consul: adding server agent-one (Addr: tcp/172.20.20.10:8300) (DC: dc1)
    2019/08/03 00:10:10 [INFO] consul: New leader elected: agent-one
    2019/08/03 00:10:11 [INFO] agent: Synced node info

Consul clients can not function without a server. All datacenters must have at least one agent running in server mode for Consul to function correctly.

In datacenters with more than one server, more than half of the servers must be in communication with each other at all times for the datacenter to function correctly. This is called maintaining quorum. You can learn more about the quorum requirements of servers in the architecture documentation.

Switch to the window where you are ssh-ed onto the second VM and run consul members on the client. The client will also list both agents as members.

# vagrant@n2:~
$ consul members
Node       Address            Status  Type    Build  Protocol
agent-two  172.20.20.11:8301  alive   client  0.5.0  2
agent-one  172.20.20.10:8301  alive   server  0.5.0  2

»Notes on auto-join

In production, new Consul agents should automatically join the datacenter without human intervention. You can configure Consul to automatically discover new agents in AWS, Google Cloud or Azure by adding the relevant cloud auto join object to your Consul configuration file. This will allow a new node to join the datacenter without any hard-coded configuration.

Alternatively, you can provide hard-coded addresses of known Consul agents to new agents using the -join flag or start_join setting.

»Query a node

You can query Consul agents using the DNS interface or HTTP API.

For the DNS API, the structure of the names is NAME.node.consul or NAME.node.DATACENTER.consul. If the datacenter is omitted, Consul will only search the local datacenter.

From the terminal window where you are ssh-ed into agent one query the DNS interface for the address of agent-two.

# vagrant@n1:~
$ dig @127.0.0.1 -p 8600 agent-two.node.consul
...

;; QUESTION SECTION:
;agent-two.node.consul. IN  A

;; ANSWER SECTION:
agent-two.node.consul.  0 IN    A   172.20.20.11

The ability to look up nodes in addition to services is useful for system administration, in addition to service discovery. For example, knowing the address of the node to SSH into is as easy as making the node a part of the Consul datacenter and querying it.

»Stop the agents

Stop both of your agents gracefully by either typing Ctrl-c in the terminal windows where they are running, or issuing the consul leave command.

»Clean up the environment

Vagrant will automatically stop and power down the virtual machines it created, remove their hard disks from your machine, and free up all of the disk space and RAM they consume.

It will not get rid of the directory you created or the Vagrant file it contains, so if you would like to re-run this guide, all you need to do is issue vagrant up again from inside the consul-getting-started-join directory.

Clean up your virtual environment by running the following command from within the consul-getting-started-join directory.

$ vagrant destroy
    n2: Are you sure you want to destroy the 'n2' VM? [y/N] y
==> n2: Forcing shutdown of VM...
==> n2: Destroying VM and associated drives...
    n1: Are you sure you want to destroy the 'n1' VM? [y/N] y
==> n1: Forcing shutdown of VM...
==> n1: Destroying VM and associated drives...

»Next steps

In this guide you set up a multi-agent Consul datacenter, by joining two Consul agents, a server and a client. Continue to the next guide to learn about the operations and development tracks that will help you put Consul into production.