HashiConf
Join us this September for 3 days of talks, training, product news & more. Book Your Ticket Now

Getting Started

Register a Service and Health Check - Service Discovery

In the previous guide you ran a local Consul agent, and checked for other members of the datacenter. In this guide you will start using Consul by registering a service and a health check.

One of the major use cases for Consul is service discovery. Consul provides a DNS interface that downstream services can use to find the IP addresses of their upstream dependencies.

Consul knows where these services are located because each service registers with its local Consul client. Operators can register services manually, configuration management tools can register services when they are deployed, or container orchestration platforms can register services automatically via integrations.

In this guide, you'll register a service and health check manually by providing Consul with a configuration file, and use Consul discover its location using the DNS interface and HTTP API. Manually registering a service will help you understand the information that your automation tooling will ultimately need to provide Consul in order to take advantage of service discovery.

Defining a Service

You can register services either by providing a service definition, which is the most common way to register services, or by making a call to the HTTP API. Here we will use a service definition.

First, create a directory for Consul configuration. Consul loads all configuration files in the configuration directory, so a common convention on Unix systems is to name the directory something like /etc/consul.d (the .d suffix implies "this directory contains a set of configuration files").

$ mkdir ./consul.d

Next, write a service definition configuration file. Pretend there is a service named "web" running on port 80. Use the following command to create a file called web.json in the configuration directory. This file will contain the service definition: name, port, and an optional tag you can use to find the service later on. (In this case copy the whole code block except for the $ to run the command and create the file.)

$ echo '{"service":
  {"name": "web",
   "tags": ["rails"],
   "port": 80
  }
}' > ./consul.d/web.json

Now, restart the agent, using command line flags to specify the configuration directory and enable script checks on the agent.

$ consul agent -dev -enable-script-checks -config-dir=./consul.d
==> Starting Consul agent...
           Version: 'v1.5.2'
           Node ID: '82f64bfa-22c2-5727-0f5d-0bae376f6584'
         Node name: 'Judiths-MBP.lan'
        Datacenter: 'dc1' (Segment: '<all>')
            Server: true (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false

==> Log data will now stream in as it occurs:

...

2019/07/16 14:09:25 [INFO] agent: Synced service "web"
2019/07/16 14:09:25 [DEBUG] agent: Node info in sync
2019/07/16 14:09:25 [DEBUG] agent: Service "web" in sync
2019/07/16 14:09:25 [DEBUG] agent: Node info in sync

You'll notice in the output that Consul "synced" the web service. This means that the agent loaded the service definition from the configuration file, and has successfully registered it in the service catalog.

In a multi-agent Consul datacenter, each service would register with its local Consul client, and the clients would forward the registration to the Consul servers, which maintain the service catalog.

If you wanted to register multiple services, you could create multiple service definition files in the Consul configuration directory.

Querying Services

Once the agent adds the service to Consul's service catalog you can query it using either the DNS interface or HTTP API.

DNS Interface

First query the web service using Consul's DNS interface. The DNS name for a service registered with Consul is NAME.service.consul, where NAME is the name you used to register the service (in this case, web). By default, all DNS names are in the consul namespace, though this is configurable.

The fully-qualified domain name of the web service is web.service.consul. Query the DNS interface (which Consul runs by default on port 8600) for the registered service.

$ dig @127.0.0.1 -p 8600 web.service.consul

; <<>> DiG 9.10.6 <<>> @127.0.0.1 -p 8600 web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28340
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.        IN  A

;; ANSWER SECTION:
web.service.consul. 0   IN  A   127.0.0.1

;; ADDITIONAL SECTION:
web.service.consul. 0   IN  TXT "consul-network-segment="

;; Query time: 2 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Jul 16 14:26:53 PDT 2019
;; MSG SIZE  rcvd: 99

As you can see, an A record was returned containing the IP address where the service was registered. A records can only hold IP addresses.

You can also use the DNS interface to retrieve the entire address/port pair as a SRV record.

$ dig @127.0.0.1 -p 8600 web.service.consul SRV

; <<>> DiG 9.10.6 <<>> @127.0.0.1 -p 8600 web.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56598
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.        IN  SRV

;; ANSWER SECTION:
web.service.consul. 0   IN  SRV 1 1 80 Judiths-MBP.lan.node.dc1.consul.

;; ADDITIONAL SECTION:
Judiths-MBP.lan.node.dc1.consul. 0 IN   A   127.0.0.1
Judiths-MBP.lan.node.dc1.consul. 0 IN   TXT "consul-network-segment="

;; Query time: 2 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Jul 16 14:31:13 PDT 2019
;; MSG SIZE  rcvd: 150

The SRV record says that the web service is running on port 80 and exists on the node Judiths-MBP.lan.node.dc1.consul.. An additional section is returned by the DNS with the A record for that node.

Finally, you can also use the DNS interface to filter services by tags. The format for tag-based service queries is TAG.NAME.service.consul. In the example below, you'll ask Consul for all web services with the "rails" tag. You'll get a successful response since you registered the web service with that tag.

$ dig @127.0.0.1 -p 8600 rails.web.service.consul

; <<>> DiG 9.10.6 <<>> @127.0.0.1 -p 8600 rails.web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37666
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;rails.web.service.consul.  IN  A

;; ANSWER SECTION:
rails.web.service.consul. 0 IN  A   127.0.0.1

;; ADDITIONAL SECTION:
rails.web.service.consul. 0 IN  TXT "consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Jul 16 14:32:53 PDT 2019
;; MSG SIZE  rcvd: 105

HTTP API

In addition to the DNS interface, you can also query for the service using the HTTP API.

$ curl http://localhost:8500/v1/catalog/service/web
[
    {
        "ID": "82f64bfa-22c2-5727-0f5d-0bae376f6584",
        "Node": "Judiths-MBP.lan",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "NodeMeta": {
            "consul-network-segment": ""
        },
        "ServiceKind": "",
        "ServiceID": "web",
        "ServiceName": "web",
        "ServiceTags": [
            "rails"
        ],
        "ServiceAddress": "",
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {},
        "ServicePort": 80,
        "ServiceEnableTagOverride": false,
        "ServiceProxyDestination": "",
        "ServiceProxy": {},
        "ServiceConnect": {},
        "CreateIndex": 10,
        "ModifyIndex": 10
    }
]

The HTTP API lists all nodes hosting a given service. As you will see later when we discuss health checks you'll typically want to filter your query for only healthy service instances, which DNS does automatically under the hood. Filter your HTTP API query to look for only healthy instances.

$ curl 'http://localhost:8500/v1/health/service/web?passing'
[
    {
        "Node": {
            "ID": "82f64bfa-22c2-5727-0f5d-0bae376f6584",
            "Node": "Judiths-MBP.lan",
            "Address": "127.0.0.1",
            "Datacenter": "dc1",
            "TaggedAddresses": {
                "lan": "127.0.0.1",
                "wan": "127.0.0.1"
            },
            "Meta": {
                "consul-network-segment": ""
            },
            "CreateIndex": 9,
            "ModifyIndex": 10
        },
...
        "Checks": [
            {
                "Node": "Judiths-MBP.lan",
                "CheckID": "serfHealth",
                "Name": "Serf Health Status",
                "Status": "passing",
                "Notes": "",
                "Output": "Agent alive and reachable",
                "ServiceID": "",
                "ServiceName": "",
                "ServiceTags": [],
                "Definition": {},
                "CreateIndex": 9,
                "ModifyIndex": 9
            }
        ]
    }
]

Updating Services

Next you'll update the web service by registering a health check for it. Remember that because you never started a service on port 80 where you registered web, the health check you register will fail.

You can update service definitions without any downtime by changing the service definition file and sending a SIGHUP to the agent or running consul reload. Alternatively, you can use the HTTP API to add, remove, and modify services dynamically. In this example, you will update the registration file.

First, edit the registration file by running the following command. Copy and paste the whole code block (excluding the $) into your terminal.

$ echo '{"service":
  {"name": "web",
    "tags": ["rails"],
    "port": 80,
    "check": {
      "args": ["curl", "localhost"],
      "interval": "10s"
    }
  }
}' > ./consul.d/web.json

The 'check' stanza of this service definition adds a script-based health check that tries to connect to the web service every 10 seconds via curl. Script based health checks run as the same user that started the Consul process.

If the command exits with an exit code >= 2, then the check will fail and Consul will consider the service unhealthy. An exit code of 1 will be considered as warning state.

Now reload Consul's configuration to make it aware of the new health check.

$ consul reload
Configuration reload triggered

Notice the following lines in Consul's logs, which indicate that the web check is critical.

    2019/08/06 16:35:03 [INFO] agent: Synced service "web"
    2019/08/06 16:35:03 [DEBUG] agent: Check "service:web" in sync
    2019/08/06 16:35:03 [DEBUG] agent: Node info in sync
...
    2019/08/06 16:35:06 [WARN] agent: Check "service:web" is now critical
    2019/08/06 16:35:16 [WARN] agent: Check "service:web" is now critical
...

Consul's DNS server only returns healthy results. Query DNS for the web service again. It shouldn't return any IP addresses since web's health check is failing.

$ dig @127.0.0.1 -p 8600 web.service.consul

; <<>> DiG 9.10.6 <<>> @127.0.0.1 -p 8600 web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 28984
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.        IN  A

;; AUTHORITY SECTION:
consul.         0   IN  SOA ns.consul. hostmaster.consul. 1565134764 3600 600 86400 0

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Aug 06 16:39:24 PDT 2019
;; MSG SIZE  rcvd: 97

Notice that there is no answer section in the response, because Consul has marked the web service as unhealthy.

Summary

In this guide you registered a service with Consul and learned how to query it using the HTTP API and DNS interface. You also added a script based health check for the service. You can find a complete list of service registration fields in the API documentation, or learn more about health checks in the check definition documentation.

Continue to the next guide to learn how to enable Consul's service mesh control plane called Consul Connect, which allows you to secure and observe network traffic between your services, and allow or deny inter-service communication.