Virtual Event
Join us for the next HashiConf Digital October 12-15, 2020 Register for Free

Run Consul on Kubernetes

Consul and Kubernetes Reference Architecture

Preparing your Kubernetes cluster to successfully deploy and run Consul is an important first step in your production deployment process. In this tutorial you will prepare your Kubernetes cluster, that can be running on any platform (AKS, EKS, GKE, etc). However, we will call out cloud specific differences when applicable. Before starting this tutorial you should have experience with Kubernetes, and have kubectl and helm configured locally.

By the end of this tutorial, you will be able to select the right resource limits for Consul pods, select the Consul datacenter design that meets your use case, and understand the minimum networking requirements.

»Infrastructure requirements

Consul server agents are responsible for the datacenter state, responding to RPC queries, and processing all write operations. Since the Consul servers are highly active and are responsible for maintaining the datacenter state, server sizing is critical for the overall performance, efficiency, and health of the Consul datacenter. Review the Consul Reference Architecture tutorial for sizing recommendations for small and large Consul datacenters.

The CPU and memory recommendations can be used when you select the resources limits for the Consul pods. The disk recommendations can also be used when selecting the resources limits and configuring persistent volumes. You will need to set both limits and requests in the Helm chart. Below is an example snippet of Helm configuration for a Consul server in a large environment.

# config.yaml

server:
  resources: |
    requests:
      memory: "32Gi"
      cpu: "4"
    limits:
      memory: "32Gi"
      cpu: "4"

  storage: 50Gi
# ...

You should also set resource limits for Consul clients, so that the client pods do not unexpectedly consume more resources than expected.

Persistent volumes (PV) allow you to have a fixed disk location for the Consul data. This ensures that if a Consul server is lost, the data will not be lost. This is an important feature of Kubernetes, but may take some additional configuration. If you are running Kubernetes on one of the major cloud platforms, persistent volumes should already be configured for you; be sure to read their documentation for more details. If you are setting up the persistent volumes resource in Kubernetes, you may need to map the Consul server to that volume with the storage class parameter.

Finally, you will need to enable RBAC on your Kubernetes cluster. Review the Kubernetes RBAC documentation. You should also review RBAC and authentication documentation if your Kubernetes cluster is running on a major cloud platform.

»Datacenter design

There are many possible configurations for running Consul with Kubernetes. In this tutorial we will cover three of the most common.

  1. Consul agents can be solely deployed within Kubernetes.
  2. Consul servers can be deployed outside of Kubernetes and clients inside of Kubernetes.
  3. Multiple Consul datacenters with agents inside and outside of Kubernetes.

Review the Consul Kubernetes-specific documentation for additional use case information.

Since all three use cases will also need catalog sync, review the implementation details for catalog sync.

»Consul datacenter deployed in Kubernetes

Deploying a Consul datacenter, servers and clients, in Kubernetes can be done with the official Helm chart. This configuration is useful for managing services within Kubernetes and is common for users who do not already have a production Consul datacenter.

Reference Diagram

The Consul datacenter in Kubernetes will function the same as a platform independent Consul datacenter, such as Consul datacenters deployed on bare metal servers or virtual machines. Agents will communicate over LAN gossip, servers will participate in the Raft consensus, and client requests will be forwarded to the servers via RPCs.

»Consul datacenter with a Kubernetes cluster

To use an existing Consul datacenter to manage services in Kubernetes, Consul clients can be deployed within the Kubernetes cluster. This will also allow Kubernetes-defined services to be synced to Consul. This design allows Consul tools such as envconsul, consul-template, and more to work on Kubernetes.

Reference Diagram

This type of deployment in Kubernetes can also be set up with the official Helm chart.

»Multiple Consul datacenters with a Kubernetes cluster

Multiple Consul datacenters can be joined by WAN links. The datacenters can operate independently and only communicate over the WAN. This type of datacenter design is detailed in the Reference Architecture tutorial. In this setup, you can have a Consul datacenter running outside of Kubernetes and a Consul datacenter running inside of Kubernetes.

»Catalog sync

To use catalog sync, you must enable it in the Helm chart. Catalog sync allows you to sync services between Consul and Kubernetes. The sync can be unidirectional in either direction or bidirectional. Read the documentation to learn more about the configuration.

Services synced from Kubernetes to Consul will be discoverable, like any other service within the Consul datacenter. Read more in the network connectivity section to learn more about related Kubernetes configuration. Services synced from Consul to Kubernetes will be discoverable with the built-in Kubernetes DNS once a Consul stub domain is deployed. When bidirectional catalog sync is enabled, it will behave like the two unidirectional setups.

»Network connectivity

When running Consul as a pod inside of Kubernetes, the Consul servers will be automatically configured with the appropriate addresses. However, when running Consul servers outside of the Kubernetes cluster and clients inside Kubernetes as pods, there are additional networking considerations.

»Network connectivity for services

When using Consul catalog sync, to sync Kubernetes services to Consul, you will need to ensure the Kubernetes services are supported service types and configure correctly in Kubernetes. If the service is configured correctly, it will be discoverable by Consul like any other service in the datacenter.

»Network security

Finally, you should consider securing your Consul datacenter with ACLs. ACLs should be used with Consul Connect service mesh to secure service to service communication. The Kubernetes cluster should also be secured.

»Next steps

You are now prepared to deploy Consul with Kubernetes. In this tutorial, you were introduced to several datacenter designs for a variety of use cases. This tutorial also outlined the Kubernetes prerequisites, resource requirements for Consul, and networking considerations. Continue onto the Deploying Consul with Kubernetes tutorial for information on deploying Consul with the official Helm chart.