Virtual Event
Join us for the next HashiConf Digital October 12-15, 2020 Register for Free

Service Mesh and Consul Connect

Understand Terminating Gateways

Consul service mesh provides encrypted communication over mTLS and provides security between applications using Consul intentions. As you migrate applications to the service mesh, applications in the mesh need to securely connect to services still outside the mesh.

Terminating gateways are egress proxies that provide connectivity to external destinations by terminating Consul Connect mTLS connections, enforcing Consul intentions, and forwarding requests to appropriate destination services.

By using terminating gateways you will only need to enable traffic coming from the gateway without having to manually enable every service that needs to connect outside the mesh.

In this tutorial, you will get an overview of the terminating gateways functionality and their use cases. In each use case section, you will gain an understanding of deployment best practices and examples of common scenarios.

»Prerequisites

Each terminating gateway needs:

  • A local Consul client agent to manage its configuration.
  • General network connectivity to services within its local Consul datacenter.

Terminating gateways also require that your Consul datacenters are configured correctly:

  • You must use Consul version 1.8.0 or newer.
  • Consul Connect service mesh must be enabled on the datacenter's Consul servers.
  • gRPC must be enabled on all client agents.

Currently, Envoy is the only proxy with terminating gateway capabilities in Consul. Terminating gateway proxies receive their configuration through Consul, which automatically generates the proxy configuratiion based on the gateway's registration. Consul can only translate terminating gateway registration information into Envoy configuration, therefore the proxies acting as terminating gateways must be Envoy.

Connect proxies that send upstream traffic through a gateway aren't affected when you deploy terminating gateways. If you are using non-Envoy proxies as service mesh sidecar proxies they will continue to work for traffic directed at services linked to a terminating gateway as long as they discover upstreams with the /health/connect endpoint.

»Service mesh and terminating gateway architecture

Inside the mesh network, services are deployed with sidecar proxies. Proxies provide authentication and encryption for the services they represent. You can control service-to-service connections using Consul intentions. Consul intentions are enforced by the proxies and determine whether one service can connect to another.

Due to the central role of proxies in the service mesh, you need to deploy them on all service nodes or Kubernetes pods to take full advantage of the mesh functionalities. In most scenarios, the transition to a service mesh architecture will happen incrementally by moving services into the service mesh.

There are cases where it might not ever be possible to deploy a sidecar proxy for a service. The service might be running on an unsupported operating system or might be running somewhere where access to the host won't ever be provided, as in the case of a managed database.

Terminating gateways are a solution to uniformly discover and connect to services currently, or permanently, outside the service mesh, while still providing several of the benefits of a service mesh such as observability and encrypted connections governed by intentions. Using terminating gateways you can introduce Consul service mesh to these hybrid deployments that would otherwise require configuration tweaks or exceptions.

»Use case: one or more services outside the service mesh

The simplest use case for terminating gateways is routing to external services in the same logical network as the Consul service mesh.

This use case represents scenarios where there are still services that cannot be added to the Consul service mesh, but are located on a node with a local Consul client or are registered with a local Consul client agent. The service must be registered with Consul so that it can be routed and you can create Consul intentions.

Terminating Gateway single datacenter with managed services.

Other supported use cases include:

  • A managed service, such as Amazon RDS or a third-party service, that needs to be integrated into the service mesh. In this scenario, you will not be able to install Consul on the machine and you will only be able to access the service via the endpoint(s) it exposes.
  • A legacy service running on an unsupported OS or on embedded devices that is still present in your infrastructure and where hardware or OS limitations do not permit you to pair it with a sidecar proxy.
  • Hybrid applications transitioning towards the service mesh that require integration testing or are required to maintain compatibility with other legacy systems.

In these situations, given the heterogeneous nature or location of these services, the gateways and the external services would be registered in the same Consul datacenter catalog. Traffic from services in the mesh would flow through their sidecar to the terminating gateway. Then the gateway will forward the traffic, potentially unencrypted, to the destination service.

To terminate TLS connections, the terminating gateway will present leaf certificates for the services it represents. Internal mesh traffic to the gateway will be encrypted with mTLS and controlled by intentions as expected with Consul service mesh. Intentions reference the source and destination services and do not require knowledge of the gateway itself.

The routing between gateways and external services is determined by the terminating-gateway configuration entry. The configuration is associated with the name of a gateway service and will apply to all instances of the gateway with that name across all federated Consul datacenters.

Gateways are registered by name, and a config entry determines which external services each gateway service is responsible for. Spinning up multiple gateway instances with the same name will add redundancy with no additional configuration.

»Use case: many services in a shared legacy datacenter

Terminating gateways can also be useful when you have multiple logical network segments residing in multiple Consul service meshes that still depend on services residing in a shared legacy datacenter. These services might have the same limitations listed in previous scenarios or might be subject to internal policies that do not permit them to be migrated or for the configuration of the nodes they run on to be modified.

In these cases, you will be able to register services from the legacy datacenter with the service mesh by configuring a terminating gateway in each Consul datacenter that acts as a bridge to the shared legacy datacenter.

Terminating Gateway multiple datacenters on Kubernetes clusters with legacy service.

You will have to configure the terminating gateways and define the external services in each Consul datacenter for them to be aware of the services you need to reach.

Depending on the number of services residing in the shared data center, or on the number of service meshes you run, replicating the configuration or running many terminating gateways might not be the best solution.

In these cases, you may choose to deploy Consul servers to each legacy datacenter, making them their own Consul DC.

Terminating Gateway multiple datacenters with legacy datacenter.

»Intentions with terminating gateways

Gateways are not considered services and do not need to be considered when defining intentions. Destination services need to be referenced by the intention rather than the gateway.

When creating intentions, the source will be the service in the mesh, and the destination will be the external service.

$ consul intention create api legacy-backend

»Security considerations

Traffic inside the Consul service mesh is automatically mTLS encrypted. This is not true for traffic from a terminating gateway to an external service. By default traffic originating from the terminating gateway that is directed towards the legacy services is unencrypted. Consul will only encrypt connections to the gateway and not from the gateway to the destination service. This simplifies monitoring upstream traffic leaving the service mesh.

It is possible to configure the gateway to establish one-way or mutual TLS with external services if the external service supports it. Check the full documentation for all available options. By specifying a path to a CA file connections from the terminating gateway will be encrypted using one-way TLS authentication. If a path to a client certificate and private key are also specified, connections from the terminating gateway will be encrypted using mutual TLS authentication.

From the network point of view, you will have to ensure that the terminating gateway is able to reach all the external services it needs to communicate with, and make sure firewall rules are in place to only allow traffic from the gateway nodes to the external services.

»ACLs with terminating getaways

Service mesh proxies in Consul require a token with service:write on the service they represent in order to be able to request leaf certificates from the Consul CA. Since terminating gateways may represent many services, they need a policy that grants service:write on all the services they are expected to forward traffic to. Following the principle of least privilege, ACL tokens used should only grant write on the services represented.

For Consul Enterprise, if the terminating gateway targets services in multiple namespaces, then the policy needs to be updated instead. This is because a role’s service identities can only target services within the role’s namespace.

Lastly, since terminating gateways can target all services in a namespace, Consul Enterprise users can also provide a token that grants service:read to all services in that namespace.

»Redundancy considerations

The routing between terminating gateways and external services will be determined by centralized configuration. Terminating gateways will be registered by name, and a config entry will determine which external services each gateway service is responsible for. Spinning up multiple gateway instances with the same name will add redundancy with no additional configuration.

»Health checks for external services

For many external services there may not be a Consul agent running on the same node. This means that you must manually register the external node, service, and checks with the /catalog/register endpoint. Since an agent isn’t present, you must register the nodes themselves, and then register one or more services on those nodes.

»Next steps

In this tutorial you learned the main use cases for Consul terminating gateways. You also learned some best practices that should be applied when securing their configuration and preparing them for production.

Read more about terminating gateways in the Consul documentation.

More info on the configuration entry for the terminating gateways are also available in the Consul documentation

Follow Connect External Services to Consul Service Mesh With Terminating Gateways tutorial to learn how to use terminating gateways to secure and route communication to an external service.