Connect Multiple HashiCorp Consul Service on Azure Datacenters

HashiCorp Consul Service (HCS) on Azure enables Microsoft Azure users to natively provision HashiCorp-managed Consul servers for service discovery and service mesh across a mix of VM, hybrid/on-premises, and Kubernetes environments in any supported Azure region directly through the Azure Marketplace, or using the official HashiCorp Azure CLI extension for HCS on Azure.

In this tutorial you will create and federate two datacenters. By federating them, you are creating an environment where services in each datacenter can be configured to securely access one another.

Specifically, you will:

• Install the HashiCorp Azure CLI extension for HCS on Azure
• Deploy two Federated HCS on Azure datacenters
• Validate the deployment with the HCS Azure CLI extension
• Inspect your environment using the Consul CLI
• Decommission the HCS Environment

»Prerequisites

To complete this tutorial you will need the following.

• Basic command line access
• The Azure CLI
• The Azure CLI Extension for HCS
• An Azure account
• A resource group in a supported region

»Install the HashiCorp extension

While HCS on Azure can be provisioned from the Azure portal, the post provisioning operator workflow requires you to use the official Azure CLI extension for HCS on Azure. You can install the extension directly from your shell using the following az command. Enter y when prompted to confirm.

$ az extension add --source https://releases.hashicorp.com/hcs/0.4.1/hcs-0.4.1-py2.py3-none-any.whl
Are you sure you want to install this extension? (y/n): y
The installed extension 'hcs' is in preview.

»Deploy two federated HCS on Azure datacenters

HCS on Azure datacenters that wish to join a federation must be federated at the time of creation. The first datacenter you create, by necessity, will be the primary datacenter, and will have no notion of federation. Additional datacenters can join the primary datacenter to grow the federation, but they must join the federation during initial provisioning. They will not be able to join later. Federating HCS datacenters, currently, can only be achieved using the Azure CLI extension for HCS on Azure.

»Considerations and constraints

• Since Azure allows you to delete any managed resource, HCS on Azure cannot guard against you deleting the primary datacenter of a federation through the Azure Portal or by using the az managedapp delete command. Deleting a datacenter from a federation is an unrecoverable operation at this time. Be careful when you delete datacenters.
• Datacenter names must be unique. This is a Consul requirement, as datacenter names will be part of the URL that Consul generates to enable service discovery through DNS.
• Subnets must be unique. Overlapping CIDR blocks will conflict.
• Secondary datacenters cannot run a Consul version greater than the primary datacenter.
• The order of operations in this tutorial is intentionally very specific. You must first create the primary datacenter, then the federation token, then add the secondary datacenter using the token.
• Tokens expire, but you can always create a new token. If you wish to add additional datacenters to the federation after your original token has expired, start at the Create a federation token step, and repeat Create a secondary datacenter until you have added all the desired datacenters to the federation.

»Setup the primary datacenter

Once you have created an Azure resource group in one of the supported regions, you are ready to create your HCS on Azure datacenters. This tutorial uses a resource group named hcs-federation, but you can name yours anything you like.

Use the HCS on Azure extension to provision your primary datacenter. Name this datacenter primary-dc so that it is easy to identify, and make note of both the vnet-cidr and consul-version options. You must ensure that the secondary datacenter's vnet-cidr range does not overlap the primary, and you must also ensure that you do not attempt to use a consul-version greater than the primary datacenter's.

Issue the following command to create your primary datacenter. This may take up to fifteen minutes, depending on the current Azure load.

$ az hcs create \
    --resource-group hcs-federation \
    --datacenter-name primary-dc \
    --vnet-cidr 172.25.16.0/24 \
    --consul-version 1.8.6 \
    --name primary-hcs \
    --email admin@some.co \
    --external-endpoint enabled

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
{- Finished ..
  "applicationDefinitionId": null,
...TRUNCATED...
  "tags": null,
  "type": "Microsoft.Solutions/applications"
}

»Create a federation token

A federation token is an HCS on Azure specific shared secret that allows the HCS on Azure control plane to authenticate and authorize requests to federate Consul datacenters.

Because HCS on Azure deployments can be provisioned under a single Azure account, but exist under different subscription IDs, the federation token provides a convenient way to authorize and authenticate requests from different subscription scopes. The federation token contains identifying information about the datacenter, but does not contain any customer specific PII. Federation tokens have a lifespan of ~2hrs, and a single federation token can be used to create multiple clusters in a federation within the 2hr period. Creating a federation can only be done via the HCS Azure CLI extension. If your token expires, you can use the same process to generate a new one.

Issue the following command to create a federation token and export it to an environment variable named FEDERATION_TOKEN so that you can reference it in later steps.

$ export FEDERATION_TOKEN=$(az hcs create-federation-token \
    --resource-group hcs-federation \
    --name primary-hcs \
    | jq -r '.federationToken') \
    && echo $FEDERATION_TOKEN

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
...TRUNCATED...
dmF1bHQ6djE6amJNbjYrUld6SGsrT1pSZUdidDFRZ2VrbG9kSDRhNlVvcHhwSjdtZGI4OD0

»Setup a secondary datacenter

Now that you have created the primary datacenter, and created a federation token, use the Azure CLI extension for HCS on Azure to create the secondary datacenter. This may take up to fifteen minutes, depending on the current Azure load.

$ az hcs create \
    --resource-group hcs-federation \
    --datacenter-name secondary-dc \
    --vnet-cidr 172.25.10.0/24 \
    --consul-version 1.8.6 \
    --name secondary-hcs \
    --email admin@some.co \
    --external-endpoint enabled \
    --federation-token $FEDERATION_TOKEN

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
{- Finished ..
  "applicationDefinitionId": null,
...TRUNCATED...
  "tags": null,
  "type": "Microsoft.Solutions/applications"
}

Note that the datacenter-name is set to secondary-dc so that it is easy to differentiate the primary and secondary datacenters. Also note that the vnet-cidr option is set to a range that does not overlap the range defined for the primary-dc, and that the consul-version option is less than or, in as in this case, equal to the value you set for the primary-dc.

»Validate the deployment with the HCS Azure CLI extension

The HCS Azure CLI extension has a helper command called show-federation that can be used to inspect the federation disposition of an HCS datacenter.

Issue the following command to inspect the primary HCS datacenter.

$ az hcs show-federation \
    --resource-group hcs-federation \
    --name primary-hcs

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
{
  "primaryDatacenter": {
    "id": "/subscriptions//resourceGroups/hcs-federation/providers/Microsoft.Solutions/applications/primary-hcs",
    "name": "primary-hcs",
    "resourceGroup": "hcs-federation",
    "subscriptionId": ""
  },
  "secondaryDatacenters": [
    {
      "id": "/subscriptions//resourceGroups/hcs-federation/providers/Microsoft.Solutions/applications/secondary-hcs",
      "name": "secondary-hcs",
      "resourceGroup": "hcs-federation",
      "subscriptionId": ""
    }
  ],
  "state": "RUNNING"
}

The output confirms that the primary-hcs datacenter is configured as the primaryDatacenter member in the federation, and that the secondary-hcs datacenter is a member of the secondaryDatacenters group. The output also indicates that the federation state is RUNNING. This is an indicator that the federation is healthy and operating as expected.

Now, issue the same command, but this time target the secondary datacenter.

$ az hcs show-federation \
    --resource-group hcs-federation \
    --name secondary-hcs

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
{
  "primaryDatacenter": {
    "id": "/subscriptions//resourceGroups/hcs-federation/providers/Microsoft.Solutions/applications/primary-hcs",
    "name": "primary-hcs",
    "resourceGroup": "hcs-federation",
    "subscriptionId": ""
  },
  "secondaryDatacenters": [
    {
      "id": "/subscriptions//resourceGroups/hcs-federation/providers/Microsoft.Solutions/applications/secondary-hcs",
      "name": "secondary-hcs",
      "resourceGroup": "hcs-federation",
      "subscriptionId": ""
    }
  ],
  "state": "RUNNING"
}

Notice that the output is identical. It doesn't matter which datacenter you use as a starting point for inspection; the federation configuration is always constant.

»Attempt to delete the primary

As mentioned before, there is no way for HCS on Azure to safeguard against an operator choosing to manually delete a cluster from within the Azure Portal UI. However, the Azure CLI extension for HCS provides a more context aware user experience, and can help safeguard against this type of operator error.

Issue the following command, and try to delete the primary datacenter.

$ az hcs delete \
    --resource-group hcs-federation \
    --name primary-hcs

Example output:

Command group 'hcs' is in preview. It may be changed/removed in a future release.
You cannot delete the primary datacenter of a federation before all secondary datacenters are deleted.

Notice that the CLI warned you of a potentially unrecoverable error. It is strongly recommended that HCS operators use the Azure CLI extension for HCS for all HCS administration and automation, as it is guaranteed to have more stringent safeguards than the native Azure UI, SDKs or APIs.

»Decommission the HCS Environment

When you are done experimenting with your datacenters, and you do not plan to use them any longer, you must delete the resources to prevent incurring additional charges.

»Delete the secondary DC

As illustrated earlier, the HCS Azure CLI will not allow you to delete the primary datacenter as long as there are subordinate federated datacenters associated with it.

Issue the following command to delete the secondary datacenter.

$ az hcs delete \
    --resource-group hcs-federation \
    --name secondary-hcs

»Delete the primary DC

Now that you have successfully deleted the secondary datacenter, you can delete the primary.

Issue the following command to delete the primary datacenter.

$ az hcs delete \
    --resource-group hcs-federation \
    --name primary-hcs

»Next steps

In this tutorial you created and federated two HCS on Azure datacenters.

Specifically, you:

• Installed the HashiCorp Azure CLI extension for HCS
• Deployed two Federated HCS datacenters
• Validated the deployment with the HCS Azure CLI extension
• Inspected your environment using the Consul CLI
• Decommissioned the HCS Environment

To learn more about the HashiCorp Consul Service (HCS) on Azure, checkout the rest of the tutorials in our HCS on Azure collection, or experiment by deploying one of your own services.

To learn more about HashiCorp's fully managed cloud platform check out our HashiCorp Cloud Platform collection.

If you need guidance on how to deploy your services to Consul service mesh, refer to our Connect services with Consul Service Mesh tutorial.

To learn even more about operating, observing, and monitoring your Consul service mesh check out the following tutorials and collections.

• Maintenance & Monitoring
• Service Discovery & Health
• Layer 7 Observability with Consul, Prometheus, Grafana, and Kubernetes