Beta / Release Candidate Features

beta

Storage Migration Guide - Consul to Integrated Storage

This guide provides steps to migrate your Vault storage from Consul to Integrated Storage. (NOTE: Integrated storage requires Vault 1.4 or later.)

Before continuing with this guide, be sure to read the Preflight Checklist - Migrating to Integrated Storage first.

Vault storage migration workflow:

  1. Backup Vault data
  2. Migrate Vault storage
    1. Recommended approach
    2. Perform the storage migration
  3. Post-migration health check

Backup Vault data

Before performing the Vault server upgrade, it is recommended to take a snapshot of the Vault data using the Consul Snapshots.

Consul Enterprise users may be using the automated Snapshot Agent to capture snapshot periodically and retained in a specified destination.

Snapshot Vault data

On the Consul server cluster that contains the Vault data to be saved in a snapshot, execute the following command from either a Consul server directly or any system running a Consul client agent connected to the server cluster.

$ consul snapshot save backup.snap
Saved and verified snapshot to index 1394

The snapshot file backup.snap will be present in the current working directory.

Inspecting the Snapshot

The snapshot file is simply a gzip compressed archive. You can perform some operational inspection on the snapshot file via the consul snapshot inspect command and also manually by decompressing the file and examining its contents.

$ consul snapshot inspect backup.snap

ID           2-1394-1515172423763
Size         481887
Index        1394
Term         2
Version      1

This output shows the snapshot ID, size in bytes, plus the snapshot index, term, and version, which can be compared with the server (e.g. via consul info) and is useful to detect corruption.

Migrate Vault storage

Vault upgrades and storage migration should be performed in lower level environments first, starting with secondary clusters if you have Vault Enterprise Replication deployments.

Vault Enterprise Replication

In this example, start the upgrade on the Cluster 3, Cluster 4, or Cluster 5 first. Within a single cluster, first upgrade all standby nodes and then upgrade the active node.

If you are upgrading existing Vault nodes to v1.4.0 or later, be sure to read the upgrade guide to learn if there are any version specific notes related to your current Vault version or any versions between your current version and the intended upgrade version.

Download and install the latest version of Vault (v1.4.0 or later) which provides the integrated storage.

Here is the recommended storage migration approach.

  1. Stop the DR secondary cluster (Cluster 4 in the diagram as an example) and perform the storage migration on the secondary.

  2. After the migration, let some workload come through and monitor that the DR secondary cluster is catching up with the primary cluster which is still using Consul as the storage backend.

  3. Stop the primary cluster (Cluster 2 in the diagram), and promote the DR secondary cluster (Cluster 4) to be the new DR primary.

  4. Perform the storage migration on the old primary cluster (Cluster 2).

    NOTE: If this cluster is a Performance Replication secondary as shown in the diagram, monitor that it syncs up with its performance primary after the storage migration.

  5. Failback to the original primary cluster (Cluster 2).

Perform the storage migration

To walkthrough the migration steps, assume that the following is your new Vault server configuration.

# Storage configuration
storage "raft" {
  path = "/vault/node_1"
  node_id = "node_1"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  cluster_address = "0.0.0.0:8201"
  tls_cert_file = "/path/to/fullchain.pem"
  tls_key_file  = "/path/to/privkey.pem"
}

api_addr = "https://13.57.14.206:8200"
cluster_addr = "https://10.0.101.22:8201"
disable_mlock = true
ui=true

Notice that the path is set to /vault/node_1 and node_id is set to node_1. (Refer to the server configuration documentation for details.)


  1. Create a migration configuration file (e.g. migrate.hcl).

    storage_source "consul" {
     address = "127.0.0.1:8500"
     path   = "vault"
    }
    
    storage_destination "raft" {
      path = "/vault/node_1"
      node_id = "node_1"
    }
    
    cluster_addr = "https://10.0.101.22:8201"
    

    The storage_source stanza should be the current storage backend (consul) configuration, and the storage_destination points to the integrated storage (raft) configuration.

    The path and node_id must match the values you set in the server configuration file.

    IMPORTANT: The /vault/node_1 path must exists. The migration command will not create the folder for you.

  2. Execute the vault operator command to perform the migration.

    $ vault operator migrate -config=migrate.hcl
    
  3. You can start the Vault server using the new server configuration pointing to the raft storage.

Post-migration health check

Once the storage is migrated, validate that the cluster is healthy and check logs for any unusual errors that are related to the cluster health. Refer to the following guides:

Help and Reference