Encryption as a Service

Java Application Demo

Once you have learned the fundamentals of Vault, the next step is to start integrating your system with Vault to secure your organization's secrets.

The purpose of this guide is to go through the working implementation demo introduced in the Manage secrets, access, and encryption in the public cloud with Vault webinar.

YouTube

The Java application in this demo leverages the Spring Cloud Vault library which provides lightweight client-side support for connecting to Vault in a distributed environment.

» Challenge

Incidents of data breaches which expose sensitive information make headlines more often than we like to hear. It becomes more and more important to protect data by encrypting it whether the data is in-transit or at-rest. However, creating a highly secure and sophisticated solution by yourself requires time and resources which are in demand when an organization is facing a constant threat.

» Solution

Vault centralizes management of cryptographic services used to protect your data. Your system can communicate with Vault easily through the Vault API to encrypt and decrypt your data, and the encryption keys never have to leave the Vault.

Encryption as a Service

» Prerequisites

To perform the tasks described in this guide:

» Steps

After downloading the demo assets from the GitHub repository, you should find the following folders:

Folder Description
aws Supporting files to deploy the demo app to AWS
kubernetes Supporting files to deploy the demo app to Kubernetes
nomad Supporting files to deploy the demo app to Nomad
scripts Scripts to setup PostgreSQL and Vault
src/main Sample app source code
vagrant-local Vagrant file to deploy the demo locally

In this guide, you will perform the following:

  1. Review the demo application implementation
  2. Deploy and review the demo environment
  3. Run the demo application
  4. Reload the Static Secrets

Encryption as a Service

» Step 1: Review the demo application implementation

The source code can be found under the src/main directory.

Sample Code

The demo Java application leverages the Spring Cloud Vault library to communicate with Vault.

In the TransitConverter class, the convertToDatabaseColumn method invokes a Vault operation to encrypt the order. Similarly, the convertToEntityAttribute method decrypts the order data.

@Override
public String convertToDatabaseColumn(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Plaintext plaintext = Plaintext.of(customer);
  String cipherText = vaultOps.opsForTransit().encrypt("order", plaintext).getCiphertext();
  return cipherText;
}

@Override
public String convertToEntityAttribute(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Ciphertext ciphertext = Ciphertext.of(customer);
  String plaintext = vaultOps.opsForTransit().decrypt("order", ciphertext).asString();
  return plaintext;
}

The VaultDemoOrderServiceApplication class defines the main method.

public class VaultDemoOrderServiceApplication  {

  private static final Logger logger = LoggerFactory.getLogger(VaultDemoOrderServiceApplication.class);

  @Autowired
  private SessionManager sessionManager;

  @Value("${spring.datasource.username}")
  private String dbUser;

  @Value("${spring.datasource.password}")
  private String dbPass;

  public static void main(String[] args) {
    SpringApplication.run(VaultDemoOrderServiceApplication.class, args);
  }

  @PostConstruct
  public void initIt() throws Exception {
    logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
    logger.info("Got DB User: " + dbUser);
  }
}

The OrderAPIController class defines the API endpoint (api/orders).

» Step 2: Deploy and review the demo environment

Now let's run the demo app and examine how it behaves.

» Task 1: Run Vagrant

In the vault-guides/secrets/spring-cloud-vault/vagrant-local folder, a Vagrantfile is provided which spins up a Linux machine where the demo components are installed and configured.

# Change the working directory to vagrant-local
$ cd /vault-guides/secrets/spring-cloud-vault/vagrant-local

# Create and configure a Linux machine. This takes about 3 minutes
$ vagrant up
...
demo: Success! Data written to: database/roles/order
demo: Success! Enabled the transit secrets engine at: transit/
demo: Success! Data written to: transit/keys/order
demo: Success! Data written to: secret/spring-vault-demo

# Verify that the virtual machine was successfully created and running
$ vagrant status
Current machine states:
demo                      running (virtualbox)
...

# Connect to the demo machine
$ vagrant ssh demo

There are 3 Docker containers running on the machine: spring, vault, and postgres.

[vagrant@demo ~]$ docker ps
CONTAINER ID     IMAGE            COMMAND                  CREATED           STATUS           NAMES
684d8fb23ae5     spring           "java -Djava.secur..."   7 minutes ago     Up 7 minutes     spring
dc6a3454b323     vault:0.10.0     "docker-entrypoint..."   7 minutes ago     Up 7 minutes     vault
4093a45c209f     postgres         "docker-entrypoint..."   7 minutes ago     Up 7 minutes     postgres

» Task 2: Examine the Vault environment

During the demo machine provisioning, the /scripts/vault.sh script was executed to perform the following:

  • Created a policy named order
  • Enabled the transit secret engine and created an encryption key named order
  • Enabled the database secret engine and created a role named order

View the vault log:

[vagrant@demo ~]$  docker logs vault

==> Vault shutdown triggered
==> Vault server configuration:
             Api Address: http://0.0.0.0:8200
                     Cgo: disabled
         Cluster Address: https://0.0.0.0:8201
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
                 Storage: inmem
                 Version: Vault v0.10.0
             Version Sha: 5dd7f25f5c4b541f2da62d70075b6f82771a650d
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
    $ export VAULT_ADDR='http://0.0.0.0:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: 2QIPWPDykRG/xWWl0quSHiXq8u+pFg3yEq0sgJPhMbA=
Root Token: root

Notice that the log indicates that the Vault server is running in the dev mode, and the root token is root.

You can visit the Vault UI at http://localhost:8200/ui. Enter root and click Sign In.

Select the transit/ secrets engine, and you should find an encryption key named order.

Vault UI

Under the Policies, verify that the order policy exists.

Vault UI

This order policy is for the application. It permits read on the database/creds/order path so that the demo app can get a dynamically generated database credential from Vault. Therefore the PostgreSQL credentials are not hard-coded anywhere.

path "database/creds/order"
{
  capabilities = ["read"]
}

An update permission allows the app to request data encryption and decryption using the order encryption key in Vault.

# ...
path "transit/decrypt/order" {
  capabilities = ["update"]
}

path "transit/encrypt/order" {
  capabilities = ["update"]
}
# ...

» Task 3: Examine the Spring container

Remember that the VaultDemoOrderServiceApplication class logs messages during the successful execution of initIt():

@PostConstruct
    public void initIt() throws Exception {
        logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
        logger.info("Got DB User: " + dbUser);
    // ...
}

Verify that the log indicates that the demo app obtained a database credentials from Vault successfully:

[vagrant@demo ~]$  docker logs spring | grep Got
...VaultDemoOrderServiceApplication : Got Vault Token: root
...VaultDemoOrderServiceApplication : Got DB User: v-token-order-rywqz61432yyx2x27w8r-1524067226

Create a new shell session in the spring container.

[vagrant@demo ~]$  docker exec -it spring sh
/ #

Find the bootstrap.yaml file:

/ #  ls -al
total 36720
drwxr-xr-x    1 root     root            51 Apr 18 16:00 .
drwxr-xr-x    1 root     root            51 Apr 18 16:00 ..
-rwxr-xr-x    1 root     root             0 Apr 18 16:00 .dockerenv
-rwxr--r--    1 root     root      37587245 Apr 18 15:59 app.jar
drwxr-xr-x    2 root     root          4096 Jan  9 19:37 bin
-rw-r--r--    1 1000     1000           426 Apr 17 17:58 bootstrap.yaml
...
# bootstrap.yaml
spring.application.name: spring-vault-demo
spring.cloud.vault:
  authentication: TOKEN
  token: ${VAULT_TOKEN}
  host: localhost
  port: 8200
  scheme: http
  fail-fast: true
  config.lifecycle.enabled: true
  generic:
    enabled: true
    backend: secret
  database:
    enabled: true
    role: order
    backend: database
spring.datasource:
  url: jdbc:postgresql://localhost:5432/postgres

The client token was injected into the spring container as an environment variable (VAULT_TOKEN) by Vagrant.

Enter exit to close the shell session in the spring container.

» Task 4: Examine the PostgreSQL database

Connect to the PostgreSQL database running in the postgres container:

[vagrant@demo ~]$ docker exec -it postgres psql -U postgres -d postgres
psql (10.3 (Debian 10.3-1.pgdg90+1))
Type "help" for help.


postgres=# \d orders
                                          Table "public.orders"
    Column     |            Type             | Collation | Nullable |              Default
---------------+-----------------------------+-----------+----------+------------------------------------
 id            | bigint                      |           | not null | nextval('orders_id_seq'::regclass)
 customer_name | character varying(60)       |           | not null |
 product_name  | character varying(20)       |           | not null |
 order_date    | timestamp without time zone |           | not null |
Indexes:
    "orders_pkey" PRIMARY KEY, btree (id)

Let's list the existing database roles.

postgres-# \du
                                                     List of roles
                   Role name                   |                         Attributes                         | Member of
-----------------------------------------------+------------------------------------------------------------+-----------
 postgres                                      | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 v-token-order-rywqz61432yyx2x27w8r-1524067226 | Password valid until 2018-04-18 20:56:31+00                | {}

Notice that there is a role name starting with v-token-order- which was dynamically created by the database secret engine.

Enter \q to exit out of the psql session, or you can open another terminal and SSH into the demo virtual machine.

» Step 3: Run the demo application

If everything looked fine in Step 2, you are ready to write some data.

Vault UI

You have verified in the spring log that the demo app successfully retrieved a database credential from the Vault server during its initialization.

The next step is to send a new order request via the demo app's orders API (http://localhost:8080/api/orders).

Create a file payload.json with the following contents.

{
  "customerName": "John",
  "productName": "Nomad"
}

Send the file in a request using cURL.

[vagrant@demo ~]$ curl --request POST --header "Content-Type: application/json" \
                       --data @payload.json http://localhost:8080/api/orders | jq

You should see this output:

{
  "id": 2,
  "customerName": "John",
  "productName": "Nomad",
  "orderDate": "2018-04-18T22:07:42.916+0000"
}

Postman

The order data you sent gets encrypted by Vault. The database only sees the ciphertext. Let's verify that the order information stored in the database is encrypted.

[vagrant@demo ~]$ docker exec -it postgres psql -U postgres -d postgres

postgres=# select * from orders;
 id |                     customer_name                     | product_name |       order_date
----+-------------------------------------------------------+--------------+-------------------------
  1 | vault:v1:Qj0lx5DSZvwcHeMOX/5UX/ErHTaDPA3mVlSSpaXd1tbM | VE           | 2018-04-18 21:56:37.924
  2 | vault:v1:UwL3HnyqTUac5ElS5WYAuNg3NdIMFtd6vvwukL+FaKun | Nomad        | 2018-04-18 22:07:42.916
(2 rows)

postgres=# \q

In this demo, Vault encrypts the customer names; therefore the values in the customer_name column do not display the names in a human readable manner (e.g. "James" and "John").

Now, retrieve the order data via the orders API:

[vagrant@demo ~]$ curl --header "Content-Type: application/json" \
                       http://localhost:8080/api/orders | jq

The output should look like this:

[
  {
    "id": 1,
    "customerName": "James",
    "productName": "VE",
    "orderDate": "2018-04-18T21:56:37.924+0000"
  },
  {
    "id": 2,
    "customerName": "John",
    "productName": "Nomad",
    "orderDate": "2018-04-18T22:07:42.916+0000"
  }
]

The customer names should be readable. Remember that the order policy permits the demo app to encrypt and decrypt data using the order encryption key in Vault.

» Web UI

Vault UI makes it easy to decrypt the data.

In the Secrets tab, select transit/ > orders, and select Key actions.

Web UI

Select Decrypt from the transit actions. Now, copy the ciphertext from the orders table and paste it in.

Web UI

Click Decrypt.

Web UI

Finally, click Decode from base64 to reveal the customer name.

Web UI

» Step 4: Reloading the Static Secrets

Now, let's test another API endpoint, api/secret provided by the demo app. A plain old Java object, Secret defines a get method for key and value. The SecretController.java defines an API endpoint, api/secret.

// SecretController.java
package com.hashicorp.vault.spring.demo;
// ...

@RefreshScope
@RestController
public class SecretController {

  @Value("${secret:n/a}")
  String secret;

  @RequestMapping("/api/secret")
  public Secret secret() {
    return new Secret("secret", secret);
  }
}

Remember from Step 2 that the order policy granted permissions on the secret/spring-vault-demo path.

path "secret/spring-vault-demo" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

The demo app retrieved the secret from secret/spring-vault-demo and has a local copy. If someone (or perhaps another app) updates the secret, it makes the secret held by the demo app to be obsolete.

Static Secret

Spring offers Spring Boot Actuator which can be used to facilitate the reloading of the static secret.

» Task 1: Read the secret

The initial key-value was set by Vagrant during the provisioning. (See the Vagrantfile at line 48.)

Let's invoke the demo app's secret API (api/secret):

$ curl -s http://localhost:8080/api/secret | jq

The output should look like this:

{
  "key": "secret",
  "value": "hello-vault"
}

This is the secret that the demo app knows about.

» Task 2: Update the Secrets

Now, update the secret stored in Vault using API:

# Update the value via API
$ curl --header "X-Vault-Token: root" \
       --request POST \
       --data '{ "secret": "my-api-key" }' \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo

# Verify that the secret value was updated
$ curl --header "X-Vault-Token: root" \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo | jq

The output will include a large data structure which starts with the following:

{
  "request_id": "514601e4-a790-3dc6-14b0-537d6982a6c6",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 2764800,
  "data": {
    "secret": "my-api-key"
  }
}

» Task 3: Refresh the secret on demo app

Run the demo app's secret API again:

$ curl -s http://localhost:8080/api/secret | jq

You'll see this output:

{
  "key": "secret",
  "value": "hello-vault"
}

The current value stored in Vault is now my-api-key; however, the demo app still holds hello-vault.

Spring provides an actuator which can be leveraged to refresh the secret value. At line 54 of the vault-guides/secrets/spring-cloud-vault/pom.xml, you see that the actuator was added to the project.

<!-- ... -->
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- ... -->

Let's refresh the secret using the actuator:

$ curl -s --request POST http://localhost:8080/actuator/refresh | jq
[
  "secret"
]

Read back the secret from the demo app again:

$ curl -s http://localhost:8080/api/secret | jq
{
  "key": "secret",
  "value": "my-api-key"
}

It should display the correct value.


When you are done exploring the demo implementation, you can destroy the virtual machine:

$ vagrant destroy
demo: Are you sure you want to destroy the 'demo' VM? [y/N] y
==> demo: Forcing shutdown of VM...
==> demo: Destroying VM and associated drives...

» Help and Reference