Java Application Demo

Once you have learned the fundamentals of Vault, the next step is to start integrating your system with Vault to secure your organization's secrets.

This tutorial is a companion to a webinar that includes a live demo of how to manage secrets, access, and encryption in the public cloud with Vault.

The Java application in this demo leverages the Spring Cloud Vault library which provides lightweight client-side support for connecting to Vault in a distributed environment.

»Challenge

Incidents of data breaches which expose sensitive information make headlines more often than we like to hear. It becomes more and more important to protect data by encrypting it whether the data is in-transit or at-rest. However, creating a highly secure and sophisticated solution by yourself requires time and resources which are in demand when an organization is facing a constant threat.

»Solution

Vault centralizes management of cryptographic services used to protect your data. Your system can communicate with Vault easily through the Vault API to encrypt and decrypt your data, and the encryption keys never have to leave the Vault.

»Prerequisites

To perform the tasks described in this tutorial:

• Install HashiCorp Vagrant

»Step 1: Review the demo application implementation

Retrieve the configuration by cloning or downloading the [hashicorp/vault-guides] repository from GitHub.

Clone the repository.

$ git clone https://github.com/hashicorp/vault-guides.git

$ git clone https://github.com/hashicorp/vault-guides.git

Or download the repository.

This repository contains supporting content for all of the Vault learn tutorials. The content specific to this tutorial can be found within a sub-directory.

The source code can be found under the src/main directory.

$ cd vault-guides/secrets/spring-cloud-vault/src/main

$ tree
.
├── java
│   └── com
│       └── hashicorp
│           └── vault
│               └── spring
│                   └── demo
│                       ├── BeanUtil.java
│                       ├── Order.java
│                       ├── OrderAPIController.java
│                       ├── OrderRepository.java
│                       ├── Secret.java
│                       ├── SecretController.java
│                       ├── TransitConverter.java
│                       └── VaultDemoOrderServiceApplication.java
└── resources
    └── application.yaml

7 directories, 9 files

$ cd vault-guides/secrets/spring-cloud-vault/src/main

$ tree
.
├── java
│   └── com
│       └── hashicorp
│           └── vault
│               └── spring
│                   └── demo
│                       ├── BeanUtil.java
│                       ├── Order.java
│                       ├── OrderAPIController.java
│                       ├── OrderRepository.java
│                       ├── Secret.java
│                       ├── SecretController.java
│                       ├── TransitConverter.java
│                       └── VaultDemoOrderServiceApplication.java
└── resources
    └── application.yaml

7 directories, 9 files

The demo Java application leverages the Spring Cloud Vault library to communicate with Vault.

In the TransitConverter class, the convertToDatabaseColumn method invokes a Vault operation to encrypt the order. Similarly, the convertToEntityAttribute method decrypts the order data.

@Override
public String convertToDatabaseColumn(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Plaintext plaintext = Plaintext.of(customer);
  String cipherText = vaultOps.opsForTransit().encrypt("order", plaintext).getCiphertext();
  return cipherText;
}

@Override
public String convertToEntityAttribute(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Ciphertext ciphertext = Ciphertext.of(customer);
  String plaintext = vaultOps.opsForTransit().decrypt("order", ciphertext).asString();
  return plaintext;
}

@Override
public String convertToDatabaseColumn(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Plaintext plaintext = Plaintext.of(customer);
  String cipherText = vaultOps.opsForTransit().encrypt("order", plaintext).getCiphertext();
  return cipherText;
}

@Override
public String convertToEntityAttribute(String customer) {
  VaultOperations vaultOps = BeanUtil.getBean(VaultOperations.class);
  Ciphertext ciphertext = Ciphertext.of(customer);
  String plaintext = vaultOps.opsForTransit().decrypt("order", ciphertext).asString();
  return plaintext;
}

The VaultDemoOrderServiceApplication class defines the main method.

public class VaultDemoOrderServiceApplication  {

  private static final Logger logger = LoggerFactory.getLogger(VaultDemoOrderServiceApplication.class);

  @Autowired
  private SessionManager sessionManager;

  @Value("${spring.datasource.username}")
  private String dbUser;

  @Value("${spring.datasource.password}")
  private String dbPass;

  public static void main(String[] args) {
    SpringApplication.run(VaultDemoOrderServiceApplication.class, args);
  }

  @PostConstruct
  public void initIt() throws Exception {
    logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
    logger.info("Got DB User: " + dbUser);
  }
}

public class VaultDemoOrderServiceApplication  {

  private static final Logger logger = LoggerFactory.getLogger(VaultDemoOrderServiceApplication.class);

  @Autowired
  private SessionManager sessionManager;

  @Value("${spring.datasource.username}")
  private String dbUser;

  @Value("${spring.datasource.password}")
  private String dbPass;

  public static void main(String[] args) {
    SpringApplication.run(VaultDemoOrderServiceApplication.class, args);
  }

  @PostConstruct
  public void initIt() throws Exception {
    logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
    logger.info("Got DB User: " + dbUser);
  }
}

The OrderAPIController class defines the API endpoint (api/orders).

»Step 2: Deploy and review the demo environment

For this tutorial, you are going to provision a Linux machine locally using Vagrant. However, the GitHub repository provides supporting files to provision the environment demonstrated in the webinar.

Now let's run the demo app and examine how it behaves.

In the vault-guides/secrets/spring-cloud-vault/vagrant-local folder, a Vagrantfile is provided which spins up a Linux machine where the demo components are installed and configured.

First, change the working directory to vagrant-local.

$ cd vault-guides/secrets/spring-cloud-vault/vagrant-local

$ cd vault-guides/secrets/spring-cloud-vault/vagrant-local

Create and configure a Linux machine. This takes about 3 minutes.

$ vagrant up
...
demo: Success! Data written to: database/roles/order
demo: Success! Enabled the transit secrets engine at: transit/
demo: Success! Data written to: transit/keys/order
demo: Success! Data written to: secret/spring-vault-demo

# Verify that the virtual machine was successfully created and running
$ vagrant status
Current machine states:
demo                      running (virtualbox)
...

$ vagrant up
...
demo: Success! Data written to: database/roles/order
demo: Success! Enabled the transit secrets engine at: transit/
demo: Success! Data written to: transit/keys/order
demo: Success! Data written to: secret/spring-vault-demo

# Verify that the virtual machine was successfully created and running
$ vagrant status
Current machine states:
demo                      running (virtualbox)
...

Finally, connect to the demo machine.

$ vagrant ssh demo

$ vagrant ssh demo

There are 3 Docker containers running on the machine: spring, vault, and postgres.

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  ...    NAMES
cd629a609847        spring              "java -Djava.secur..."   ...    spring
623217149b68        vault:latest        "docker-entrypoint..."   ...    vault
32ff06ac02da        postgres            "docker-entrypoint..."   ...    postgres

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  ...    NAMES
cd629a609847        spring              "java -Djava.secur..."   ...    spring
623217149b68        vault:latest        "docker-entrypoint..."   ...    vault
32ff06ac02da        postgres            "docker-entrypoint..."   ...    postgres

»Task 2: Examine the Vault environment

During the demo machine provisioning, the /scripts/vault.sh script was executed to perform the following:

• Created a policy named order
• Enabled the transit secrets engine and created an encryption key named order
• Enabled the database secrets engine and created a role named order

View the vault log:

$ docker logs vault

==> Vault server configuration:
             Api Address: http://0.0.0.0:8200
                     Cgo: disabled
         Cluster Address: https://0.0.0.0:8201
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
           Recovery Mode: false
                 Storage: inmem
                 Version: Vault v1.3.1
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
    $ export VAULT_ADDR='http://0.0.0.0:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: Wgx++e73Y/htvXYzSZHAQ/3iE+Q0cuaBvTsM4ujn2xA=
Root Token: root

$ docker logs vault

==> Vault server configuration:
             Api Address: http://0.0.0.0:8200
                     Cgo: disabled
         Cluster Address: https://0.0.0.0:8201
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
           Recovery Mode: false
                 Storage: inmem
                 Version: Vault v1.3.1
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
    $ export VAULT_ADDR='http://0.0.0.0:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: Wgx++e73Y/htvXYzSZHAQ/3iE+Q0cuaBvTsM4ujn2xA=
Root Token: root

Notice that the log indicates that the Vault server is running in the dev mode, and the root token is root.

You can visit the Vault UI at http://localhost:8200/ui. Enter root and click Sign In.

Select the transit/ secrets engine, and you should find an encryption key named order.

Under the Policies, verify that the order policy exists.

This order policy is for the application. It permits read on the database/creds/order path so that the demo app can get a dynamically generated database credential from Vault. Therefore the PostgreSQL credentials are not hard-coded anywhere.

path "database/creds/order"
{
  capabilities = ["read"]
}

path "database/creds/order"
{
  capabilities = ["read"]
}

An update permission allows the app to request data encryption and decryption using the order encryption key in Vault.

## ...snip...
path "transit/decrypt/order" {
  capabilities = ["update"]
}

path "transit/encrypt/order" {
  capabilities = ["update"]
}
## ...snip...

## ...snip...
path "transit/decrypt/order" {
  capabilities = ["update"]
}

path "transit/encrypt/order" {
  capabilities = ["update"]
}
## ...snip...

»Task 3: Examine the Spring container

Remember that the VaultDemoOrderServiceApplication class logs messages during the successful execution of initIt():

@PostConstruct
    public void initIt() throws Exception {
        logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
        logger.info("Got DB User: " + dbUser);
    // ...
}

@PostConstruct
    public void initIt() throws Exception {
        logger.info("Got Vault Token: " + sessionManager.getSessionToken().getToken());
        logger.info("Got DB User: " + dbUser);
    // ...
}

Verify that the log indicates that the demo app obtained a database credentials from Vault successfully:

$ docker logs spring | grep Got

...VaultDemoOrderServiceApplication : Got Vault Token: root
...VaultDemoOrderServiceApplication : Got DB User: v-token-order-rywqz61432yyx2x27w8r-1524067226

$ docker logs spring | grep Got

...VaultDemoOrderServiceApplication : Got Vault Token: root
...VaultDemoOrderServiceApplication : Got DB User: v-token-order-rywqz61432yyx2x27w8r-1524067226

Create a new shell session in the spring container.

$ docker exec -it spring sh
/ #

$ docker exec -it spring sh
/ #

Review the bootstrap.yaml file:

/ #  cat bootstrap.yaml
spring.application.name: spring-vault-demo
spring.cloud.vault:
  authentication: TOKEN
  token: ${VAULT_TOKEN}
  host: localhost
  port: 8200
  scheme: http
  fail-fast: true
  config.lifecycle.enabled: true
  generic:
    enabled: true
    backend: secret
  database:
    enabled: true
    role: order
    backend: database
spring.datasource:
  url: jdbc:postgresql://localhost:5432/postgres

/ #  cat bootstrap.yaml
spring.application.name: spring-vault-demo
spring.cloud.vault:
  authentication: TOKEN
  token: ${VAULT_TOKEN}
  host: localhost
  port: 8200
  scheme: http
  fail-fast: true
  config.lifecycle.enabled: true
  generic:
    enabled: true
    backend: secret
  database:
    enabled: true
    role: order
    backend: database
spring.datasource:
  url: jdbc:postgresql://localhost:5432/postgres

The client token was injected into the spring container as an environment variable (VAULT_TOKEN) by Vagrant.

Enter exit to close the shell session in the spring container.

/ # exit

/ # exit

»Task 4: Examine the PostgreSQL database

Connect to the PostgreSQL database running in the postgres container:

$ docker exec -it postgres psql -U postgres -d postgres

psql (10.3 (Debian 10.3-1.pgdg90+1))
Type "help" for help.


postgres=# \d orders
                                          Table "public.orders"
    Column     |            Type             | Collation | Nullable |              Default
---------------+-----------------------------+-----------+----------+------------------------------------
 id            | bigint                      |           | not null | nextval('orders_id_seq'::regclass)
 customer_name | character varying(60)       |           | not null |
 product_name  | character varying(20)       |           | not null |
 order_date    | timestamp without time zone |           | not null |
Indexes:
    "orders_pkey" PRIMARY KEY, btree (id)

$ docker exec -it postgres psql -U postgres -d postgres

psql (10.3 (Debian 10.3-1.pgdg90+1))
Type "help" for help.


postgres=# \d orders
                                          Table "public.orders"
    Column     |            Type             | Collation | Nullable |              Default
---------------+-----------------------------+-----------+----------+------------------------------------
 id            | bigint                      |           | not null | nextval('orders_id_seq'::regclass)
 customer_name | character varying(60)       |           | not null |
 product_name  | character varying(20)       |           | not null |
 order_date    | timestamp without time zone |           | not null |
Indexes:
    "orders_pkey" PRIMARY KEY, btree (id)

Let's list the existing database roles.

postgres-# \du
                                                     List of roles
                   Role name                   |                         Attributes                         | Member of
-----------------------------------------------+------------------------------------------------------------+-----------
 postgres                                      | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 v-token-order-rywqz61432yyx2x27w8r-1524067226 | Password valid until 2018-04-18 20:56:31+00                | {}

postgres-# \du
                                                     List of roles
                   Role name                   |                         Attributes                         | Member of
-----------------------------------------------+------------------------------------------------------------+-----------
 postgres                                      | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 v-token-order-rywqz61432yyx2x27w8r-1524067226 | Password valid until 2018-04-18 20:56:31+00                | {}

Notice that there is a role name starting with v-token-order- which was dynamically created by the database secrets engine.

Enter \q to exit out of the psql session, or you can open another terminal and SSH into the demo virtual machine.

»Step 3: Run the demo application

If everything looked fine in Step 2, you are ready to write some data.

You have verified in the spring log that the demo app successfully retrieved a database credential from the Vault server during its initialization.

The next step is to send a new order request via the demo app's orders API (http://localhost:8080/api/orders).

Create a file payload.json with the following contents.

$ tee payload.json <<EOF
{
  "customerName": "John",
  "productName": "Nomad"
}
EOF

$ tee payload.json <<EOF
{
  "customerName": "John",
  "productName": "Nomad"
}
EOF

Send the file in a request using cURL.

$ curl --request POST --header "Content-Type: application/json" \
       --data @payload.json http://localhost:8080/api/orders | jq

$ curl --request POST --header "Content-Type: application/json" \
       --data @payload.json http://localhost:8080/api/orders | jq

You should see this output:

{
  "id": 1,
  "customerName": "John",
  "productName": "Nomad",
  "orderDate": "2018-04-18T22:07:42.916+0000"
}

{
  "id": 1,
  "customerName": "John",
  "productName": "Nomad",
  "orderDate": "2018-04-18T22:07:42.916+0000"
}

The order data you sent gets encrypted by Vault. The database only sees the ciphertext. Let's verify that the order information stored in the database is encrypted.

$ docker exec -it postgres psql -U postgres -d postgres

postgres=# select * from orders;
 id |                     customer_name                     | product_name |       order_date
----+-------------------------------------------------------+--------------+-------------------------
  1 | vault:v1:UwL3HnyqTUac5ElS5WYAuNg3NdIMFtd6vvwukL+FaKun | Nomad        | 2018-04-18 22:07:42.916
(1 rows)

postgres=# \q

$ docker exec -it postgres psql -U postgres -d postgres

postgres=# select * from orders;
 id |                     customer_name                     | product_name |       order_date
----+-------------------------------------------------------+--------------+-------------------------
  1 | vault:v1:UwL3HnyqTUac5ElS5WYAuNg3NdIMFtd6vvwukL+FaKun | Nomad        | 2018-04-18 22:07:42.916
(1 rows)

postgres=# \q

Now, retrieve the order data via the orders API:

$ curl --header "Content-Type: application/json" \
       http://localhost:8080/api/orders | jq

$ curl --header "Content-Type: application/json" \
       http://localhost:8080/api/orders | jq

The output should look like this:

[
  {
    "id": 1,
    "customerName": "John",
    "productName": "Nomad",
    "orderDate": "2018-04-18T22:07:42.916+0000"
  }
]

[
  {
    "id": 1,
    "customerName": "John",
    "productName": "Nomad",
    "orderDate": "2018-04-18T22:07:42.916+0000"
  }
]

The customer names should be readable. Remember that the order policy permits the demo app to encrypt and decrypt data using the order encryption key in Vault.

»Web UI

Vault UI makes it easy to decrypt the data.

1. In the Secrets tab, select transit > orders, and select Key actions.

2. Select Decrypt from the transit actions. Now, copy the ciphertext from the orders table and paste it in.

3. Click Decrypt.

4. Click Decode from base64 to reveal the customer name.

»Step 4: Reloading the Static Secrets

Now, let's test another API endpoint, api/secret provided by the demo app. A plain old Java object, Secret defines a get method for key and value. The SecretController.java defines an API endpoint, api/secret.

// SecretController.java
package com.hashicorp.vault.spring.demo;
// ...

@RefreshScope
@RestController
public class SecretController {

  @Value("${secret:n/a}")
  String secret;

  @RequestMapping("/api/secret")
  public Secret secret() {
    return new Secret("secret", secret);
  }
}

// SecretController.java
package com.hashicorp.vault.spring.demo;
// ...

@RefreshScope
@RestController
public class SecretController {

  @Value("${secret:n/a}")
  String secret;

  @RequestMapping("/api/secret")
  public Secret secret() {
    return new Secret("secret", secret);
  }
}

Remember from Step 2 that the order policy granted permissions on the secret/spring-vault-demo path.

path "secret/spring-vault-demo" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

path "secret/spring-vault-demo" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

The demo app retrieved the secret from secret/spring-vault-demo and has a local copy. If someone (or perhaps another app) updates the secret, it makes the secret held by the demo app to be obsolete.

Spring offers Spring Boot Actuator which can be used to facilitate the reloading of the static secret.

»Task 1: Read the secret

The initial key-value was set by Vagrant during the provisioning. (See the Vagrantfile at line 48.)

Let's invoke the demo app's secret API (api/secret).

$ curl -s http://localhost:8080/api/secret | jq

$ curl -s http://localhost:8080/api/secret | jq

The output should look like this:

{
  "key": "secret",
  "value": "hello-vault"
}

{
  "key": "secret",
  "value": "hello-vault"
}

This is the secret that the demo app knows about.

»Task 2: Update the Secrets

Now, update the secret stored in Vault using the API.

$ curl --header "X-Vault-Token: root" \
       --request POST \
       --data '{ "secret": "my-api-key" }' \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo

$ curl --header "X-Vault-Token: root" \
       --request POST \
       --data '{ "secret": "my-api-key" }' \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo

Verify that the secret value was updated.

$ curl --header "X-Vault-Token: root" \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo | jq

$ curl --header "X-Vault-Token: root" \
       http://127.0.0.1:8200/v1/secret/spring-vault-demo | jq

The output will include a large data structure which starts with the following:

{
  "request_id": "514601e4-a790-3dc6-14b0-537d6982a6c6",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 2764800,
  "data": {
    "secret": "my-api-key"
  }
}

{
  "request_id": "514601e4-a790-3dc6-14b0-537d6982a6c6",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 2764800,
  "data": {
    "secret": "my-api-key"
  }
}

»Task 3: Refresh the secret on the demo app

Run the demo app's secret API again:

$ curl -s http://localhost:8080/api/secret | jq

$ curl -s http://localhost:8080/api/secret | jq

You'll see this output:

{
  "key": "secret",
  "value": "hello-vault"
}

{
  "key": "secret",
  "value": "hello-vault"
}

The current value stored in Vault is now my-api-key; however, the demo app still holds hello-vault.

Spring provides an actuator which can be leveraged to refresh the secret value. At line 54 of the vault-guides/secrets/spring-cloud-vault/pom.xml, you see that the actuator was added to the project.

<!-- ... -->
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- ... -->

<!-- ... -->
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- ... -->

Let's refresh the secret using the actuator:

$ curl -s --request POST http://localhost:8080/actuator/refresh | jq
[
  "secret"
]

$ curl -s --request POST http://localhost:8080/actuator/refresh | jq
[
  "secret"
]

Read back the secret from the demo app again:

$ curl -s http://localhost:8080/api/secret | jq
{
  "key": "secret",
  "value": "my-api-key"
}

$ curl -s http://localhost:8080/api/secret | jq
{
  "key": "secret",
  "value": "my-api-key"
}

It should display the correct value.

When you are done exploring the demo implementation, you can destroy the virtual machine:

$ vagrant destroy

demo: Are you sure you want to destroy the 'demo' VM? [y/N] y
==> demo: Forcing shutdown of VM...
==> demo: Destroying VM and associated drives...

$ vagrant destroy

demo: Are you sure you want to destroy the 'demo' VM? [y/N] y
==> demo: Forcing shutdown of VM...
==> demo: Destroying VM and associated drives...

»Help and Reference

• Manage secrets, access, and encryption in the public cloud with Vault
• Spring Cloud Vault
• Transit Secrets Engine