Cassandra and Vault – distributed secret store

Hello all, I’m doing a mini-series of blog posts on using Cassandra and Hashicorp Vault.

For the first blog in this series, I’ll look at using Cassandra as a Datastore for Vault secrets, then how to generate Cassandra secrets using Vault, and I’ll also demonstrate how to manage Cassandra credentials from Vault.

Attention: This is not intended to be followed straight on for production systems as several key security features are not done properly (ex: Disabling Cassandra superuser).

The definition of Vault is:

Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.

There are several ways to deploy Vault. And several storage engines, Cassandra being one of them. Cassandra gives the advantage of a distributed, highly-available, geo-replicated storage. Since we want to manage secrets, this can be an interesting point.

From here, I’m going to guide you how to use Cassandra as a Vault datastore.

1. Setup Cassandra

First, we need to setup Cassandra for storing the data. In this guide, I’m using CCM to achieve this.

Create a 3.11.1, 3 node cassandra cluster

$ ccm create triple3111 -v3.11.2 -n 3 -s

$ ccm status
Cluster: 'triple3111'
---------------------
node3: UP
node2: UP
node1: UP
$ ccm node1 nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  166.89 KiB  1            100.0%            6c160c38-9041-4109-a98f-c9100f0f297a  rack1
UN  127.0.0.2  153.52 KiB  1            100.0%            900c1007-b3e9-402b-8ab1-9f1f093a0720  rack1
UN  127.0.0.3  174.13 KiB  1            100.0%            5f54cf00-533e-4400-a898-ab2bd3fe8c17  rack1

$ ccm node1 cqlsh -e "CREATE KEYSPACE \"vault\" WITH REPLICATION = {'class': 'NetworkTopologyStrategy','datacenter1': 3};"
$ ccm node1 cqlsh -e "CREATE TABLE \"vault\".\"entries\" (bucket text,key text,value blob,PRIMARY KEY (bucket, key)) WITH CLUSTERING ORDER BY (key ASC);"

$ ccm stop
$ # Edit cassandra.yaml on the 3 nodes
$ ccm start

$ ccm node1 cqlsh -u cassandra -p cassandra
[email protected]> list users;
name | super
-----------+-------
cassandra | True

$ mkdir vault
$ cd vault
$ wget "https://releases.hashicorp.com/vault/0.9.5/vault_0.9.5_linux_amd64.zip"
$ unzip vault_0.9.5_linux_amd64.zip

$ ./vault -h
Usage: vault <command> [args]
Common commands:
    read        Read data and retrieves secrets
    write       Write data, configuration, and secrets
    delete      Delete secrets and configuration
    list        List data or secrets
    login       Authenticate locally
    server      Start a Vault server
    status      Print seal and HA status
    unwrap      Unwrap a wrapped secret
Other commands:
    audit          Interact with audit devices
    auth           Interact with auth methods
    lease          Interact with leases
    operator       Perform operator-specific tasks
    path-help      Retrieve API help for paths
    policy         Interact with policies
    secrets        Interact with secrets engines
    ssh            Initiate an SSH session
    token          Interact with tokens

$ vim config.json
storage "cassandra" {
  hosts            = "localhost"
  consistency      = "LOCAL_QUORUM"
  protocol_version = 3
  username = "cassandra"
  password = "cassandra"
}
listener "tcp" {
  address     = "127.0.0.1:8200"
  tls_disable = 1
}

$ ./vault server -dev -config config.json
==> Vault server configuration:
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", tls: "disabled")
              Listener 2: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
        Redirect Address: https://127.0.0.1:8200
                 Storage: cassandra
                 Version: Vault v0.9.5
             Version Sha: 36edb4d42380d89a897e7f633046423240b710d9
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
You may need to set the following environment variable:
    $ export VAULT_ADDR='https://127.0.0.1:8200'
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
Unseal Key: irNaBhZFlHjbx/2y76/+Hd0u41XxxaE6HbuCGWzc0ME=
Root Token: f467dc15-4381-061a-7fe3-2561baaaa6ce
Development mode should NOT be used in production installations!
==> Vault server started! Log data will stream in below:

At this point we have vault running. Let’s export the variable and insert some secrets

$ export VAULT_ADDR='https://127.0.0.1:8200'
$ vault write secret/hello value=world
Success! Data written to: secret/hello

$ ccm node1 cqlsh -u cassandra -p cassandra
[email protected]> SELECT count(*) FROM "vault"."entries";
 count
-------
    48