Cassandra and Vault – Distributed Secret Store

Posted in: Cassandra, DevOps, Open Source

Hello all, I’m doing a mini-series of blog posts on using Cassandra and Hashicorp Vault.

For the first blog in this series, I’ll look at using Cassandra as a Datastore for Vault secrets, then how to generate Cassandra secrets using Vault, and I’ll also demonstrate how to manage Cassandra credentials from Vault.

Attention: This is not intended to be followed straight on for production systems as several key security features are not done properly (ex: Disabling Cassandra superuser).

The definition of Vault is:

Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.

There are several ways to deploy Vault. And several storage engines, Cassandra being one of them. Cassandra gives the advantage of a distributed, highly-available, geo-replicated storage. Since we want to manage secrets, this can be an interesting point.

From here, I’m going to guide you how to use Cassandra as a Vault datastore.

1. Setup Cassandra

First, we need to setup Cassandra for storing the data. In this guide, I’m using CCM to achieve this.

Create a 3.11.1, 3 node cassandra cluster

$ ccm create triple3111 -v3.11.2 -n 3 -s
Check if the cluster is running
$ ccm status
Cluster: 'triple3111'
node3: UP
node2: UP
node1: UP

$ ccm node1 nodetool status
Datacenter: datacenter1
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  166.89 KiB  1            100.0%            6c160c38-9041-4109-a98f-c9100f0f297a  rack1
UN  153.52 KiB  1            100.0%            900c1007-b3e9-402b-8ab1-9f1f093a0720  rack1
UN  174.13 KiB  1            100.0%            5f54cf00-533e-4400-a898-ab2bd3fe8c17  rack1
Create the Vault keyspaces (You can find more information here)
$ ccm node1 cqlsh -e "CREATE KEYSPACE \"vault\" WITH REPLICATION = {'class': 'NetworkTopologyStrategy','datacenter1': 3};"
$ ccm node1 cqlsh -e "CREATE TABLE \"vault\".\"entries\" (bucket text,key text,value blob,PRIMARY KEY (bucket, key)) WITH CLUSTERING ORDER BY (key ASC);"
Now, we need to enable Authentication and Authorization, for that, we stop the cluster, edit the configs, and start again
$ ccm stop
$ # Edit cassandra.yaml on the 3 nodes
$ ccm start
Confirm that this is working as expected:
$ ccm node1 cqlsh -u cassandra -p cassandra
cassandra@cqlsh> list users;
name | super
cassandra | True
Everything Cassandra is done at this point! Let’s move to Vault!

2. Install and Configure Vault

Now that we have Cassandra set and running, let’s get vault working!
Create a directory to install vault, and download vault.
$ mkdir vault
$ cd vault
$ wget ""
$ unzip
Let’s test vault:
$ ./vault -h
Usage: vault <command> [args]

Common commands:
    read        Read data and retrieves secrets
    write       Write data, configuration, and secrets
    delete      Delete secrets and configuration
    list        List data or secrets
    login       Authenticate locally
    server      Start a Vault server
    status      Print seal and HA status
    unwrap      Unwrap a wrapped secret

Other commands:
    audit          Interact with audit devices
    auth           Interact with auth methods
    lease          Interact with leases
    operator       Perform operator-specific tasks
    path-help      Retrieve API help for paths
    policy         Interact with policies
    secrets        Interact with secrets engines
    ssh            Initiate an SSH session
    token          Interact with tokens
At this moment, Vault is not yet running, and we just confirmed that the vault binary is in working condition. For this blog, I’m running vault in dev mode (-dev), but since there is a storage engine present, the second time you would start vault with the storage engine and in dev mode, it will fail to start. To avoid this, either TRUNCATE the Cassandra table, or use Vault in non-dev mode.
First, create a vault config file (documentation), setting Cassandra as the data engine:
$ vim config.json
storage "cassandra" {
  hosts            = "localhost"
  consistency      = "LOCAL_QUORUM"
  protocol_version = 3
  username = "cassandra"
  password = "cassandra"

listener "tcp" {
  address     = ""
  tls_disable = 1
Now let’s start vault:
$ ./vault server -dev -config config.json
==> Vault server configuration:

                     Cgo: disabled
         Cluster Address:
              Listener 1: tcp (addr: "", cluster address: "", tls: "disabled")
              Listener 2: tcp (addr: "", cluster address: "", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
        Redirect Address:
                 Storage: cassandra
                 Version: Vault v0.9.5
             Version Sha: 36edb4d42380d89a897e7f633046423240b710d9

WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variable:

    $ export VAULT_ADDR=''

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: irNaBhZFlHjbx/2y76/+Hd0u41XxxaE6HbuCGWzc0ME=
Root Token: f467dc15-4381-061a-7fe3-2561baaaa6ce

Development mode should NOT be used in production installations!

==> Vault server started! Log data will stream in below:

At this point we have vault running. Let’s export the variable and insert some secrets

$ export VAULT_ADDR=''
$ vault write secret/hello value=world
Success! Data written to: secret/hello
Now we can check if Cassandra is getting data:
$ ccm node1 cqlsh -u cassandra -p cassandra
cassandra@cqlsh> SELECT count(*) FROM "vault"."entries";

It is working! In my next blog post, I will demonstrate how to configure and use the Cassandra database engine to generate Cassandra secrets!



Interested in working with Carlos? Schedule a tech call.

About the Author

Carlos Rolo is a Datastax Certified Cassandra Architect, and has deep expertise with distributed architecture technologies. Carlos is driven by challenge, and enjoys the opportunities to discover new things and new ways of learning that come with working at Pythian. He has become known and trusted by customers and colleagues for his ability to understand complex problems, and to work well under pressure. He prides himself on being a tenacious problem solver, while remaining a calm and positive presence on any team. When Carlos isn’t working he can be found playing water polo or enjoying the his local community. Carlos holds a Bachelor of Electro-technical Engineering, and a Master of Control Systems and Automation.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *