Cassandra and Vault - securing C* secrets
Hello all! This is the second blog post in this mini-series (You can read the first one
here.) This time I will go through the steps and demonstrate how to setup Vault to manage Cassandra credentials! Including generating, leasing, revoking, etc. Vault, as explained on the previous blog post, is a secret store with a lot of control over those secrets. So let's setup Vault to do this for our Cassandra cluster. Note, that a Secret Engine, as explained in this blog post, is not ideal for Super User credentials, as those credentials would expire and regenerate. Making impossible, for example, for Vault to communicate with the Cassandra Cluster. Use the Vault K/V store for those kind of secrets.
Learn more about Pythian's services for Cassandra.
Setup Secret Engine
Vault needs to be setup to allow a Secret Engine. Secret Engines allow Vault to manage secrets, via an API, of a given engine (ex: Cassandra, AWS, SSH, etc...). So we are going to now setup our Vault installation to use the Cassandra Secret Engine ( official documentation). I will explain in detail what each configuration does, so you can adapt for your specific needs. Let's start!Enabling Vault Secret Engine
Assuming your Vault is running, enabling the secret engine with the following command: [code language="bash"] $ ./vault secrets enable database </code> Now, we need to setup the Cassandra Secret Engine: <code language='bash'> $ ./vault write database/config/my-cassandra-database \ > plugin_name="cassandra-database-plugin" \ > hosts=127.0.0.1 \ > protocol_version=4 \ > username=cassandra \ > password=cassandra \ > tls=false \ > allowed_roles=read-role,cassandra-all WARNING! The following warnings were returned from Vault: * Read access to this endpoint should be controlled via ACLs as it will return the connection details as is, including passwords, if any. [/code] The breakdown of this command is the following:- vault write database/config/my-cassandra-database: Write a database configuration called "my-cassandra-database"
- plugin_name="cassandra-database-plugin": Use the Cassandra plugin
- protocol_version=4: Protocol version to connect to Cassandra
- username=cassandra: Cassandra super user
- password=cassandra: Cassandra super user password
- tls=false: Use TLS or not
- allowed_roles=read-role,cassandra-all: What roles will be allowed to use this plugin
Configuring Cassandra Roles
The secret engine is useless if it doesn't know how and what credentials to create. So now we are going to setup 2 roles for our Secret Engine. One role read-role will give full READ only to all keyspaces. Another role, cassandra-all will create a role that has ALL access on all keyspaces. For this part, is important to know how to create a user and granting access to resources in Cassandra. For more information regarding that, consult the user documentation and the grant documentation for Cassandra. NOTE: CREATE USER is currently deprecated, being replaced by CREATE ROLE, but for backward compatibility and Vault plugin internals, CREATE USER is used. I will start creating a READ only role. [code language="bash"] $ ./vault write database/roles/read-role \ db_name=my-cassandra-database \ creation_statements="CREATE USER '' WITH PASSWORD '' NOSUPERUSER; \ GRANT SELECT ON ALL KEYSPACES TO ;" \ default_ttl="1h" \ max_ttl="24h" [/code] And now a ALL role: [code language="bash"] $ ./vault write database/roles/cassandra-all \ db_name=my-cassandra-database \ creation_statements="CREATE USER '' WITH PASSWORD '' NOSUPERUSER; \ GRANT ALL ON ALL KEYSPACES TO ;" \ default_ttl="1h" \ max_ttl="24h" [/code] Breakdown of the commands:- vault write database/roles/cassandra-all: Write a database role called "cassandra-all"
- db_name=my-cassandra-database: Use the database called "my-cassandra-database"
- creation_statements="...": How to create this user
- default_ttl="1h": If a lease is not Renewed, how long it will last
- max_ttl="24h": The lease will expire after this time, regardless of renewal
Using the Secret Engine
To use the secret generation, we just call Vault commands to generate credentials. Lets create both Roles: [code language="bash"] $ ./vault read database/creds/read-role Key Value --- ----- lease_id database/creds/read-role/d671e0ca-9c45-6408-d0ee-7973f81285fb lease_duration 1h lease_renewable true password A1a-8p8sp7w99u8v6p62 username v_root_read_role_5754r5vy9uwrww1qs4pq_1520264273 ./vault read database/creds/cassandra-all Key Value --- ----- lease_id database/creds/cassandra-all/15c576fa-3156-7f61-3293-e570831f1279 lease_duration 1h lease_renewable true password A1a-rs7uqwwswp5z71w2 username v_root_cassandra_all_63p22r2s60vxsswwq7xt_1520520093 [/code] The commands are explanatory, you tell Vault that you want to READ, vault read, a credential for a given role, database/creds/read-role. Vault will generate a credential each time you read. Then you can use the displayed username and password to connect to Cassandra. As I said, each time you read, a credential is generated!! So if you need to re-use the same credential, you renew the lease. For that, you take note of the lease_id (ex: database/creds/my-role/d671e0ca-9c45-6408-d0ee-7973f81285fb) , and issue a renew command: [code language="bash"] $ ./vault lease renew database/creds/read-role/d671e0ca-9c45-6408-d0ee-7973f81285fb [/code] But on the contrary, you need to revoke a lease, issue a revoke command: [code language="bash"] $ ./vault lease revoke database/creds/read-role/d671e0ca-9c45-6408-d0ee-7973f81285fb [/code] Now, that we know how to manage and generate dynamic secrets, let's test!Testing
Testing is the easy part, it's a matter of logging in and test! Since we revoked the read role, I will create a new one: [code language="bash"] $ ./vault read database/creds/read-role Key Value --- ----- lease_id database/creds/read-role/71af19de-9038-6aab-fa0e-6e587108e94e lease_duration 1h lease_renewable true password A1a-25wyp295q476u848 username v_root_read_role_wptz6yrw7q3649ss13yt_1520265497 [/code] Now using the Cassandra super user, lets see if all the users are there: [code language="bash"] $ ccm node3 cqlsh -u cassandra -p cassandra cassandra@cqlsh> select * from system_auth.roles; role | can_login | is_superuser | member_of | salted_hash ------------------------------------------------------+-----------+--------------+-----------+-------------------------------------------------------------- v_root_cassandra_all_63p22r2s60vxsswwq7xt_1520520093 | True | False | null | $2a$10$xfq5Mx8a6iAJSmKisC8m4eVn24dIfcV4cYFnMX5j5InbFNTJaVuh2 v_root_read_role_w6xur1p369rrzv5xvv9r_1520520122 | True | False | null | $2a$10$FId4t.N./ep8YND3vN9zsuXIjXyF.qxlhndHwhcrHk4UPJvoCuirC cassandra | True | True | null | $2a$10$Nt/bO/pcyZ1dkKriipC.meL4Pdrke2w2MG2Q6uhb5fjlB4p9Qc8sy (3 rows) cassandra@cqlsh> exit [/code] All good! Now lets test the GRANT permissions: [code language="bash"] $ ccm node1 cqlsh -u v_root_read_role_wptz6yrw7q3649ss13yt_1520265497 -p A1a-25wyp295q476u848 Connected to triple3111 at 127.0.0.1:9042. [cqlsh 5.0.1 | Cassandra 3.11.1 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. v_root_read_role_wptz6yrw7q3649ss13yt_1520265497@cqlsh> create KEYSPACE vault_test with replication = {'class': 'NetworkTopologyStrategy', 'datacenter1':3}; Unauthorized: Error from server: code=2100 [Unauthorized] message="User v_root_read_role_wptz6yrw7q3649ss13yt_1520265497 has no CREATE permission on <all keyspaces> or any of its parents" v_root_read_role_wptz6yrw7q3649ss13yt_1520265497@cqlsh> exit; [/code] Again, all good! Now the other user: [code language="bash"] $ ccm node1 cqlsh -u v_root_cassandra_all_63p22r2s60vxsswwq7xt_1520520093 -p A1a-rs7uqwwswp5z71w2 Connected to triple3111 at 127.0.0.1:9042. [cqlsh 5.0.1 | Cassandra 3.11.1 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. v_root_cassandra_all_63p22r2s60vxsswwq7xt_1520520093@cqlsh> create KEYSPACE vault_test with replication = {'class': 'NetworkTopologyStrategy', 'datacenter1':3}; v_root_cassandra_all_63p22r2s60vxsswwq7xt_1520520093@cqlsh> [/code] All good! It seems all is working, and I hope that this is useful for anyone that needs to manage their Cassandra secrets!Learn more about Pythian's services for Cassandra.
