Encrypting sensitive data in puppet using hiera-eyaml
Puppet manifests can hold a lot of sensitive information. Sensitive information like passwords or certificates are used in the configuration of many applications. Exposing them in a puppet manifest is not ideal and may conflict with an organization's compliance policies. That is why data separation is very important aspect of secure puppet code.
Hiera is a pluggable Hierarchical Database. Hiera can help by keeping data out of puppet manifests. Puppet classes can look for data in hiera and hiera would search hierarchically and provide the first instance of value. Although Hiera is able to provide data separation, it cannot ensure security of sensitive information. Anyone with access to the Hiera data store will be able to see the data. Enter
Hiera-eyaml. Hiera-eyaml is a backend for Hiera that provides per-value encryption of sensitive data within yaml files to be used by Puppet. The following puppet module can be used to manage hiera with eyaml support. https://forge.puppetlabs.com/hunner/hiera puppet module. The module class can be used like below,
modules/profile/manifests/hieraconf.ppclass profile::hieraconf { # hiera configuration class { 'hiera': hierarchy => [ '%{environment}/%{calling_class}', '%{environment}', '%{fqdn}', 'common', 'accounts', 'dev' ], } }
The /etc/hiera.conf would look like following after the puppet run,
/etc/puppet/hiera.yaml# managed by puppet --- :backends: - yaml :logger: console :hierarchy: - "%{environment}/%{calling_class}" - "%{environment}" - "%{fqdn}" - common - accounts - dev :yaml: :datadir: /etc/puppet/hieradata
Moving data to Hiera In following example, diamond collector for Mongodb does have data like, hosts, user and password. The collector is only enabled for grafana.pythian.com host.
modules/profile/manifests/diamond_coll.pp [..] diamond::collector { 'MongoDBCollector': options => { enabled => $fqdn ? { /grafana.pythian.com/ =>True, default => false }, hosts => 'abc.pythian.com,xyz.pythian.com', user => 'grafana', passwd => 'xxxx', } }
To move the data to hiera, create_resources function can be used in the manifest.
modules/profile/manifests/diamond_coll.ppclass profile::diamond_coll{ [..] $mycollectors = hiera('diamond::collectors', {}) create_resources('diamond::collector', $mycollectors) [..] }
Then a new yaml file can be created and diamond::collectors code for MongoDBCollector can be abstracted like below,
hieradata/grafana.pythian.com.yaml --- diamond::collectors: MongoDBCollector: options: enabled: True hosts: abc.pythian.com,xyz.pythian.com user: grafana passwd: xxxx
Moving data to Hiera-eyaml Hiera puppet code can be changed to following to enable eyaml.
class profile::hieraconf { # hiera configuration class { 'hiera': hierarchy => [ '%{environment}/%{calling_class}', '%{environment}', '%{fqdn}', 'common', 'accounts', 'dev' ], eyaml => true, eyaml_datadir => '/etc/puppet/hieradata', eyaml_extension => 'eyaml', } }
This will add eyaml backend to puppet after a puppet run on puppet server. Puppet modules does following to achieve this. Update 1. The hiera-eyaml gem will be installed. 2. Following keys will be created for hiera-eyaml using 'eyaml createkeys'. /etc/puppet/keys/private_key.pkcs7.pem /etc/puppet/keys/public_key.pkcs7.pem 3. Update /etc/hiera.conf. The /etc/hiera.conf would look like following after the puppet run,
/etc/puppet/hiera.yaml# managed by puppet --- :backends: - eyaml - yaml :logger: console :hierarchy: - "%{environment}/%{calling_class}" - "%{environment}" - "%{fqdn}" - common - accounts - dev :yaml: :datadir: /etc/puppet/hieradata :eyaml: :datadir: /etc/puppet/hieradata :extension: eyaml :pkcs7_private_key: /etc/puppet/keys/private_key.pkcs7.pem :pkcs7_public_key: /etc/puppet/keys/public_key.pkcs7.pem
Puppetmaster need to be restarted after this as changes to hiera.conf would need a restart to apply.
Using eyaml command line Eyaml commands need to be used in a directory with keys directory(In this example /etc/puppet). Following command can be used to encrypt a password. The command would give us two options, string and block.
# eyaml encrypt -p Enter password: **** string: ENC[PKCS7,MIIBeQYJKoZIhvcN[..]Fg3jAmdlCLbQ] OR block: > ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAQMo0dyWRmBC30TVDVxEOoClgUsxtzDSXmrJL pz3ydhvG0Ll96L6F2WbclmGtpaqksbpc98p08Ri8seNmSp4sIoZWaZs6B2Jk BLOIJBZfSIcHH8tAdeV4gBS1067OD7I+ucpCnjUDTjYAp+PdupyeaBzkoWMA X/TrmBXW39ndAATsgeymwwG/EcvaAMKm4b4qH0sqIKAWj2qeVJTlrBbkuqCj qjOO/kc47gKlCarJkeJH/rtErpjJ0Et+6SJdbDxeSbJ2QhieXKGAj/ERCoqh hXJiOofFuAloAAUfWUfPKnSZQEhHCPDkeyhgDHwc8akWjC4l0eeorZgLPcs1 1oQJqTA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBC+JDHdj2M2++mFu+pv ORXmgBA/Ng596hsGFg3jAmdlCLbQ]
To decrypt following command can be used.
# eyaml decrypt -s 'ENC[PKCS7,MIIBeQYJKoZIhvcN[..]Fg3jAmdlCLbQ]' test
The encrypted string or block can be used in hiera. While using our previous example, the hiera file would look like following. We also have to rename the file to .eyaml from .yaml.
hieradata/grafana.pythian.com.eyaml --- diamond::collectors: MongoDBCollector: options: enabled: True hosts: abc.pythian.com,xyz.pythian.com user: grafana passwd: ENC[PKCS7,MIIBeQYJKoZIhvcN[..]Fg3jAmdlCLbQ]
Encrypting certificates Following is a standard file resource used to copy an ssl certificate..
environments/production/manifests/logstash.pythian.com.ppfile { '/etc/logstash/certs/logstash-forwarder.crt': ensure => present, mode => '0644', owner => 'root', group => 'root', source => 'puppet:///modules/logstash/logstash-forwarder.crt', }
The file resource can be moved to hiera using hiera_hash.
environments/production/manifests/logstash.pythian.com.pp$fileslog = hiera_hash('fileslog') create_resources ( file, $fileslog )
The data can be added to a yaml file.
hieradata/common.yaml--- files: '/etc/logstash-forwarder/certs/logstash-forwarder.crt': ensure : present mode : '0644' owner : 'root' group : 'root' source : 'puppet:///modules/logstash/logstash-forwarder.key'
To encrypt data, following command can be used.
# eyaml encrypt -f modules/logstash/files/logstash-forwarder.crt
The returned string value can be added using content parameter of file resource.
hieradata/common.eyaml [..] files: '/etc/logstash-forwarder/certs/logstash-forwarder.crt': ensure : present mode : '0644' owner : 'root' group : 'root' content : 'ENC[PKCS7,MIIB+wYJKoZI[..]C609Oc2QUvxARaw==]'
The above examples covers encrypting strings and files, which constitutes most of the sensitive data used in puppet code. Incorporating hiera-eyaml into puppet work-flow will ensure compliance and security of sensitive data.
