Making it easier to graph your infrastructure’s performance data
Today I would like to share a story with you about the development of a Puppet module for
Grafana and its release on the
Puppet Forge. But first, I'd like to provide some context so you can understand where Grafana fits in and why I feel this is important. Those of you that know me have likely heard me talk about the importance of data-driven decision making, and more specifically some of the tools that can be used to help enable individuals to make smart decisions about their IT infrastructure. A common approach is to deploy a graphing system such as
Graphite, which stores performance data about your infrastructure to aide you in performing a number of functions including problem diagnosis, performance trending, capacity planning, and data analytics. If you are unfamiliar with the software, I’ll briefly describe its architecture. Graphite consists of a daemon, called
carbon, which listens for
time series data and writes it to a fixed-size database called
whisper. It also provides a web application to expose the data and allow the user to create and display graphs on demand using a powerful API. While Graphite does a good job of storing time series data and providing a rich API for visualizing it, one of the things it does not really focus on is providing a dashboard for the data. Thankfully we have Grafana to fill this role and it happens to do it quite well. If you have ever worked with the ELK stack (
Elasticsearch,
Logstash, and
Kibana) before, Grafana’s interface should be familiar to you, as it is based on Kibana. It is a frontend for Graphite or
InfluxDB, and runs as a client side application in your browser. Its only (optional) external dependency is Elasticsearch, as it can use it to store, load and search for dashboards. Below are some of Grafana’s most notable features (see its
feature highlights for a more comprehensive list):
- Dashboard search
- Templated dashboards
- Save / load from Elasticsearch and / or JSON file
- Quickly add functions (search, typeahead)
- Direct link to Graphite function documentation
- Graph annotation
- Multiple Graphite or InfluxDB data sources
- Ability to switch between data sources
- Show graphs from different data sources on the same dashboard
class { 'graphite':
gr_web_server => 'none',
gr_web_cors_allow_from_all => true,
}
Since my module does not manage Apache (or Nginx), it is necessary to add something like the following to your node’s manifest to create a virtual host for Grafana:
# Grafana is to be served by Apache
class { 'apache':
default_vhost => false,
}
# Create Apache virtual host
apache::vhost { 'grafana.example.com':
servername => 'grafana.example.com',
port => 80,
docroot => '/opt/grafana',
error_log_file => 'grafana-error.log',
access_log_file => 'grafana-access.log',
directories => [
{
path => '/opt/grafana',
options => [ 'None' ],
allow => 'from All',
allow_override => [ 'None' ],
order => 'Allow,Deny',
}
]
}
And the Grafana declaration itself:
class { 'grafana':
elasticsearch_host => 'elasticsearch.example.com',
graphite_host => 'graphite.example.com',
}
Now that my module was working, it was time to publish it to the Puppet Forge. I converted my
Modulefile to
metadata.json, added a
.travis.yml file to my repository and enabled integration with
Travis CI, built the module and uploaded it to the Forge. Since its initial release, I have updated the module to deploy Grafana version 1.6.1 by default, including updating the content of the
config.js ERB template, and have added support for InfluxDB. I am pretty happy with the module and hope that you find it useful. I do have plans to add more capabilities to the module, including support of more of Grafana’s configuration file settings, having the module manage the web server’s configuration similar to how Daniel’s module does it, and adding a stronger test suite so I can ensure compatibility with more operating systems and Ruby / Puppet combinations. I welcome any questions, suggestions,
bug reports and / or pull requests you may have. Thanks for your time and interest! Project page:
https://github.com/bfraser/puppet-grafana Puppet Forge URL:
https://forge.puppetlabs.com/bfraser/grafana
