Harnessing the power of Jinja2 with Ansible

Posted in: Articles, DevOps, Open Source, Site Reliability Engineering, Technical Track

Configuration files management is the core component of every configuration management tool. Templates are one of the best methods for creating configuration files as they allow dynamic variables. Ansible has a template module that uses Jinja2 in its core. Jinja2 is written in Python and allows Python-like expressions which allows us to have Python features and flexibility to be embedded into our configuration file management strategy.

Looping and Testing

The following is a standard template usage to create multiple files using the same template. Using the loop method with_items, multiple files can be created based on the declared variable log_path.

  vars:
    set_category: true
    log_path:
      - name: "auth"
        path: "/var/log/auth.log"
        category: auth
      - name: "syslog"
        path: "/var/log/syslog"
        category: syslog
  tasks:
    - name: create config files
      template: src=./templates/sources.j2 dest=/etc/sumo/sumo.d/{{item.name}}.json
      with_items: "{{ log_path }}"

With this, variables like log_path.path can be called as item.path from the template.

Tests using “if” expressions can be used as seen in the following example. If only set_category variable is set to TRUE, the category will be updated.

{
  "api.version": "v1",
  "source": {
      "name": "{{ item.name }}",
      "sourceType": "LocalFile",
      "pathExpression": "{{ item.path  }}",
{% if set_category %}
      "category": "{{ item.category  }}"
{% endif %}
     }
} 

The above code would create the following files:

/etc/sumo/sumo.d/auth.json

{
  "api.version": "v1",
  "source": {
      "name": "auth",
      "sourceType": "LocalFile",
      "pathExpression": "/var/log/auth.log",
      "category": "auth"
     }
} 

/etc/sumo/sumo.d/syslog.json

{
  "api.version": "v1",
  "source": {
      "name": "syslog",
      "sourceType": "LocalFile",
      "pathExpression": "/var/log/syslog",
      "category": "syslog"
     }
} 

Repeated configuration becomes an easy task and if it is required in a file – “for loop” can be used from a template.

{% for item in log_path %}
{{ item.name }} :  {{ item.path }}
{% endfor %}

This would create an entry like below:

auth :  /var/log/auth.log
syslog :  /var/log/syslog

Filters

Jinja2 provides a wide range of filters that can transform the data inside the template.

For example, join filter would take a list and concatenate it to a string with a separator. The following is an example:

{{ allhosts | join(',')}}

The above join filter would take the following list:

    allhosts:
      - localhost
      - 10.0.0.1
      - 10.0.0.2

and create the following:

localhost,10.0.0.1,10.0.0.2

json_query is a filter which would allow searching inside a data structure. The below query would search inside the log_paths variable for a name key with value auth and would print the value of associated path:

{{ log_paths | json_query('[?name==`auth`].path') }}

/var/log/auth.log

Another advantage of filters is that multiple filters can be lined up using pipe. For example, adding a basename filter to the above query would get the last name of the above file path.

{{ log_paths | json_query('[?name==`boom`].path') | basename}}

auth.log

map filter is another filter, which is very useful. It can either look up an attribute from a list or apply another filter on the objects.

For example, the following would look for all values with attribute category:

{{ log_paths | map(attribute='category') | join(', ')  }}

auth, syslog

In the following example, another filter called regex_replace is called by map filter to attach http:// to all the ip addresses which starts with 10:

{{ allhosts | map('regex_replace', '^(10.*)$', 'http://\\1') | join(', ')  }}

localhost, http://10.0.0.1, http://10.0.0.2

As we can see, Ansible Jinja2 templates use the power of Python, allowing complex data manipulation while managing diverse environments. Custom filters can also be created using Python and added to filter_plugins/ under the ansible top directory.

email

Interested in working with Minto? Schedule a tech call.

About the Author

Devops Engineer
Minto Joseph is an expert in opensource technologies with a deep understanding of Linux. This allows him to troubleshoot issues from kernel to the application layer. He also has extensive experience in debugging Linux performance issues. Minto uses his skills to architect, implement and debug enterprise environments.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *