Author: Kevin Pedersen

Creating dynamic tasks using Apache Airflow

What’s Airflow? Apache Airflow is an open source scheduler built on Python. It uses a topological sorting mechanism, called a DAG (Directed Acyclic Graph) to generate dynamic tasks for execution according to dependency, schedule, dependency task completion, data partition and/or…

Read More >

7 tips for managing Infrastructure with Terraform

Kevin Pedersen, Team Manager, Service Reliability Engineering at Pythian, provides some considerations to keep your Terraform environment manageable.

Read More >

A repeatable test environment with Terraform and chef-solo

Kevin Pedersen, Team Manager, Service Reliability Engineering at Pythian, discusses creating a repeatable test environment with Terraform and Chef-solo.

Read More >

Essential Hadoop concepts for systems administrators

Of course, everyone knows Hadoop as the solution to Big Data. What’s the problem with Big Data? Well, mostly it’s just that Big Data is too big to access and process in a timely fashion on a conventional enterprise system….

Read More >