One of the great joys of Perl and CPAN is how it allows you to stand on the shoulders of giants. By picking the right tools, applications that are not that trivial can be built in a matter of days, if not hours. The goal of today’s little project is to demonstrate that very thing.
Grab a helmet and put your mouth-piece on, for this time I aim to do nothing other than blow your mind to awestruck smithereens.
The Specs
In a vague, semi-related follow-up to Dumuzi, I was wondering last week if I could have system checks that I could install on different machines and query via a web server. Those checks would come with two modes: passive checking, where we only collect information, and testing, where we check if everything’s peachy.
On top of that, why not have the results of the checks stored in a local history database?
And it would be great if I could also run the same checks, with a minimum of code change. Or, since we’re in full dream mode, maybe no code change at all.
Sounds like a fair application. In those three little paragraphs, we managed to squeeze needs for http, cli and database stacks, which need to be all put together in a seamless way. Okay. So, how many lines of code will be required to build that app (that, for giggles, I’ll call Varys). Well, let’s see…
A Check for Disk Usage
First thing, we need checks. As a sample, we’ll write one that reports on the disk partitions and, possibly, check that none is getting too full.
[perl]package Varys::Check::DiskUsage;use 5.10.0;
use strict;
use warnings;
use Sys::Statistics::Linux::DiskUsage;
use Method::Signatures;
use Moose;
use MooseX::ClassAttribute;
extends ‘Varys::Check’;
class_has ‘+store_model’ => ( default => ‘DiskUsage’, );
has skip => (
traits => [ ‘Input’, ‘Array’ ],
is => ‘ro’,
isa => ‘AutoArray’,
coerce => 1,
lazy => 1,
default => sub { [] },
handles => {
‘skip_all’ => ‘elements’,
},
);
has test_percent => (
traits => [ ‘Input’ ],
is => ‘ro’,
isa => ‘Int’,
default => 60,
);
has partitions => (
traits => [ ‘Info’, ‘Hash’ ],
is => ‘ro’,
isa => ‘HashRef’,
lazy => 1,
default => sub {
# TODO S::S::L::DU clobbers all ‘none’ together
my $p = Sys::Statistics::Linux::DiskUsage->new->get;
delete $p->{$_} for $_[0]->skip_all;
return $p;
},
handles => {
partitions_kv => ‘kv’,
},
);
method test {
my @full = map { $_->[0] }
grep { $_->[1]{usageper} > $self->test_percent }
$self->partitions_kv;
return {
success => @full ? 0 : 1,
( filled_partitions => \@full ) x [email protected]
};
}
1;
[/perl]Checks are going to be what we write over and over again, so we want to make it as easy to use as possible. All we are expecting from it are attributes that can either be parameters passed to the check (labeled via the Input trait) or collected data (labeled via the Info trait). Add to that a test() function that will return a hashref with a success result and whatever other information we want to provide (in this case, the list of bloated partitions).
In truth, the only piece of boilerplate that we need is the class_has '+store_model' stanza, which is required as we are going to use the DBIx::NoSQL::Store::Manager system I put together a few weeks ago. But more details on that later on.
The Checks Inner Mechanisms
Of course, checks don’t run on pure pixie magic. It’s close to it, but not quite. The role of the sparkling dust, in this case, is played by the parent class Varys::Check, which takes care of setting all the common stuff and hook points for the overall systems that will be using those checks:
use 5.10.0;
use strict;
use warnings;
use Method::Signatures;
use Data::Printer;
use DateTime::Functions;
use Moose;
use Moose::Util::TypeConstraints;
extends ‘MooseX::App::Cmd::Command’;
with ‘DBIx::NoSQL::Store::Model::Role’;
# tweaking to allow double-life as cli command
# and web action
has ‘+usage’ => ( required => 0, isa => ‘Any’ );
has ‘+app’ => ( required => 0, isa => ‘Any’ );
has ‘+store_key’ => (
default => method {
return join ‘ : ‘, $self->check_name, $self->timestamp;
},
);
has check_name => (
traits => [ ‘Varys::Trait::Input’ ],
is => ‘ro’,
isa => ‘Str’,
default => method { ref $self },
);
has timestamp => (
traits => [ ‘Varys::Trait::Input’, ‘StoreIndex’ ],
is => ‘ro’,
isa => ‘Str’,
default => sub { now()->iso8601; },
);
has run_test => (
isa => ‘Bool’,
is => ‘ro’,
default => 0,
);
has test_result => (
is => ‘rw’,
lazy => 1,
default => method {
return $self->run_test ? $self->test : undef;
},
);
subtype ‘AutoArray’ => as ‘ArrayRef[Str]’;
coerce ‘AutoArray’ => from ‘Str’ => via { [ $_ ] };
method info {
my %data;
# indexes are first class citizens
for( grep { $_->does( ‘DBIx::NoSQL::Store::Model::Role::StoreIndex’ ) }
$self->meta->get_all_attributes ) {
my $m = $_->name;
$data{$m} = $self->$m;
}
# split the rest between info and input
for my $type ( qw/ Input Info / ) {
$data{lc($type)} = {
map { my $m = $_->name; $m => $self->$m }
grep { $_->does( "Varys::Trait::$type" ) }
$self->meta->get_all_attributes
};
}
# and the results, if any
$data{test_result} = $self->test_result;
return \%data;
}
# TO_JSON is for Dancer,
# pack for DBIx::NoSQL::Store
*TO_JSON = *pack = *info;
# for MooseX::App::Cmd
method execute(@args) { say p $self->info; }
package Varys::Trait::Input;
use Moose::Role;
Moose::Util::meta_attribute_alias(‘Input’);
package Varys::Trait::Info;
use Moose::Role;
Moose::Util::meta_attribute_alias(‘Info’);
package Varys::Store;
use Moose;
extends ‘DBIx::NoSQL::Store::Manager’;
has ‘+model_path’ => (
default => ‘Varys::Check’,
);
1;
[/perl]As you can see, we’re using MooseX::App::Cmd for our cli invocation of the checks. In consequence, we have to tweak things so that the same class will not complain when used outside of that harness (lines 20-23), and we have to provide an execute method (line 92).
We’re also using that DBIx::NoSQL::Store::Manager add-on I wrote on top of DBIx::NoSQL, so we need to have a store key (lines 25-29).
The rest are things all checks will share: attributes for the name of the check, the timestamp of when it is run, if the test has to be executed and a last attribute to store the results of the said potential test, and the info method, which serializes the information of the check in a format we’ll be able to bandy around.
The Web Service
For the web service, we are using dear lithe and nimble Dancer:
[perl]package Varys;use Dancer ‘:syntax’;
use Dancer::Plugin::Auth::Basic;
use Varys::Store;
use Module::Pluggable
search_path => [ ‘Varys::Check’ ],
require => 1;
my $store = Varys::Store->connect( ‘checks.sqlite’ );
$store->register;
for my $check ( __PACKAGE__->plugins ) {
( my $name = $check ) =~ s/.*:://;
get "/$name" => sub {
return $store->new_model_object( $name,
params
);
};
post "/$name" => sub {
my $o = $store->new_model_object( $name,
params,
run_test => 1,
);
Dancer::SharedData->response->status(500) unless $o->test_result->{success};
return $o;
};
}
hook ‘before_serializer’ => sub {
$_[0]->content->store;
};
1;
[/perl]The brevity of the code speaks for itself. For each check, we are creating a GET action (for simple information retrieval) and a POST action (for running the test). A pre-serializing hook ensures that all results are kept in our store, and the serializing itself is taken care of by Dancer and our checks’ info() method. Oh yes, and we’ve thrown in some basic auth, because letting anybody run stuff on your machines? Not smart.
The result:
[bash]$ curl https://enkidu:3000/DiskUsageAuthorization required
$ curl -u yanick:hush https://enkidu:3000/DiskUsage
{
"info" : {
"partitions" : {
"/dev/sda6" : {
"usage" : "1142828",
"free" : "3417840",
"usageper" : "26",
"mountpoint" : "/var",
"total" : "4804736"
},
"none" : {
"usage" : "0",
"free" : "3967264",
"usageper" : "0",
"mountpoint" : "/var/lock",
"total" : "3967264"
},
…
}
},
"input" : {
"check_name" : "Varys::Check::DiskUsage",
"skip" : [],
"timestamp" : "2012-07-22T15:19:12",
"test_percent" : 60
},
"timestamp" : "2012-07-22T15:19:12",
"test_result" : null
}
$ POST -C yanick:hush https://enkidu:3000/DiskUsage
{
"info" : {
"partitions" : {
"/dev/sda6" : {
"usage" : "1142828",
"free" : "3417840",
"usageper" : "26",
"mountpoint" : "/var",
"total" : "4804736"
},
"none" : {
"usage" : "0",
"free" : "3967264",
"usageper" : "0",
"mountpoint" : "/var/lock",
"total" : "3967264"
},
…
},
"input" : {
"check_name" : "Varys::Check::DiskUsage",
"skip" : [],
"timestamp" : "2012-07-22T15:21:08",
"test_percent" : 60
},
"timestamp" : "2012-07-22T15:21:08",
"test_result" : {
"success" : 0,
"filled_partitions" : [
"/dev/sda8",
"/dev/sda11"
]
}
}
$ sqlite3 checks.sqlite ‘select * from DiskUsage’
Varys::Check::DiskUsage : 2012-07-22T15:19:12|2012-07-22T15:19:12
Varys::Check::DiskUsage : 2012-07-22T15:21:08|2012-07-22T15:21:08
[/bash]
The CLI Application
With what we already have, adding the cli application is only a question of throwing a script called varys.pl:
package Varys::CLI;
use strict;
use warnings;
use Moose;
extends ‘MooseX::App::Cmd’;
sub plugin_search_path { ‘Varys::Check’ }
Varys::CLI->run;
[/perl]
Yup, just that. And we can now do:
[bash]$ varys.plAvailable commands:
commands: list the application’s commands
help: display a command’s help screen
diskusage: (unknown)
$ varys.pl diskusage –run_test –test_percent 80
\ {
info {
partitions {
/dev/sda1 {
free 414659,
mountpoint "/boot",
total 472036,
usage 33006,
usageper 8
},
/dev/sda5 {
free 12549112,
mountpoint "/",
total 19223252,
usage 5697656,
usageper 32
},
…
},
input {
check_name "Varys::Check::DiskUsage",
skip [],
test_percent 80,
timestamp "2012-07-22T16:24:28"
},
test_result {
filled_partitions [
[0] "/dev/sda8"
],
success 0
},
timestamp "2012-07-22T16:24:28"
}
Ta-dah!
Aaaand that’s it for now. The code, as usual, is available on GitHub. The DBIx::NoSQL::Store::Manager-related classes are still hidden in my Galuga project, but should be CPANized in a not-so-distant future.
No comments