Mass-Testing Dancer’s Plugins

Posted in: Technical Track

One of the highlights of YAPC::EU 2012 was to meet a few of the core Dancer dudes. (And, yes, a blog entry about the conference is forthcoming,.It only has been delayed by two weeks of utterly decadent vacations in the heart of the Rhine Valley.) Amongst other things, dams filled me in on the progress of Dancer 2 and the tasks that still lay ahead before this rewrite of our favorite micro-web framework can hit the streets. One of the items on the to-do list is to verify that most of the plugins already written for Dancer 1 will still work for Dancer 2. Well, I thought, that’s just like doing smoke testing for a small subset of Perl modules. How hard can it be?

Yeah, I do tend to say silly things like that…

Corraling the Modules

First task: Gather the modules to test. Finding all of Dancer’s plugins is relatively simple. Mostly if your CPAN client is configured to use the SQLite back-end:

[bash]$ sqlite3 ~/.cpan/cpandb.sql ‘select dist_name from dists \
where dist_name like "Dancer-Plugin%"’ \
| perl -pe’s/-/::/g’ > plugins
[/bash]

If the output of that command is to be trusted, there are 76 Dancer plugins out there. Not a bad number. To keep them around, I decided to have a go at Pinto, which turned out to be a joy.

[bash]perl -ne’chomp; next if /^\s*#/; `pinto -v pull $_`’ plugins
[/bash]

Mind you, the command could have been as simple as

[bash]pinto -v pull < plugins
[/bash]

but for two things: I wanted to be able to comment out problematic plugins in the list and skip over them, and it seems that Pinto would abort commiting all of the pulled distributions if one of them goes wrong. And while pulling 76 distros and all their dependencies did generate a lot of wrongness, calling Pinto on each plugin independently was the fastest way out.

Testing the Herd

With all the modules nicely accessing, I thought I had it done.

AH!

It turns out that while Perl has an awesome ecosystem for testing, it’s also quite befuddling at first, and most of the modules are almost but not quite what I was wanting. They are mostly aimed at smoking ALL THE THINGS on potentially many Perls and pushing the results to a smoking mothership rather than feeding a local database. A specific module, SmokeRunner::Multi, seemed more promising for the task and could even talk to a Smolder instance. Well, to make a long story short, at the end I didn’t use it (although I still keep it on the wing), but I ended up becoming its maintainer. Just don’t ask me how that happened.

Instead, I ended up leveraging the --test-only option of cpanm. I have yet to find out how to use it in such a way that I would get the full TAP results for a distro (although I’m sure there’s a way), but it reports the overall success or failure, which is already a start.

So, I have a way to test the modules. Now, I need to store the results. How about going easy and sleazy and using that DBIx::NoSQL::Store::Manager I hacked a few months back? (No, it’s not on CPAN yet. Yes, I’m going to do something about that Real Soontm)

[perl]package DancerTest;

use strict;
use warnings;

use Moose;

extends ‘DBIx::NoSQL::Store::Manager’;

__PACKAGE__->meta->make_immutable(inline_constructor => 0);

1;
[/perl]

 

[perl]package DancerTest::Types;

use MooseX::Storage::Engine;

use MooseX::Types -declare => [qw/
DateTimeClass
/];

use Moose::Util::TypeConstraints;
use DateTime;
use DateTime::Format::ISO8601;

class_type ‘DateTimeClass’ => { class => ‘DateTime’ };

MooseX::Storage::Engine->add_custom_type_handler(
‘DateTimeClass’ => (
expand => sub {
DateTime::Format::ISO8601->parse_datetime(shift)
},
collapse => sub { (shift)->iso8601 },
),
);

1;

[/perl]

 

[perl]package DancerTest::Model::Plugin;

use 5.10.0;

use strict;
use warnings;

use Method::Signatures;
use IPC::Cmd ‘run’;

use DancerTest;
use DancerTest::Types qw/ DateTimeClass /;

use Moose;

with ‘DBIx::NoSQL::Store::Model::Role’;

has ‘+store_key’ => (
default => method {
join ‘ : ‘, $self->name, $self->timestamp;
},
);

has name => (
traits => [ ‘StoreIndex’ ],
isa => ‘Str’,
is => ‘ro’,
required => 1,
);

has timestamp => (
traits => [ ‘StoreIndex’ ],
isa => ‘DateTimeClass’,
is => ‘ro’,
lazy => 1,
default => sub { DateTime->now },
);

has version => (
traits => [ ‘StoreIndex’ ],
isa => ‘Str’,
is => ‘ro’,
lazy => 1,
default => method {
my $version;
run( command => "pinto install –info " . $self->name, buffer => \$version );

chomp $version;
$version =~ s/^.*-//;
$version =~ s/\.tar.gz$//;

return $version;
},
);

has dancer1_pass => (
traits => [ ‘StoreIndex’ ],
isa => ‘Bool’,
is => ‘ro’,
lazy => 1,
default => method {
$self->run_tests;
},
);

has dancer2_pass => (
traits => [ ‘StoreIndex’ ],
isa => ‘Bool’,
is => ‘ro’,
lazy => 1,
default => method {
local $ENV{PERL5LIB} = ‘/home/yanick/work/perl-modules/Dancer2/lib’;
$self->run_tests;
},
);

method run_tests {
return scalar( run( command => "pinto install –test-only " . $self->name
) ) ? 1 : 0;
}

has verbose => (
is => ‘ro’,
isa => ‘Bool’,
default => 1,
);

before ‘store’ => method {
say "testing ", $self->name;
};

after ‘store’ => method {
use Data::Printer;
p $self->pack;
};

__PACKAGE__->meta->make_immutable;

1;

[/perl]

And with that in hand, we just need one more script to turn in the crank:

[perl]#!/usr/bin/perl

use 5.14.0;

use DancerTest;
use DancerTest::Model::Plugin;

my $store = DancerTest->connect( ‘plugins.db’ );

$store->register;

while( my $module = <> ) {
chomp $module;
next if $module=~ /^\s*#/;
$store->new_model_object(‘Plugin’,
name => $module,
verbose => 1,
)->store;
}

[/perl]

Aaaand:

[bash]$ ./test_plugins.pl plugins
testing Dancer::Plugin::ExtDirect
\ {
__CLASS__ "DancerTest::Model::Plugin",
dancer1_pass 1,
dancer2_pass 1,
name "Dancer::Plugin::ExtDirect",
timestamp "2012-09-18T01:41:52",
verbose 1,
version 1.03
}
testing Dancer::Plugin::TTHelpers
\ {
__CLASS__ "DancerTest::Model::Plugin",
dancer1_pass 1,
dancer2_pass 1,
name "Dancer::Plugin::TTHelpers",
timestamp "2012-09-18T01:42:01",
verbose 1,
version 0.004
}
[/bash]

Final stroke, reporting the latest results:

[perl]package DancerReport;

use 5.10.0;

use base qw/DBIx::Class::Schema::Loader/;

package main;

my $schema = DancerReport->connect(‘dbi:SQLite:plugins.db’);

my $plugins = $schema->resultset(‘Plugin’);
my $rs = $plugins->search({
‘me.timestamp’ => {
‘=’ => $plugins->search(
{ ‘x.name’ => { ‘=’ => { -ident => ‘me.name’ } } },
{ alias => ‘x’ }
)->get_column(‘timestamp’)->max_rs->as_query,
},
});

say sprintf "%40s %10s %10s %10s", qw/ distro version dancer_1 dancer_2 /;

while( my $p = $rs->next ) {
next if $p->version !~ /^\d/;
say sprintf "%40s %10s %10s %10s",
map( { $p->$_ } qw/ name version / ),
map { $_ ? ‘passed’ : ‘failed’ } map { $p->$_ }
qw/ dancer1_pass dancer2_pass /;
}

[/perl]

Which gives us:

[bash]$ ./report.pl
distro version dancer_1 dancer_2
Dancer::Plugin::SQLSearch 0.04 passed failed
Dancer::Plugin::I18N 0.40 passed failed
Dancer::Plugin::Lucy 0.001 passed passed
Dancer::Plugin::EmptyGIF 0.3 passed failed
Dancer::Plugin::Locale 0.01 passed passed
Dancer::Plugin::Locale::Wolowitz 0.122470 passed failed
Dancer::Plugin::RequireSSL 0.121370 passed passed
Dancer::Plugin::Res 0.0003 passed passed
Dancer::Plugin::Lexicon 0.05 passed passed
Dancer::Plugin::Cerberus 0.03 passed passed
Dancer::Plugin::Resource 1.122280 passed failed
Dancer::Plugin::ElasticModel 0.05 failed failed
Dancer::Plugin::Database 1.82 passed failed
Dancer::Plugin::Email 0.1300 passed passed
Dancer::Plugin::Scoped 0.02fix passed passed
Dancer::Plugin::SiteMap 0.11 passed passed
Dancer::Plugin::DBIC 0.1506 passed failed
Dancer::Plugin::Authorize 1.110720 passed passed
Dancer::Plugin::FormValidator 1.122460 passed failed
Dancer::Plugin::Params::Normalization 0.51 passed failed
Dancer::Plugin::REST 0.07 passed failed
Dancer::Plugin::Mongo 0.03 passed failed
Dancer::Plugin::ORMesque 1.113100 passed failed
Dancer::Plugin::Browser 0.4 passed passed
Dancer::Plugin::DebugDump 0.03 passed passed
Dancer::Plugin::Memcached 0.01 passed failed
Dancer::Plugin::MPD 0.03 failed failed
Dancer::Plugin::Feed 0.8 passed failed
Dancer::Plugin::MobileDevice 0.03 passed failed
Dancer::Plugin::Auth::RBAC 1.110720 passed failed
Dancer::Plugin::Auth::Twitter 0.02 passed failed
Dancer::Plugin::SimpleCRUD 0.61 passed failed
Dancer::Plugin::SporeDefinitionControl 0.11 passed failed
Dancer::Plugin::WebSocket 0.0100 passed failed
Dancer::Plugin::Redis 0.2 passed failed
Dancer::Plugin::FormattedOutput 0.01 passed passed
Dancer::Plugin::FlashMessage 0.314 passed failed
Dancer::Plugin::MemcachedFast 0.110770 passed failed
Dancer::Plugin::Facebook 0.900 failed failed
Dancer::Plugin::ValidationClass 0.120490 passed failed
Dancer::Plugin::ProxyPath 0.03 passed failed
Dancer::Plugin::GearmanXS 0.110570 failed failed
Dancer::Plugin::FlashNote 1.0.3 passed failed
Dancer::Plugin::SMS 0.01 passed passed
Dancer::Plugin::Fake::Response 0.03 passed failed
Dancer::Plugin::Cache::CHI 1.3.0 passed failed
Dancer::Plugin::Progress 0.001 passed passed
Dancer::Plugin::TimeRequests 0.03 passed passed
Dancer::Plugin::Async 0.001 passed passed
Dancer::Plugin::XML::RSS 0.01 passed passed
Dancer::Plugin::Auth::RBAC::Credentials::DBIC 0.003 passed failed
Dancer::Plugin::Passphrase 1.0.0 passed failed
Dancer::Plugin::ValidateTiny 0.05 passed passed
Dancer::Plugin::Hosts 0.01 passed passed
Dancer::Plugin::Bcrypt 0.4.1 passed failed
Dancer::Plugin::Stomp 1.0101 passed passed
Dancer::Plugin::LibraryThing 0.0003 passed passed
Dancer::Plugin::DebugToolbar 0.016 passed failed
Dancer::Plugin::EscapeHTML 0.22 passed failed
Dancer::Plugin::EncodeID 0.02 passed failed
Dancer::Plugin::DirectoryView 0.02 passed failed
Dancer::Plugin::NYTProf 0.21 passed passed
Dancer::Plugin::Auth::Basic 0.02 failed failed
Dancer::Plugin::Mongoose 0.00002 passed failed
Dancer::Plugin::Dispatcher 0.12 passed failed
Dancer::Plugin::Preprocess::Sass 0.01 passed passed
Dancer::Plugin::Auth::Htpasswd 0.014 passed passed
Dancer::Plugin::ElasticSearch 0.002 passed passed
Dancer::Plugin::ExtDirect 1.03 passed passed
Dancer::Plugin::TTHelpers 0.004 passed passed
[/bash]

According to the report, 37 plugins pass their test under Dancer 1, but not Dancer 2. Now we know exactly what remains to do. So… Let’s get crackin’!

email

Interested in working with Yanick? Schedule a tech call.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *