Lazy test development
Making the Perl debugger do the work
Joe McMahon
Bugs happen
- We all write programs, or have programs handed to us
- These programs don't always work for one reason or another
- We end up having to debug them
Test-driven development
- Integrates writing and testing the code
- Decreases the amount of debugging time
- But sometimes you still have to debug
The cycle
- Debug for a while
- Figure out the problem
- Quit the debugger
- Write the test(s)
- ... do I still remember how to fix it?
Lazy Testing
- If only ...
- ... we could write the tests while we were in the debugger
- ... and have it build a test file for us
- Then we'd only have to remember how to do the fix
- So let's make the debugger do that
- And in the process we'll visit some interesting little corners of both Perl and the debugger
Here be dragons
- From
perldoc perldebguts
:
This manpage describes low-level details concerning the debugger's internals, which range from difficult to impossible to understand for anyone who isn't incredibly intimate with Perl's guts.
Here be dragons
- From
perldoc perldebguts
:
This manpage describes low-level details concerning the debugger's internals, which range from difficult to impossible to understand for anyone who isn't incredibly intimate with Perl's guts.
- Welcome to the debugger internals.
Here be dragons
- From
perldoc perldebguts
:
This manpage describes low-level details concerning the debugger's internals, which range from difficult to impossible to understand for anyone who isn't incredibly intimate with Perl's guts.
- Welcome to the debugger internals.
- Now go away.
It's not that bad, really
- From the debugger's comments:
# Unreadable code begins here. It needs fixing.
- Main command loop is ~1200 lines long.
- Okay, so maybe it is that bad
Documentation to the rescue
- Debugger now has real readable internal docs
perldoc perl5db.pl
- Or look at the HTML version
- So now it's at least possible to figure it out and modify the debugger
Seems like an awful lot of work though
Finding an easier way
- But we're lazy, so let's take some shortcuts
- The debugger has external interfaces
- Can we use those instead of actually messing with the code?
.perldb
.perldb
is the debugger's init file
- Runs before debugger initialization
$alias{len} = 's/^len(.*)/p length($1)/';
$alias{stop} = 's/^stop (at|in)/b/';
$alias{ps} = 's/^ps\b/p scalar /';
$alias{quit} = 's/^quit(\s*)/exit/';
$alias{help} = 's/^help\s*$/|h/';
Set options
parse_options("NonStop=1 LineInfo=db.out AutoTrace=1 frame=2");
Define an afterinit
function
afterinit()
- Called in the debugger's namespace (DB)
- Lets you do whatever you want
- ... including fool around with the debugger itself
So what do we want to do?
- We want to load up
Test::More
to support our testing
- We want to define new commands
- We want it to be really, really easy
What about testing this?
- Good question
- The debugger itself is inherently difficult to test
- So our best bet is to
- write code that we can test outside the debugger
- embed this in as generic a "command" as possible
Test::More
- Provides a set of functions for testing Perl programs
- Conforms to the Perl testing standard
- Print the number of tests (the 'plan') expected
1..20
Each test prints ok
or not ok
and its test number
ok 1
ok 2 - test worked
not ok 3
...
prove
- The
prove
utility (supplied with perl 5.8.X and up) lets you run these tests anywhere, anytime
prove *.t
- Provides a test harness without having to put together a Makefile, etc.
- It interprets the output from all the tests and summarizes it for us
flatbox:~ joe$ prove t/*.t
t/001_load..............ok
t/002_is_a_test.........ok
t/003_is_a_sub..........ok
t/004_get_test_names....ok
t/005_tdump.............ok
All tests successful.
Files=5, Tests=71, 1 wallclock secs ( 0.36 cusr + 0.13 csys = 0.49 CPU)
flatbox:~ joe$
Getting loaded
- Step one: load
Test::More
into the debugger's namespace
- Really ugly to do from the command line
perl -MTest::More=no_plan -demo
- Let's have the debugger load it for us
Getting lazier
- We could type it at the debugger prompt
DB<1> use Test::More qw(no_plan)
- Still too much work
- Let's make the debugger type it
Making the debugger type things
@DB::typeahead
is the debugger's command buffer
- Anything in this array gets read first and executed before the debugger prompts again
- So we'll just stack the
use
in the typeahead buffer
sub afterinit {
@DB::typeahead = ("use Test::More qw(no_plan)");
}
We get this when we run it:
flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2>
Integrated testing
- Now we can use all of the
Test::More
tests
- But now all those tests are in the scrollback and we'd have to cut and paste them
- Too much work - let's make the debugger do it
The dawn of recorded history
- The debugger records everything you do if
Term::ReadLine
is available...
- ... and can save it to a file with its
save
command
- Command history is in
@DB::truehist
save
just dumps this out
- Not directly usable
hdump
first cut
- Let's mock up a save command of our own to start
-
sub hdump($) {
print OUT "Recording history for this session ...";
unless (open HFH, ">$ENV{HOME}/.perldb.history") {
print OUT " can't write history: $!\n";
}
else {
print HFH join("\n",@DB::truehist),"\n";
close HFH;
my $count = int @DB::truehist;
print OUT " done ($count lines).\n";
}
}
Getting it in
- As long as it's defined in the
.perldb
file, the debugger can see it
- So even though
afterinit
doesn't call it directly, we can
DB<2> DB::hdump "debugger.history"
Recording history for this session in debugger.history ... done (3 lines).
DB<3>
hdump
demo
flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2> is(1,1,"constant")
ok 1 - constant
DB<3> isnt(2,1,"constant too")
ok 2 - constant too
DB<4> like("I like coconut cream pie", qr/I like .* pie/,"likes pie");
ok 3 - likes pie
DB<5> DB::hdump sample
Recording history for this session in sample ... done (6 lines).
DB<6> q
1..3
flatbox:~ joe$
hdump
output
?
use Test::More qw(no_plan)
is(1,1,"constant")
isnt(2,1,"constant too")
like("I like coconut cream pie", qr/I like .* pie/,"likes pie");
DB::hdump sample
Where'd the question mark come from?
- It's a bug
- The original history manipulation command was
?
- And nobody ever changed it
- Apparently nobody fiddles with history much
This is a recording
- Nearly right, but ...
- ... the question mark is still there
- ... the debugger already loads
Test::More
Playback
- Now we can use another little-known debugger command -
source
- Reads a file and executes its contents as debugger commands
- Now we can save and reuse our work
- Better than the
R
(restart) command
Kick it up a notch
- Dumping the tests ... as a standalone test
- We can munge the data in
@DB::truehist
as well as just dump it
- We can filter out the tests and write a new test file
Inspecting Test::More
- part 1
# Returns true if this is a sub in Test::More, false otherwise
sub is_a_sub {
local $_ = shift;
(!/^_/) and eval "defined &Test::More::$_";
}
Inspecting Test::More
- part 3
# Build a hash to check names against.
%test_names = map { $_ => 1 } get_test_names()
unless keys %test_names;
Are you a test?
# Returns true if this line of history is a Test::More test.
sub is_a_test {
my $line = shift;
if (($possible, $paren) = /^\s*(\w+)\(/) {
return $test_names{$possible};
}
}
The test dumper - tdump
sub tdump($) {
my $outfile = shift;
%test_names = map { $_ => 1 } get_test_names()
unless keys %test_names;
print OUT "Recording tests for this session in $outfile ...";
unless (open TFH, ">$outfile") {
print OUT " can't write history: $!\n";
}
else {
my @tests = map { "$_;\n" }
grep { is_a_test($_, \%test_names) } @DB::truehist;
print TFH "use Test::More tests=>", int @tests, ";\n";
print TFH join "\n", @tests, "\n";
close TFH;
print OUT " done (", int @tests, " tests).\n";
}
}
tdump
in action
Flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2> is(1,1,"constants are constant")
ok 1 - constants are constant
DB<3> isnt(2,1, "still constant")
ok 2 - still constant
DB<4> use_ok("CGI")
ok 3 - use CGI;
DB<5> tdump "sample.t"
Can't locate object method "tdump" via package "sample"
(perhaps you forgot to load "sample"?) at
(eval 13)[/System/Library/Perl/5.8.5/perl5db.pl:620] line 2.
DB<6> DB::tdump sample.t
Recording tests for this session in samplet ... done (3 tests).
DB<7> q
1..3
The resulting test
Flatbox:~ joe$ cat samplet
use Test::More tests=>3;
is(1,1,"constants are constant");
isnt(2,1, "still constant");
use_ok("CGI");
Flatbox: ~ joe $ prove -v samplet
sample....1..3
ok 1 - constants are constant
ok 2 - still constant
ok 3 - use CGI;
ok
All tests successful.
Files=1, Tests=3, 1 wallclock secs ( 0.16 cusr + 0.05 csys = 0.21 CPU)
Flatbox:~ joe$
Not bad, but let's do better
- What if we want to to have some setup in there too?
- Exclude all of the debugger commands
- But writing something to parse debugger commands would be hard
- So we'll do something easy instead
Comment hack
- If a comment follows a command, keep both with the tests
DB<12> $x=5
DB<13> $obj = new CGI;
DB<14> # Keep 13, lose 12
DB<15>
Leverage the hack
- Let's make it work for us
- We'll grab all the comments and put them out before the line they follow
- That way we can get comments for our setup
DB<8> $x = Foo->new();
DB<9> # Create a Foo for class testing
DB<10> isa_ok($x, "Foo")
ok 5 - The object isa Foo
The test file will get
# Create a Foo for testing
$x = Foo->new();
isa_ok($x, "Foo");
tdump
version 2
sub tdump($) {
...
my @tests = map { "$_;\n" }
grep { is_a_test($_, \%test_names) } @DB::truehist;
print TFH "use Test::More tests=>", int @tests, ";\n";
my @lines = @DB::truehist;
while (@lines) {
my $line = shift @lines;
my $forced_capture = 0;
while ($lines[0] =~ /^\s*#/) {
print TFH $lines[0],"\n";
$forced_capture = 1;
shift @lines;
}
next unless $forced_capture or is_a_test($line, \%test_names);
$line = "$line;\n" unless $line =~ /;$/;;
print TFH $line;
}
...
Sample run
flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2> is(1,1)
ok 1
DB<3> $x=2
DB<4> # This sets up the next test
DB<5> isnt($x,1)
ok 2
DB<6> DB::tdump "comments.t"
Recording tests for this session in comments.t ... done (2 tests).
DB<7> q
1..2
flatbox:~ joe$
Sample output
flatbox:~ joe$ cat comments.t
use Test::More tests=>2;
is(1,1);
# This sets up the next test
$x=2;
isnt($x,1);
flatbox:~ joe$
Even lazier version
flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2> is(1,1)
ok 1
DB<3> tdump "demo.t"
Recording tests for this session in demo.t ... done (1 tests).
DB<4> q
1..1
flatbox:~ joe$ cat demo.t
use Test::More tests=>1;
is(1,1);
flatbox:~ joe$
The whole ball of wax
flatbox:~ joe$ perl -demo
Loading DB routines from perl5db.pl version 1.27
Editor support available.
Enter h or `h h' for help, or `man perldebug' for more help.
main::(-e:1): mo
auto(-1) DB<1> use Test::More qw(no_plan)
DB<2> is(1,1)
ok 1
DB<3> use_ok("CGI")
ok 2 - use CGI;
DB<4> $foo = new CGI;
DB<5> # Create a CGI object and see if type is right
DB<6> isa_ok($foo, "CGI")
ok 3 - The object isa CGI
DB<7> tdump
Recording tests for this session in test.t ... done (3 tests).
DB<8> q
1..3
flatbox:~ joe$
Test with setup and comments
flatbox:~ joe$ cat demo.t
use Test::More tests=>3;
is(1,1);
use_ok("CGI");
# Create a CGI object and see if type is right
$foo = new CGI;
isa_ok($foo, "CGI");
flatbox:~ joe$