diff options
Diffstat (limited to 'Master/tlpkg/tlperl/lib/Test/Tutorial.pod')
| -rw-r--r-- | Master/tlpkg/tlperl/lib/Test/Tutorial.pod | 610 |
1 files changed, 610 insertions, 0 deletions
diff --git a/Master/tlpkg/tlperl/lib/Test/Tutorial.pod b/Master/tlpkg/tlperl/lib/Test/Tutorial.pod new file mode 100644 index 00000000000..b89fd07ca5d --- /dev/null +++ b/Master/tlpkg/tlperl/lib/Test/Tutorial.pod @@ -0,0 +1,610 @@ +=head1 NAME + +Test::Tutorial - A tutorial about writing really basic tests + +=head1 DESCRIPTION + + +I<AHHHHHHH!!!! NOT TESTING! Anything but testing! +Beat me, whip me, send me to Detroit, but don't make +me write tests!> + +I<*sob*> + +I<Besides, I don't know how to write the damned things.> + + +Is this you? Is writing tests right up there with writing +documentation and having your fingernails pulled out? Did you open up +a test and read + + ######## We start with some black magic + +and decide that's quite enough for you? + +It's ok. That's all gone now. We've done all the black magic for +you. And here are the tricks... + + +=head2 Nuts and bolts of testing. + +Here's the most basic test program. + + #!/usr/bin/perl -w + + print "1..1\n"; + + print 1 + 1 == 2 ? "ok 1\n" : "not ok 1\n"; + +Because 1 + 1 is 2, it prints: + + 1..1 + ok 1 + +What this says is: C<1..1> "I'm going to run one test." [1] C<ok 1> +"The first test passed". And that's about all magic there is to +testing. Your basic unit of testing is the I<ok>. For each thing you +test, an C<ok> is printed. Simple. L<Test::Harness> interprets your test +results to determine if you succeeded or failed (more on that later). + +Writing all these print statements rapidly gets tedious. Fortunately, +there's L<Test::Simple>. It has one function, C<ok()>. + + #!/usr/bin/perl -w + + use Test::Simple tests => 1; + + ok( 1 + 1 == 2 ); + +That does the same thing as the previous code. C<ok()> is the backbone +of Perl testing, and we'll be using it instead of roll-your-own from +here on. If C<ok()> gets a true value, the test passes. False, it +fails. + + #!/usr/bin/perl -w + + use Test::Simple tests => 2; + ok( 1 + 1 == 2 ); + ok( 2 + 2 == 5 ); + +From that comes: + + 1..2 + ok 1 + not ok 2 + # Failed test (test.pl at line 5) + # Looks like you failed 1 tests of 2. + +C<1..2> "I'm going to run two tests." This number is a I<plan>. It helps to +ensure your test program ran all the way through and didn't die or skip some +tests. C<ok 1> "The first test passed." C<not ok 2> "The second test failed". +Test::Simple helpfully prints out some extra commentary about your tests. + +It's not scary. Come, hold my hand. We're going to give an example +of testing a module. For our example, we'll be testing a date +library, L<Date::ICal>. It's on CPAN, so download a copy and follow +along. [2] + + +=head2 Where to start? + +This is the hardest part of testing, where do you start? People often get +overwhelmed at the apparent enormity of the task of testing a whole module. +The best place to start is at the beginning. C<Date::ICal> is an +object-oriented module, and that means you start by making an object. Test +C<new()>. + + #!/usr/bin/perl -w + + # assume these two lines are in all subsequent examples + use strict; + use warnings; + + use Test::Simple tests => 2; + + use Date::ICal; + + my $ical = Date::ICal->new; # create an object + ok( defined $ical ); # check that we got something + ok( $ical->isa('Date::ICal') ); # and it's the right class + +Run that and you should get: + + 1..2 + ok 1 + ok 2 + +Congratulations! You've written your first useful test. + + +=head2 Names + +That output isn't terribly descriptive, is it? When you have two tests you can +figure out which one is #2, but what if you have 102 tests? + +Each test can be given a little descriptive name as the second +argument to C<ok()>. + + use Test::Simple tests => 2; + + ok( defined $ical, 'new() returned something' ); + ok( $ical->isa('Date::ICal'), " and it's the right class" ); + +Now you'll see: + + 1..2 + ok 1 - new() returned something + ok 2 - and it's the right class + + +=head2 Test the manual + +The simplest way to build up a decent testing suite is to just test what +the manual says it does. [3] Let's pull something out of the +L<Date::ICal/SYNOPSIS> and test that all its bits work. + + #!/usr/bin/perl -w + + use Test::Simple tests => 8; + + use Date::ICal; + + $ical = Date::ICal->new( year => 1964, month => 10, day => 16, + hour => 16, min => 12, sec => 47, + tz => '0530' ); + + ok( defined $ical, 'new() returned something' ); + ok( $ical->isa('Date::ICal'), " and it's the right class" ); + ok( $ical->sec == 47, ' sec()' ); + ok( $ical->min == 12, ' min()' ); + ok( $ical->hour == 16, ' hour()' ); + ok( $ical->day == 17, ' day()' ); + ok( $ical->month == 10, ' month()' ); + ok( $ical->year == 1964, ' year()' ); + +Run that and you get: + + 1..8 + ok 1 - new() returned something + ok 2 - and it's the right class + ok 3 - sec() + ok 4 - min() + ok 5 - hour() + not ok 6 - day() + # Failed test (- at line 16) + ok 7 - month() + ok 8 - year() + # Looks like you failed 1 tests of 8. + +Whoops, a failure! [4] C<Test::Simple> helpfully lets us know on what line the +failure occurred, but not much else. We were supposed to get 17, but we +didn't. What did we get?? Dunno. You could re-run the test in the debugger +or throw in some print statements to find out. + +Instead, switch from L<Test::Simple> to L<Test::More>. C<Test::More> +does everything C<Test::Simple> does, and more! In fact, C<Test::More> does +things I<exactly> the way C<Test::Simple> does. You can literally swap +C<Test::Simple> out and put C<Test::More> in its place. That's just what +we're going to do. + +C<Test::More> does more than C<Test::Simple>. The most important difference at +this point is it provides more informative ways to say "ok". Although you can +write almost any test with a generic C<ok()>, it can't tell you what went +wrong. The C<is()> function lets us declare that something is supposed to be +the same as something else: + + use Test::More tests => 8; + + use Date::ICal; + + $ical = Date::ICal->new( year => 1964, month => 10, day => 16, + hour => 16, min => 12, sec => 47, + tz => '0530' ); + + ok( defined $ical, 'new() returned something' ); + ok( $ical->isa('Date::ICal'), " and it's the right class" ); + is( $ical->sec, 47, ' sec()' ); + is( $ical->min, 12, ' min()' ); + is( $ical->hour, 16, ' hour()' ); + is( $ical->day, 17, ' day()' ); + is( $ical->month, 10, ' month()' ); + is( $ical->year, 1964, ' year()' ); + +"Is C<$ical-E<gt>sec> 47?" "Is C<$ical-E<gt>min> 12?" With C<is()> in place, +you get more information: + + 1..8 + ok 1 - new() returned something + ok 2 - and it's the right class + ok 3 - sec() + ok 4 - min() + ok 5 - hour() + not ok 6 - day() + # Failed test (- at line 16) + # got: '16' + # expected: '17' + ok 7 - month() + ok 8 - year() + # Looks like you failed 1 tests of 8. + +Aha. C<$ical-E<gt>day> returned 16, but we expected 17. A +quick check shows that the code is working fine, we made a mistake +when writing the tests. Change it to: + + is( $ical->day, 16, ' day()' ); + +... and everything works. + +Any time you're doing a "this equals that" sort of test, use C<is()>. +It even works on arrays. The test is always in scalar context, so you +can test how many elements are in an array this way. [5] + + is( @foo, 5, 'foo has 5 elements' ); + + +=head2 Sometimes the tests are wrong + +This brings up a very important lesson. Code has bugs. Tests are +code. Ergo, tests have bugs. A failing test could mean a bug in the +code, but don't discount the possibility that the test is wrong. + +On the flip side, don't be tempted to prematurely declare a test +incorrect just because you're having trouble finding the bug. +Invalidating a test isn't something to be taken lightly, and don't use +it as a cop out to avoid work. + + +=head2 Testing lots of values + +We're going to be wanting to test a lot of dates here, trying to trick +the code with lots of different edge cases. Does it work before 1970? +After 2038? Before 1904? Do years after 10,000 give it trouble? +Does it get leap years right? We could keep repeating the code above, +or we could set up a little try/expect loop. + + use Test::More tests => 32; + use Date::ICal; + + my %ICal_Dates = ( + # An ICal string And the year, month, day + # hour, minute and second we expect. + '19971024T120000' => # from the docs. + [ 1997, 10, 24, 12, 0, 0 ], + '20390123T232832' => # after the Unix epoch + [ 2039, 1, 23, 23, 28, 32 ], + '19671225T000000' => # before the Unix epoch + [ 1967, 12, 25, 0, 0, 0 ], + '18990505T232323' => # before the MacOS epoch + [ 1899, 5, 5, 23, 23, 23 ], + ); + + + while( my($ical_str, $expect) = each %ICal_Dates ) { + my $ical = Date::ICal->new( ical => $ical_str ); + + ok( defined $ical, "new(ical => '$ical_str')" ); + ok( $ical->isa('Date::ICal'), " and it's the right class" ); + + is( $ical->year, $expect->[0], ' year()' ); + is( $ical->month, $expect->[1], ' month()' ); + is( $ical->day, $expect->[2], ' day()' ); + is( $ical->hour, $expect->[3], ' hour()' ); + is( $ical->min, $expect->[4], ' min()' ); + is( $ical->sec, $expect->[5], ' sec()' ); + } + +Now we can test bunches of dates by just adding them to +C<%ICal_Dates>. Now that it's less work to test with more dates, you'll +be inclined to just throw more in as you think of them. +Only problem is, every time we add to that we have to keep adjusting +the C<use Test::More tests =E<gt> ##> line. That can rapidly get +annoying. There are ways to make this work better. + +First, we can calculate the plan dynamically using the C<plan()> +function. + + use Test::More; + use Date::ICal; + + my %ICal_Dates = ( + ...same as before... + ); + + # For each key in the hash we're running 8 tests. + plan tests => keys(%ICal_Dates) * 8; + + ...and then your tests... + +To be even more flexible, use C<done_testing>. This means we're just +running some tests, don't know how many. [6] + + use Test::More; # instead of tests => 32 + + ... # tests here + + done_testing(); # reached the end safely + +If you don't specify a plan, C<Test::More> expects to see C<done_testing()> +before your program exits. It will warn you if you forget it. You can give +C<done_testing()> an optional number of tests you expected to run, and if the +number ran differs, C<Test::More> will give you another kind of warning. + + +=head2 Informative names + +Take a look at the line: + + ok( defined $ical, "new(ical => '$ical_str')" ); + +We've added more detail about what we're testing and the ICal string +itself we're trying out to the name. So you get results like: + + ok 25 - new(ical => '19971024T120000') + ok 26 - and it's the right class + ok 27 - year() + ok 28 - month() + ok 29 - day() + ok 30 - hour() + ok 31 - min() + ok 32 - sec() + +If something in there fails, you'll know which one it was and that +will make tracking down the problem easier. Try to put a bit of +debugging information into the test names. + +Describe what the tests test, to make debugging a failed test easier +for you or for the next person who runs your test. + + +=head2 Skipping tests + +Poking around in the existing Date::ICal tests, I found this in +F<t/01sanity.t> [7] + + #!/usr/bin/perl -w + + use Test::More tests => 7; + use Date::ICal; + + # Make sure epoch time is being handled sanely. + my $t1 = Date::ICal->new( epoch => 0 ); + is( $t1->epoch, 0, "Epoch time of 0" ); + + # XXX This will only work on unix systems. + is( $t1->ical, '19700101Z', " epoch to ical" ); + + is( $t1->year, 1970, " year()" ); + is( $t1->month, 1, " month()" ); + is( $t1->day, 1, " day()" ); + + # like the tests above, but starting with ical instead of epoch + my $t2 = Date::ICal->new( ical => '19700101Z' ); + is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" ); + + is( $t2->epoch, 0, " and back to ICal" ); + +The beginning of the epoch is different on most non-Unix operating systems [8]. +Even though Perl smooths out the differences for the most part, certain ports +do it differently. MacPerl is one off the top of my head. [9] Rather than +putting a comment in the test and hoping someone will read the test while +debugging the failure, we can explicitly say it's never going to work and skip +the test. + + use Test::More tests => 7; + use Date::ICal; + + # Make sure epoch time is being handled sanely. + my $t1 = Date::ICal->new( epoch => 0 ); + is( $t1->epoch, 0, "Epoch time of 0" ); + + SKIP: { + skip('epoch to ICal not working on Mac OS', 6) + if $^O eq 'MacOS'; + + is( $t1->ical, '19700101Z', " epoch to ical" ); + + is( $t1->year, 1970, " year()" ); + is( $t1->month, 1, " month()" ); + is( $t1->day, 1, " day()" ); + + # like the tests above, but starting with ical instead of epoch + my $t2 = Date::ICal->new( ical => '19700101Z' ); + is( $t2->ical, '19700101Z', "Start of epoch in ICal notation" ); + + is( $t2->epoch, 0, " and back to ICal" ); + } + +A little bit of magic happens here. When running on anything but MacOS, all +the tests run normally. But when on MacOS, C<skip()> causes the entire +contents of the SKIP block to be jumped over. It never runs. Instead, +C<skip()> prints special output that tells C<Test::Harness> that the tests have +been skipped. + + 1..7 + ok 1 - Epoch time of 0 + ok 2 # skip epoch to ICal not working on MacOS + ok 3 # skip epoch to ICal not working on MacOS + ok 4 # skip epoch to ICal not working on MacOS + ok 5 # skip epoch to ICal not working on MacOS + ok 6 # skip epoch to ICal not working on MacOS + ok 7 # skip epoch to ICal not working on MacOS + +This means your tests won't fail on MacOS. This means fewer emails +from MacPerl users telling you about failing tests that you know will +never work. You've got to be careful with skip tests. These are for +tests which don't work and I<never will>. It is not for skipping +genuine bugs (we'll get to that in a moment). + +The tests are wholly and completely skipped. [10] This will work. + + SKIP: { + skip("I don't wanna die!"); + + die, die, die, die, die; + } + + +=head2 Todo tests + +While thumbing through the C<Date::ICal> man page, I came across this: + + ical + + $ical_string = $ical->ical; + + Retrieves, or sets, the date on the object, using any + valid ICal date/time string. + +"Retrieves or sets". Hmmm. I didn't see a test for using C<ical()> to set +the date in the Date::ICal test suite. So I wrote one: + + use Test::More tests => 1; + use Date::ICal; + + my $ical = Date::ICal->new; + $ical->ical('20201231Z'); + is( $ical->ical, '20201231Z', 'Setting via ical()' ); + +Run that. I saw: + + 1..1 + not ok 1 - Setting via ical() + # Failed test (- at line 6) + # got: '20010814T233649Z' + # expected: '20201231Z' + # Looks like you failed 1 tests of 1. + +Whoops! Looks like it's unimplemented. Assume you don't have the time to fix +this. [11] Normally, you'd just comment out the test and put a note in a todo +list somewhere. Instead, explicitly state "this test will fail" by wrapping it +in a C<TODO> block: + + use Test::More tests => 1; + + TODO: { + local $TODO = 'ical($ical) not yet implemented'; + + my $ical = Date::ICal->new; + $ical->ical('20201231Z'); + + is( $ical->ical, '20201231Z', 'Setting via ical()' ); + } + +Now when you run, it's a little different: + + 1..1 + not ok 1 - Setting via ical() # TODO ical($ical) not yet implemented + # got: '20010822T201551Z' + # expected: '20201231Z' + +C<Test::More> doesn't say "Looks like you failed 1 tests of 1". That '# +TODO' tells C<Test::Harness> "this is supposed to fail" and it treats a +failure as a successful test. You can write tests even before +you've fixed the underlying code. + +If a TODO test passes, C<Test::Harness> will report it "UNEXPECTEDLY +SUCCEEDED". When that happens, remove the TODO block with C<local $TODO> and +turn it into a real test. + + +=head2 Testing with taint mode. + +Taint mode is a funny thing. It's the globalest of all global +features. Once you turn it on, it affects I<all> code in your program +and I<all> modules used (and all the modules they use). If a single +piece of code isn't taint clean, the whole thing explodes. With that +in mind, it's very important to ensure your module works under taint +mode. + +It's very simple to have your tests run under taint mode. Just throw +a C<-T> into the C<#!> line. C<Test::Harness> will read the switches +in C<#!> and use them to run your tests. + + #!/usr/bin/perl -Tw + + ...test normally here... + +When you say C<make test> it will run with taint mode on. + + +=head1 FOOTNOTES + +=over 4 + +=item 1 + +The first number doesn't really mean anything, but it has to be 1. +It's the second number that's important. + +=item 2 + +For those following along at home, I'm using version 1.31. It has +some bugs, which is good -- we'll uncover them with our tests. + +=item 3 + +You can actually take this one step further and test the manual +itself. Have a look at L<Test::Inline> (formerly L<Pod::Tests>). + +=item 4 + +Yes, there's a mistake in the test suite. What! Me, contrived? + +=item 5 + +We'll get to testing the contents of lists later. + +=item 6 + +But what happens if your test program dies halfway through?! Since we +didn't say how many tests we're going to run, how can we know it +failed? No problem, C<Test::More> employs some magic to catch that death +and turn the test into a failure, even if every test passed up to that +point. + +=item 7 + +I cleaned it up a little. + +=item 8 + +Most Operating Systems record time as the number of seconds since a +certain date. This date is the beginning of the epoch. Unix's starts +at midnight January 1st, 1970 GMT. + +=item 9 + +MacOS's epoch is midnight January 1st, 1904. VMS's is midnight, +November 17th, 1858, but vmsperl emulates the Unix epoch so it's not a +problem. + +=item 10 + +As long as the code inside the SKIP block at least compiles. Please +don't ask how. No, it's not a filter. + +=item 11 + +Do NOT be tempted to use TODO tests as a way to avoid fixing simple +bugs! + +=back + +=head1 AUTHORS + +Michael G Schwern E<lt>schwern@pobox.comE<gt> and the perl-qa dancers! + +=head1 COPYRIGHT + +Copyright 2001 by Michael G Schwern E<lt>schwern@pobox.comE<gt>. + +This documentation is free; you can redistribute it and/or modify it +under the same terms as Perl itself. + +Irrespective of its distribution, all code examples in these files +are hereby placed into the public domain. You are permitted and +encouraged to use this code in your own programs for fun +or for profit as you see fit. A simple comment in the code giving +credit would be courteous but is not required. + +=cut |