Running the tests and understanding the results ----------------------------------------------- To run all the tests just do $ ./test.sh For each test the output begins with the line: making test: dir/file options... dir is the dir where the test is made, file is the texinfo file processed and the optional options are the options passed to texi2html when doing the test. Then the test results are outputted. There are 5 type of results outputed: status -> always stderr line count -> always diffs -> optionnal test with tidy -> optionnal test validity -> optionnal For each of these, the type is outputted indented. The results appear on lines below, more indented. When the test fails there is !!! on the line. For example: making test: formatting/not_closed_in_menu.texi status: passed stderr line count: !!! bad line count: 7 != 0 I now turn the the individual result type: * status The exit status of the command is tested for success or failure. * stderr line count The number of stderr lines is compared against a reference. * diffs out files are diffed against reference files present in dir_res. A '.' means a successfull comparison. If there are more than one test in that dir, there might be files differing or files not found if the files haven't allready been regenerated. * test with tidy tidy is run on resulting html files. * test validity validate is run on resulting html files. Adding a test ------------- To add a test, you have to add a line in test.sh, near the end. Here is the syntax: test_texi dir file "option1 option2" stderr_lines suffix basename ignore_tags validate test_tidy fail dir is the only mandatory argument. There are defaults for the other arguments when not given. However if you want to specify, say, basename and use the defaults for the preceding ones, you have to set them explicitely. dir: the directory where the files are. file: the texinfo file. If not present dir.texi is used "option1 option2": options passed to texi2html. Default is no option stderr_lines: number of lines of stderr. Default is 0 suffix: suffix of the file, used to find the basename. 'texi' is assumed basename: basename of the generated files. The basename of file is the default ignore_tags: if set to 'ignore_tags' or 'yes' rcs tags are ignored validate: if unset or set to 'validate' validation is tried test_tidy: if unset or set to 'tidy' or 'test_tidy', tidy is tried fail: if unset or set to 'success' the exit code should be 0 * Simple example: test_texi macros test macros.texi in the macros dir, without options, assuming generated files beginning with macros. * Complex example: test_texi sectionning no_node.texi "-prefix chapter_split_no_node -split chapter" 0 texi chapter_split_no_node Test no_node.texi in the sectionning dir, with the options: -prefix chapter_split_no_node -split chapter no lines on stderr, suffix is texi and basename for generated files is chapter_split_no_node (which matches the option -prefix chapter_split_no_node). Remarks: The stderr output is in basename.2, the result of the first pass in basename.passtexi, of the second pass in basename.passfirst. the -test option is always passed. To run a single test you can run test.sh with the arguments you would have passed to test_texi. For example $./test.sh viper viper.texi "-split chapter" will run a test for viper.texi in the viper directory with option -split chapter Regenerating tests ------------------ make regenerate copy all the *.html, *.2, *.passtexi and *.passfirst from the test directories to the result directories (test directory with _res appended). This is usefull when the new result is the good result and you want to avoid manually copying the good files to the result directory. To add a new test directory add it to the SUBDIR_TESTS variable in Makefile.am (and rerun automake, autoconf...).