CloudHost/zonemaster.es
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: populate ldns submodule and add autotools to LDNS build stage

Browse Source 
- Re-cloned zonemaster-ldns with --recurse-submodules so the bundled
  ldns C library source (including Changelog and configure.ac) is present
- Added autoconf, automake, libtool to Dockerfile.backend ldns-build stage
  so libtoolize + autoreconf can generate ldns/configure during make

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Malin
2026-04-21 08:33:38 +02:00
parent 8d4eaa1489
commit eaaa8f6a11
541 changed files with 138189 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
zonemaster-ldns/ldns/test/tpkg/ChangeLog Normal file
View File

@@ -0,0 +1,81 @@
8 December 2006: Wouter Wijngaards: 1.12
* More portable way to extract info from desc files, using
grep and sed, instead of awk IFS/-F.
3 August 2006: Wouter Wijngaards: 1.11
* tar usage more portable.
* Multiple tests in Depends: possible.
* Usage of tar and awk more portable.
25 Apr 2006: Miek Gieben: 1.10
* Allow tpkg report to work again
* Base all operations on tar (for speed)
* Make tar work on the open
* list/extract/etc work again. they call tpkg extract now
* cleandir target added, clear *.dir directories
05 Apr 2006: Miek Gieben: 1.07
* Added -p priorty for the logging
* Always add return code to result file
* Add default action: exe when none is given
04 Apr 2006: Miek Gieben: 1.06
* Added: CmdDepends: [cmd], [cmd]
* Solaris support was added/tested
* -k now also works for execute: don't delete the tmp. directory.
* post-commit handles branches/tags and trunk, although prob.
not combined.
* Added -n NUM switch, to influence tpkg report's exit code
* When cloning rename the testfiles mentioned in the files too
* Add pre/post output to the output of tpkg report
* Add logging to syslog -l "mesg"
08 Nov 2005: Miek Gieben: 1.05
* Various little updates that make 'tpkg' complete
31 Oct 2005: Miek Gieben: 1.04
* When a test fails show the result log in "tpkg report"
* When running with -q don't show succeses
31 Oct 2005: Miek Gieben: 1.03
* Added status line for tpkg report
* Cleanup documentation to match .tkpg.var.master
and .tpkg.var.test
28 Oct 2005: Miek Gieben: 1.02
* svnserve doesn't set USERNAME nor LOGNAME, add this
to the post-commit script
* create post and pre file when target is tmpl -
stuff them with stuff handy when running from svn(serve)
* -q now only prints the errors
* Use a new system to convey information between the tests
and scripts; .tpkg.var.master sets up stuff each test
must know. Tests use a .tpkg.var.test file in their
local directory to pass info from .pre to .test to .post
* Allow shorter targets -> exe -> e, extract -> ext, etc.
27 Oct 2005: Miek Gieben: 1.01
* Added -q flag
* Don't abort when a pre/post script fails, but
continue the test and hope it writes a result file
* Updated the manual and docs
* Updated the post-commit hook script
27 Oct 2005: Miek Gieben: 1.00
* Added help file and updated documentation
* When cloning update the date in the new test
* Consistency of the .dsc is checked
* -k now works, also when cloning
* Abort on unknown options
* Some updates to the layout of the tpkg script
26 Oct 2005: Miek Gieben: 0.90
* Tagged this release
26 Oct 2005: Miek Gieben: 0.82
* Made tpkg clone more robust
* Reversed the tpkg report order make PASSED/FAILED standout more
26 Oct 2005: Miek Gieben: 0.81 (not tagged)
* Added -k option for tpkg create, which doesn't remove the test.dir/
* Documentation updates

225
zonemaster-ldns/ldns/test/tpkg/README Normal file
View File

@@ -0,0 +1,225 @@
TEST PACKAGE -- TPKG
(c) NLnet Labs, 2005, 2006. Miek Gieben
Licensed under the GPL version 2.
Test Package
The most important file in each test is the description
file, the one with the .dsc extension. The file is line based,
so options spanning multiple lines are not recognized or worse,
make tpkg fail.
A typical example looks like this:
BaseName: test1
Version: 0.1
Description: Test negative SOA TTL, added to test BUG #6
CreationDate: 05-10-2005
Maintainer: Miek Gieben
Category:
Component:
Depends:
Help: test1.help
Pre: test1.pre
Post: test1.post
Test: test1.test
AuxFiles: test1.a, test2.b
Passed:
Failure:
Details of each line follow:
BaseName:
The basename of the test. This is the name of the .dsc
file without the .dsc extension
Version:
Currently nothing is done with this. It could either
evolve in a version for the .dsc file or it could
be used to version your tests.
Description:
A short explanation of what this test is supposed to test.
'tpkg report' uses this.
CreationDate:
Automaticly set by 'tpkg tpml'. The original creation date
for this test.
Maintainer:
Who created/maintains this test
Category:
Under what category does this test fall. This is not
used by tpkg, but is a hint to the users of the tests.
This could something like: 'building', 'running', etc.
Component:
What software component are you testing, this could be
the name of the executeable. User decides what to put
here, but it should be consistend for all tests.
Depends:
On what other test does this test depend. The full
package name should be given, with the .tpkg extension.
Currently this is limited to one test.
Help:
Name of a file that has a few lines of usefull information
to the user of the test. The file must have a .help suffix.
Pre:
Name of a script that should be executed before the
test is run. If the pre-script fails the test fails.
Post:
Name of a script that should be executed after the
test has run. If the post-script fails the test fails.
Test:
Name fo the main test script.
Auxfiles:
Other files that are needed to run this test, i.e. to
compare against. Currently this list is limited to 8
other files. The list should use commas as delimiters:
fileA, fileB, FileC, ...
Passed:
A regular expression that is matched against the output
of the main test script. If the expression matches the
test is a succes.
Failure:
A regular expression that is matched against the output
of the main test script. If the expression matches the
test is a failure.
All files used in 1 test must have the same basename, otherwise
'tpkg create' will not pick them up.
When is a Test Successfull and When Not
A test is failed when:
* The test script returns with a non-zero value.
* The test script returns with zero, and
the 'Failure' regexp matches.
A test is passed when:
* The test script returns with zero.
* The test script returns with zero, and
the 'Passed' regexp matches.
A test is neither failed nor passed when:
* Test test script returns with zero, and
neither the 'Passed' nor 'Failure' regular
expressions matched.
If this happens you should rewrite your test.
Writting Test Packages
.test file
Start each test with a source of the .tpkg.var.master, like
this:
[ -f ../.tpkg.var.master ] && source ../.tpkg.var.master
This is automaticly added when =tpkg tmpl TEST=
is called.
We use '../' here cause '.' is cleaned up after
the test is run
It might also be prudent to source the .tpkg.var.test file for test
specific enviroment variables.
Each test is executed in a temporary directory, keep this
in mind when using relative pathnames. ../exe must become
../../exe will the test be able to find it.
Test your test before creating a .tpkg file, this can
be as simple as: bash ./testX.test
Test Output
result. files
A typical example of a result. file is this:
** PASSED ** ** PASSED **
BaseName: svn-build
Description: Check out and build libdns/trunk
DateRun: Fri Oct 7 20:28:23 CEST 2005
--------------- Test Output ------------------
A summery of result files of test can be seen by running
tpkg report.
Running Tests:
Each test is executed in the new temporary directory.
tpkg exe test1
tpkg exe test2
...
tpkg exe testN
tpkg report
generate a report of all the tests
tpkg clean
remove all the result. files
Reading Reports:
Typical example:
** PASSED ** : 4 s : ldns-splint : Run a make lint test on the libdns sources
** PASSED ** : 2 s : svn-build: : Check out and build libdns/trunk
PASSED: 2 (100.00 %) FAILED: 0 (0 %) UNKNOWN: 0 (0 %)
Sample Output For NSD
I've written a few tests for unit testing NSD, running these tests
looks like this:
% tpkg exe rfc3597-good.tpkg && tpkg exe rfc3597-bad.tpkg
[rfc3597-good] [log] Extracting...
[rfc3597-good] [log] executing dependency test: nsd-compile.tpkg
[nsd-compile] [log] Extracting...
[nsd-compile] [log] Executing test
[nsd-compile] [log] ** PASSED **
[nsd-compile] [log] Cleaning up
[rfc3597-good] [log] Executing test
[rfc3597-good] [log] ** PASSED **
[rfc3597-good] [log] Cleaning up
[rfc3597-bad] [log] Extracting...
[rfc3597-bad] [log] executing dependency test: nsd-compile.tpkg
[nsd-compile] [log] Extracting...
[nsd-compile] [log] Found .done-nsd-compile. Not executing this test.
[nsd-compile] [log] Cleaning up
[rfc3597-bad] [log] Executing test
[rfc3597-bad] [log] ** PASSED **
[rfc3597-bad] [log] Cleaning up
% tpkg report
nsd-compile: 1 s : ** PASSED ** : Compile NSD
rfc3597-bad: 3 s : ** PASSED ** : Compile a zone with unknown rr data (the zone is BAD)
rfc3597-good: 2 s : ** PASSED ** : Compile a zone with unknown rr data (the zone is good)
PASSED: 3 (100.00 %) FAILED: 0 (0 %) UNKNOWN: 0 (0 %)
% tpkg clean
[log] rm result.nsd-compile
[log] rm result.rfc3597-bad
[log] rm result.rfc3597-good
[log] rm .tpkg.var
[log] rm .done files

150
zonemaster-ldns/ldns/test/tpkg/WRITING-TESTS Normal file
View File

@@ -0,0 +1,150 @@
WRITING TESTS WITH TPKG
(c) NLnet Labs, 2005. Miek Gieben
Licensed under the GPL version 2.
Introduction
In itself tpkg doesn't make writing tests any easier, it is just a
container format. The following sections will over some guidelines
and guidance on creating new tests.
As always, looking at other tests will help you understand tpkg
better :-)
Handy Commands
tpkg list <testname.tpkg> - list the contents of a test
tpkg fake <testname.tpkg> - fake a testrun - this write a
.done file
Writing a Test
First choose a language. tpkg itself is writting in bash, and the
scripts are also assumed to be shell scripts, but nothing prevents
you from using the #!-syntax to fork another interpreter.
Next comes to actual scripting/writing.
Step 1
Execute a 'tpkg tmpl <testname>', where <testname>
is a descriptive name for you test. Assume our test
is called 'bug001'.
tpkg will create a subdirectory named bug001.dir and
will put the following files in there:
- bug001.dsc - description file
- bug001.test - main test script
- bug001.help - help file for the user
The description file is line based, multiline entries
will not be parsed correctly. Also tpkg using the
colon ':' as a delimeter, so that character cannot
be used in the for instance the "Description:" line.
All files that are used in a test MUST have the same
prefix. In this case that is 'bug001'.
Step 2
Fill out the bug001.dsc file, i.e. add a description
in there. You are now ready to create your test in
bug001.test. (Also check step 3, 4 and 5).
Step 2a
Fill out the bug001.help file. Thing like special
arguments to the test can be put in this file.
Step 3
Are there things that need to be done before the
test can be run, i.e. setting up some environment?
This can be done best in a .pre script. If you create
such a script you must also add that to bug001.dsc,
under 'Pre:'. If this script fails, the main test
script is not execeuted and the test is aborted.
Step 4
Anything that was setup with a pre script should be
taken down with a post script. Everything that applies
for the pre-script also applies for a post script, except
that a failure of the post script doesn't abort the test.
Step 5
Do you need any extra files? If yes, list them under
"AuxFiles:" in the description file.
Step 6
When you are happy about your test, and it works:
bash ./bug001.test executes correctly, you can create
your test package. Go up one directory:
cd ..
And create your test:
tpkg create bug001.tpkg
This will get all files from the bug001.dir/ and will
create the shell archive named bug001.tpkg
Step 7
Test your test:
tpkg exe bug001.tpkg
This should create a .done-bug001 and a result-bug001
file. If the test failed you can re-examine the test
by unpacking it:
tpkg extract bug001.tpkg
This will re-create the bug001.dir/ with all the files
in it.
Step 8
If everything is gone allright the test can be added
to the archive of the package that you are testing.
Pittfalls
The following list are known pittfalls and how to avoid them:
* Path issues
When a test is executed from subversion, by using the
post-commit hook, the $PATH is cleared for security
reasons. This also means that some utilities that might be
used in the test cannot be found.
It is prudent to (re)set the $PATH at the top of each
test script, or use absolute pathnames for /all/ external
commands in your test.
* Relative path issues
Each test is executed in a random subdirectory, there it
is also unpacked. So anything you reference with ../ will
need an extra ../ when the final test is ready. Almost
always this will not be an issue because the writing and
testing of the test also happens in a subdirectory. But
keep this in mind when a test doen't work as a final
package.
Conventions
.tpkg.var.master
Sometimes you want to provide the main test script with information
only available in the pre script. Also some tests may need
information from other tests.
tpkg uses a files named ".tpkg.var.master" for this purpose. This file
is /not/ stored in the temporary directory where the test is
executed, because it must survive after the test has completed.
Things like usernames, paths or other environment variables
might be stored in this file.
When writing to the file be sure to use >> in stead of > which
destroys any previous content.
.tpkg.var.master lives for the duration of all the tests.
It is deleted when 'tpkg clean' runs
.tpkg.var.test
During the run of a test one might want to exchange
information from a .pre script to the main test. For
instance when the pre script sets up a server it may want
to communicate the port number to the test script.
"tpkg.var.test" is meant for these local exchanges.
The file lives in the temporary subdirectory where the
test is executed. This also means it is removed after
the test has run.

BIN
zonemaster-ldns/ldns/test/tpkg/examples/compile-nsd.tpkg Normal file
View File

Binary file not shown.

6
zonemaster-ldns/ldns/test/tpkg/examples/dotest Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/sh
# setup the master file
echo "HALLO=\"biet\"" > .tpkg.var.master
# the test will pick this up
tpkg exe test2.tpkg

92
zonemaster-ldns/ldns/test/tpkg/examples/post-commit Normal file
View File

@@ -0,0 +1,92 @@
#!/bin/sh
# set the path for our tests
PATH=$PATH:/usr/local/bin:/usr/local/sbin:.
REPOS="$1" # /svn/nsd or /svn/libdns # get these from svn
REV="$2" # which revision number
SVN_BAR="------------------------------------------------------------------------"
# our environment is extremely limited we don't even
# get a LOGNAME here... this makes all the pre script
# fail in the test. We do get a pwd from svnserve
LOGNAME=`basename $PWD`
USERNAME=$LOGNAME
id $USERNAME
if [[ $1 -eq 1 ]]; then
# no legal username
exit 0
fi
# please edit these to suit your needs
TPKG=/home/miekg/svn/tpkg/trunk/tpkg # tpkg version to use
MAILTO="ldns-team@nlnetlabs.nl" # where to send the status email
NAME="ldns" # name of the project
MAILSUB="[svn: $NAME|$BRANCH] $REV status" # subject of status emails
REPOPROT="file://" # local or remove (svn+ssh://)
TESTDIR="test" # dir where the tests live in the repo
# get the branch name
LOOK=`/usr/bin/svnlook changed -r$REV $REPOS | /usr/bin/awk '{ print $2 }'`
if [[ $LOOK =~ '^trunk' ]]; then
REPOPATH="trunk"
fi
if [[ $LOOK =~ '^branches' ]]; then
REPOPATH=`echo $LOOK | cut -f1,2 -d /`
fi
if [[ $LOOK =~ '^tags' ]]; then
REPOPATH=`echo $LOOK | cut -f1,2 -d /`
fi
BUILD_DIR=`mktemp -d /tmp/XXXXXX`
if [ ! -d $BUILD_DIR ]; then
exit 0
fi
START=`date +%s`
MAIL_FILE=`mktemp /tmp/XXXXXXX`
# checkout to $BUILD_DIR, assume local checkout
svn co -r$REV $REPOPROT$REPOS/$REPOPATH $BUILD_DIR
( cd $BUILD_DIR ; svn log -r$REV > $MAIL_FILE )
svnlook changed -r$REV $REPOS >> $MAIL_FILE
echo $SVN_BAR >> $MAIL_FILE
# set the path and other stuff for our sibling test scripts
echo "export PATH=$PATH" > $BUILD_DIR/$TESTDIR/.tpkg.var.master
echo "export LOGNAME=$LOGNAME" >> $BUILD_DIR/tpkg/.tpkg.var.master
echo "export USERNAME=$USERNAME" >> $BUILD_DIR/tpkg/.tpkg.var.master
###
# This should be the only thing you need to tweak
###
if [[ -d $BUILD_DIR/$TESTDIR ]]; then
# RUN THE TESTS
for tests in $BUILD_DIR/$TESTDIR/*.tpkg ; do
nice $TPKG -b $BUILD_DIR/$TESTDIR -a $BUILD_DIR \
exe `basename $tests` >> $MAIL_FILE
done
END=`date +%s`
echo ELAPSED: $((END - START)) s >> $MAIL_FILE
echo $SVN_BAR >> $MAIL_FILE
# add -q to /not/ show PASSED tests
$TPKG -b $BUILD_DIR/$TESTDIR report >> $MAIL_FILE
$TPKG -b $BUILD_DIR/$TESTDIR clean
###
# End of tweaking
###
fi
# SVN STUFF
( cd $BUILD_DIR ; svn diff -rPREV:$REV >> $MAIL_FILE )
rm -rf $BUILD_DIR
cat $MAIL_FILE | mail -s "${MAILSUB}" $MAILTO
rm -f $MAIL_FILE

BIN
zonemaster-ldns/ldns/test/tpkg/examples/test1.tpkg Normal file
View File

Binary file not shown.

BIN
zonemaster-ldns/ldns/test/tpkg/examples/test2.tpkg Normal file
View File

Binary file not shown.

15
zonemaster-ldns/ldns/test/tpkg/examples/test3.tpkg/test3.dsc Normal file
View File

@@ -0,0 +1,15 @@
BaseName: test3
Version: 1.0
Description: Test negative SOA TTL, added to test BUG #6
CreationDate: Thu Oct 27 11:57:07 CEST 2005
Maintainer: Miek Gieben
Category:
Component:
Depends:
Help:
Pre: test3.pre
Post: test3.post
Test: test3.test
AuxFiles: test3.querygood
Passed: "matched"
Failure:

3
zonemaster-ldns/ldns/test/tpkg/examples/test3.tpkg/test3.post Normal file
View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "He! Ruim toch eens een keer je troep op!"

3
zonemaster-ldns/ldns/test/tpkg/examples/test3.tpkg/test3.pre Normal file
View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Dit script word vooraf uitgevoerd"

1
zonemaster-ldns/ldns/test/tpkg/examples/test3.tpkg/test3.querygood Normal file
View File

@@ -0,0 +1 @@
elektron.atoom.net.3600INAAAA2001:7b8:32a:0:280:48ff:feb3:26eb

10
zonemaster-ldns/ldns/test/tpkg/examples/test3.tpkg/test3.test Normal file
View File

@@ -0,0 +1,10 @@
#!/bin/sh
dig miek.nl | grep AAAA | sed 's/[ \t]*//g' > $$.digout
diff -u test3.querygood $$.digout
if [ $? -ne 0 ]; then
echo "Query failed"
else
echo "Query matched"
fi
rm -f $$.digout

945
zonemaster-ldns/ldns/test/tpkg/tpkg Executable file
View File

@@ -0,0 +1,945 @@
#!/usr/bin/env bash
# a utlity to run a shar test archive (aka tpkg)
# Created by Miek Gieben, NLnetLabs, (c) 2005, 2006
# Licensed under GPL version 2
export TPKG_VAR_MASTER="../.tpkg.var.master"
export TPKG_VAR_TEST=".tpkg.var.test"
export TPKG_VERSION="1.12";
export SHELL="/bin/sh"
TPKG_LOGGER=/usr/bin/logger
TPKG_BASE="."
TPKG_ARGS=""
TPKG_CURRENT=`pwd`
TPKG_QUIET=0 # only output err() msgs
TPKG_VERBOSE=0 # Show live output of test execution
TPKG_KEEP=0 # tpkg create doesn't remove dir/
TPKG_PASS=0 # how much must succeed
TPKG_LOG=0 # don't log
TPKG_PRI="" # log facility
TPKG_FORMAT="targz" # format of underlying tpkg files
TPKG_FORCE=0 # execute tests even when .done file is found
_DESC_WIDTH=${COLUMNS:-55}
if [ $_DESC_WIDTH -ge 70 ]; then
_DESC_WIDTH=70
fi
### Helper functions
function cleanup() {
out "[log] Cleaning up"
[[ -f result.$dsc_basename ]] && cp result.$dsc_basename ../
cd ..
if [[ ! -z "$dir" ]]; then
rm -rf `basename $dir`
fi
cd $TPKG_CURRENT
}
function cleanup_and_exit() {
cleanup; exit 1
}
function err() {
if [[ -z $testname ]]; then
echo -e " $1"
else
echo -e "[$testname] $1"
fi
}
function tpkg_log() {
if [[ $TPKG_LOG -eq 0 ]]; then
return
fi
if [[ -e $TPKG_LOGGER ]]; then
if [[ -z $TPKG_PRI ]]; then
$TPKG_LOGGER "$1"
else
$TPKG_LOGGER -p "$TPKG_PRI" "$1"
fi
fi
}
function out() {
if [[ $TPKG_QUIET -eq 1 ]]; then
return
fi
if [[ -z $testname ]]; then
echo -e " $1"
else
echo -e "[$testname] $1"
fi
}
function write_result() {
if [[ $TPKG_VERBOSE -gt 0 ]]; then
tee -a $1
else
cat >> $1
fi
}
function epoch() {
# make this sorta portable although not needed now
epoch=0
case $OSTYPE in
linux*)
epoch=`date +%s`
;;
freebsd*)
epoch=`date +%s`
;;
esac
}
function post() {
if [ -f "${dsc_post}" ]; then
err "[log] Executing post script: ${dsc_post} ${TPKG_ARGS}"
echo "--------- Start Post Output ------------------ " | write_result result.$dsc_basename
${SHELL} ${dsc_post} ${TPKG_ARGS} | write_result result.$dsc_basename
echo "----------- End Post Output ------------------ " | write_result result.$dsc_basename
result=$?
if [ $result -ne 0 ]; then
err "[warning] Post-script executed with errors: $result."
fi
fi
}
function pre() {
if [ -f "${dsc_pre}" ]; then
err "[log] Executing pre script: ${dsc_pre} ${TPKG_ARGS}"
echo "--------- Start Pre Output ------------------- " | write_result result.$dsc_basename
${SHELL} ${dsc_pre} ${TPKG_ARGS} | write_result result.$dsc_basename
echo "----------- End Pre Output ------------------- " | write_result result.$dsc_basename
result=$?
if [ $result -ne 0 ]; then
err "[warning] Pre-script executed with errors: $result."
fi
fi
}
function write_done() {
# we are executing in a subdir
if [ -f "../.done-${testname}" -a $TPKG_FORCE -ne 1 ]; then
err "[warning] Overwriting .done-${testname}"
fi
> ../.done-${testname}
}
# write done file in current dir
function write_fake_done() {
if [ -f ".done-${testname}" -a $TPKG_FORCE -ne 1 ]; then
err "[warning] Overwriting .done-${testname}"
fi
> .done-${testname}
}
function mktempdir() {
# check if mktemp is there, if not use plain mkdir with $$
# as a side effect set $dir
dir=
case $OSTYPE in
solaris*)
# use mkdir
dir="$1.$$"
mkdir "$dir"
return
;;
*)
dir=`mktemp -d "$1"`
return
;;
esac
}
function usage() {
out "Usage:"
out "$0 [OPTIONS] [exe|create|extract|tmpl|fake] test.tpkg"
out "or:"
out "$0 [OPTIONS] [report|clean|list|desc|help] test.tpkg"
out "or:"
out "$0 [OPTIONS] clone test1.tpkg test2.tpkg"
out
out "Testing"
out " exe.........:\texecute a test, safe the result result.testname"
out " c | create..:\tcreate a .tpkg out of the test.{pre, post, test} files"
out " e | extract.:\textract a .tpkg to tmp. dir"
out " t | tmpl....:\tcreate empty template files for a new test"
out " f | fake....:\tfake the running of test, but do create a .done file"
out
out "When no action is given a test is executed"
out
out "Reporting/Cleanup"
out " clean........:\tremove all the result files"
out " cd | cleandir:\tremove all .dir directories"
out " r | report..:\tcreate a nice report from all the result files"
out " cl | clone...:\tclone test1.tpkg to test2.tkpg"
out " l | list....:\tprint the files of the test to stdout"
out " d | desc....:\tprint the test's description to stdout"
out " h | help....:\tprint the help message for this test, if available"
out
out " When multiple tests depend on a single other test, this"
out " other test is only executed once."
out
out "OPTIONS"
out " -h\t\tshow this help"
out " -v\t\tshow version"
out " -q\t\tonly print errors"
out " -V\t\tshow live output when executing tests"
out " -l\t\tlog test name to syslog when starting the test (using logger)"
out " -p PRI\tlog using PRI as priority"
out " -k\t\tdon't remove test directory when creating/executing a tpkg package"
out " -n NUM\tif less than NUM of the tests are passed exit with 1"
out " \t\tOtherwise exit with 0. When NUM is -1, no tests may fail"
out " \t\tOnly valid when running tpkg report"
out " \t\tOtherwise exit with 0. Only valid when running tpkg report"
out " -b DIR\tuse DIR is a base directory in stead of ."
out " -a ARGS\tpass the string ARGS through to the test scripts"
out " -d\t\tUse directories instead of tar.gz for tpkg archive format"
out " -f\t\tForce test to be re-run if already executed"
out
out " (C) NLnetLabs, Miek Gieben. Licensed under the GPL version 2."
}
function version() {
out "tpkg (test package), version $TPKG_VERSION"
out "Written by Miek Gieben, NLnet Labs"
out
out "Copyright (C) 2005, 2006 NLnet Labs"
out
out "This is free software; see the source for copying conditions. There is no"
out "warranty; even not for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE"
}
function cleanreport() {
# cleanup all the result. files
for result in `ls result.* 2>/dev/null`; do
err "[log] rm $result"
rm $result
done
# rm any .var files
out "[log] rm `basename $TPKG_VAR_MASTER`"
rm -f `basename $TPKG_VAR_MASTER`
rm -f $TPKG_VAR_TEST
out "[log] rm .done files"
rm -f .done*
cd $TPKG_CURRENT
}
function cleandirs() {
for result in `ls -d *.dir 2> /dev/null`; do
err "[log] rm -rf $result"
rm -rf $result
done
}
function report() {
# generate a report from the result. files.
passed=0
failed=0
unknown=0
first=0
tp="passed"
tf="failed"
tu="unknown"
for result in `ls result.* 2>/dev/null` ; do
passfailed=`head -1 $result | awk ' { print $2 }'`
basename=`head -3 $result | grep BaseName | awk -F': ?' ' { print $2 }'`
description=`head -4 $result | grep Description | awk -F': ?' ' { print $2 }'`
runend=`head -2 $result | grep DateRunEnd | awk -F': ?' ' { print $2 }'`
runstart=`head -5 $result | grep DateRunStart | awk -F': ?' ' { print $2 }'`
# truncate the description to 35 chars
if [ ${#description} -gt $_DESC_WIDTH ]; then
description=${description:0:$_DESC_WIDTH}
description=$description"..."
fi
if [ -z $runend ]; then
runend=0
fi
if [ -z $runstart ]; then
runstart=0
fi
((period=$runend - $runstart))
# prefix period if < 9
if [ $period -lt 10 ]; then
period="0$period"
fi
case $passfailed in
"FAILED")
if [ $first -eq 0 ]; then
echo " STATUS : ELAPSED : TESTNAME : TESTDESCRIPTION"
first=1
fi
echo -e "!! $passfailed !! : $period s : $basename : $description"
((failed=$failed + 1))
failed_tests="$failed_tests $basename"
tf="FAILED"
;;
"PASSED")
if [ $TPKG_QUIET -eq 0 ]; then
if [ $first -eq 0 ]; then
echo " STATUS : ELAPSED : TESTNAME : TESTDESCRIPTION"
first=1
fi
echo -e "** $passfailed ** : $period s : $basename : $description"
fi
((passed=$passed + 1))
tp="PASSED"
;;
*)
if [ $first -eq 0 ]; then
echo " STATUS : ELAPSED : TESTNAME : TESTDESCRIPTION"
first=1
fi
echo -e "-- $passfailed -- : $period s : $basename : $description"
((unknown=$unknown + 1))
failed_tests="$failed_tests $basename"
tu="UNKNOWN"
;;
esac
done
((total=$passed + $failed + $unknown))
if [[ $total -eq 0 ]]; then
fper=0
pper=0
uper=0
else
fper=`awk -vN=$failed -vT=$total 'BEGIN{printf("%.0f",(N/T*100))}'`
pper=`awk -vN=$passed -vT=$total 'BEGIN{printf("%.0f",(N/T*100))}'`
uper=`awk -vN=$unknown -vT=$total 'BEGIN{printf("%.0f",(N/T*100))}'`
fi
echo
echo -e "$tp: $passed ($pper %)\t$tf: $failed ($fper %)\t$tu: $unknown ($uper %)"
# for each failed test include the complete result file
# $i is basename
echo
for i in $failed_tests; do
echo --------------- Start Output: $i ------------------
cat result.$i
echo --------------- End Output: $i ------------------
done
cd $NT
if [[ $TPKG_PASS -gt 0 ]]; then
if [[ $passed -lt $TPKG_PASS ]]; then
exit 1
fi
elif [[ $TPKG_PASS -lt 0 ]]; then
if [[ $failed -gt 0 ]]; then
exit 1
fi
fi
exit 0
}
# clone test1 to test2
function clone() {
$0 extract $test1.tpkg
if [ $? -ne 0 ]; then
err "[fatal] Extract of $test1.tpkg failed. Abort."
exit 1
fi
if [ ! -d "$test1.dir" ]; then
err "[fatal] No $test1.dir directory? Abort."
exit 1
fi
cd $test1.dir
for i in $test1.* ; do
ext=`echo $i | sed s/$test1//`
if [ ! -z "$ext" ]; then
# rename the content of the files too
sed "s/$test1/$test2/g" < $i > $i.$$
mv $i.$$ $i
# rename
mv $i $test2$ext
fi
done
# edit the dsc file too
# update the date
sed "s/^CreationDate:.*/CreationDate: `date`/" < $test2.dsc > $test2.dsc.$$
mv $test2.dsc.$$ $test2.dsc
cd ..
# rename the dir
mv $test1.dir $test2.dir
if [ $TPKG_KEEP -eq 0 ]; then
if [ $TPKG_FORMAT = "dir" ]; then
$0 -d create $test2.tpkg
else
$0 create $test2.tpkg
fi
else
if [ $TPKG_FORMAT = "dir" ]; then
$0 -d -k create $test2.tpkg
else
$0 -k create $test2.tpkg
fi
fi
if [ $? -ne 0 ]; then
err "[warning] Creating of $test2.tpkg failed."
fi
cd $TPKG_CURRENT
}
# try to find the specific cmd
function find_cmd {
which "${i}" >/dev/null
if [ $? -ne 0 ]; then
err "[fatal] CmdDepend \"$i\" could not be satisfied: not found. Abort."
cleanup; exit 1
fi
}
# extract a tpkg to the given dir. The dir must exist already.
function extract_tpkg_to { # <dir>
out "[log] Extracting..."
# tar xfz ${testname}.tpkg -C $1 2>/dev/null
if [ -d "${test_pkg}" ]
then
cp -pr "${test_pkg}" "$1/${testname}.dir"
else
gzip -cd "${test_pkg}" | (cd $1; tar xf -) 2>/dev/null
fi
if [ $? -ne 0 ]; then
err "[fatal] Could not untar archive. Abort."
cd $TPKG_CURRENT; exit 1
fi
# now stuff is in: $1/testname.dir/...
mv $1/${testname}.dir $1/${testname}.dir.tmp$$
mv $1/${testname}.dir.tmp$$/* $1/.
rm -rf $1/${testname}.dir.tmp$$
}
### MAIN
# check the arguments
while getopts ":vhkqVb:a:n:lp:df" o
do case "$o" in
b) TPKG_BASE="$OPTARG";;
h) usage; exit 0;;
v) version; exit 0;;
l) TPKG_LOG=1;;
p) TPKG_PRI="$OPTARG";;
a) if [ -z "$TPKG_ARGS" ]
then
TPKG_ARGS="$OPTARG"
else
TPKG_ARGS="$TPKG_ARGS $OPTARG"
fi
;;
q) TPKG_QUIET=1;;
V) TPKG_VERBOSE=`expr $TPKG_VERBOSE + 1`;;
k) TPKG_KEEP=1;;
n) TPKG_PASS=$OPTARG
if [ $TPKG_PASS -eq 0 ]; then
err "[fatal] A null or non numerical value is not valid. Abort."
exit 1
fi
;;
d) TPKG_FORMAT="dir";;
f) TPKG_FORCE=1;;
*) err "[fatal] Unknown option. Abort."; exit 1;;
esac
done
shift $(($OPTIND - 1))
# go to the base dir
if [ ! -d $TPKG_BASE ]; then
err "[fatal] Directory $TPKG_BASE does not exist. Abort"
exit 1
else
cd $TPKG_BASE
fi
# either create a tpkg (ie. call shar) or exe (do a test)
goal=$1
archive=$2
if [ -z "${goal}" ]; then
usage
cd $TPKG_CURRENT; exit 0
fi
# allow short goals
case $goal in
# none for exe - short enough
c) goal="create";;
e) goal="extract";;
t) goal="tmpl";;
f) goal="fake";;
cd) goal="cleandir";;
r) goal="report";;
# none for clean
cl) goal="clone";;
l) goal="list";;
d) goal="desc";;
h) goal="help";;
esac
### REPORT ###
# no extra args required
if [ "${goal}" = "report" ]; then
report;
fi
if [ "${goal}" = "clean" ]; then
cleanreport; exit 0
fi
if [ "${goal}" = "cleandir" ]; then
cleandirs; exit 0
fi
if [ "${goal}" = "clone" ]; then
test1=`basename $2 .tpkg`
test2=`basename $3 .tpkg`;
if [ -z "$test1" -o -z "$test2" ]; then
usage; cd $TPKG_CURRENT; exit 0
fi
clone; exit 0
fi
if [ -z "${archive}" ]; then
out "[log] Defaulting to \`execute'"
archive=$1
goal="exe"
fi
if [ -z "${archive}" ]; then
usage; cd $TPKG_CURRENT; exit 0
fi
testname=`basename "${archive}" .tpkg`
export TPKG_SRCDIR=`dirname "${archive}"`
export TPKG_NAME=${testname}
test_pkg="${TPKG_SRCDIR}/${testname}.tpkg"
dsc_file=$testname.dsc
if [ -z $testname ]; then
err "[fatal] The test package should have a .tpkg extension. Abort."
cd $TPKG_CURRENT; exit 1
fi
if [ "${goal}" = "fake" ]; then
out "[log] Writing .done-$testname file."
write_fake_done; exit 0
fi
if [ $goal = "create" ]; then
### CREATE ###
# get all files with the same basename except those that ends in .tpkg
# check for shar
which tar >/dev/null
if [ $? -ne 0 ]; then
err "[fatal] Tar command not found. Abort."
cd $TPKG_CURRENT; exit 1
fi
# assume there is a dir named $testname.dir
if [ ! -d "${testname}.dir" ]; then
err "[fatal] No $testname.dir directory found. Abort."
cd $TPKG_CURRENT; exit 1
fi
cd $testname.dir
# rm unwanted files
cleanreport # this cd's to $TPKG_CURRENT
cd - >/dev/null # jump back
# tar is smart enough to handle this
cd ../
i=$( ls ${testname}.dir/$testname.* 2>/dev/null )
if [ -z "${i}" ]; then
err "[fatal] No $testname.* files found. Abort."
cd $TPKG_CURRENT; exit 1
fi
# tar --create --file $testname.tpkg --gzip ${testname}.dir
(
if [ -d "${test_pkg}" ]
then
TPKG_FORMAT="dir"
fi
if [ $TPKG_FORMAT = "dir" ]
then
if [ -e "${test_pkg}" ]
then
rm -fr "${test_pkg}"
fi
cp -pr "${testname}.dir" "${test_pkg}" \
|| cp -pr "${testname}.dir" "${test_pkg}" \
|| ( mkdir "${test_pkg}" \
&& ( ( cd "${testname}.dir" ; tar cf - . ) \
| ( cd "${test_pkg}"; tar xf -)
)
)
else
tar -cf - ${testname}.dir | gzip - > "${test_pkg}"
fi
)
if [ $? -ne 0 ]; then
err "[fatal] Archive create error. Abort."
cd $TPKG_CURRENT; exit 1
fi
if [ $TPKG_KEEP -eq 0 ]; then
out "[log] Removing member files"
rm $i
fi
if [ $TPKG_KEEP -eq 0 ]; then
out "[log] Removing directory"
rmdir $testname.dir
fi
cd $TPKG_CURRENT; exit 0
fi
### TMPL ####
# write out a .dsc and touch a .pre/.post/.test
if [ $goal = "tmpl" ]; then
if [ -f $testname.dsc ]; then
err "[fatal] $testname.dsc already exists. Abort."
cd $TPKG_CURRENT; exit 1
fi
# make tmp dir
dir="$testname.dir"
mkdir $dir
if [ ! -d $dir ]; then
err "[fatal] Failure to create a temporary working directory. Abort."
cd $TPKG_CURRENT; exit 1
fi
cd $dir
cat <<TMPL_EOF > $testname.dsc
BaseName: $testname
Version: 1.0
Description: [Put something nice here]
CreationDate: `date`
Maintainer: `grep $LOGNAME /etc/passwd | awk -F: ' { print $5 }' | sed s/,//g`
Category:
Component:
CmdDepends:
Depends:
Help: $testname.help
Pre: $testname.pre
Post: $testname.post
Test: $testname.test
AuxFiles:
Passed:
Failure:
TMPL_EOF
# .help file
echo "Please describe how to use this test." > $testname.help
echo "i.e. tpkg -a ARG exe testname:" >> $testname.help
echo " ARG is used to ..." >> $testname.help
# .test file
echo "# #-- $testname.test --#" > $testname.test
echo "# source the master var file when it's there" >> $testname.test
echo "[ -f $TPKG_VAR_MASTER ] && source $TPKG_VAR_MASTER" >> $testname.test
echo "# use $TPKG_VAR_TEST for in test variable passing" >> $testname.test
echo "[ -f $TPKG_VAR_TEST ] && source $TPKG_VAR_TEST" >> $testname.test
# .post file
echo "# #-- $testname.post --#" > $testname.post
echo "# source the master var file when it's there" >> $testname.post
echo "[ -f $TPKG_VAR_MASTER ] && source $TPKG_VAR_MASTER" >> $testname.post
echo "# source the test var file when it's there" >> $testname.post
echo "[ -f $TPKG_VAR_TEST ] && source $TPKG_VAR_TEST" >> $testname.post
echo "#" >> $testname.post
echo "# do your teardown here" >> $testname.post
# .pre file
echo "# #-- $testname.pre--#" > $testname.pre
echo "# source the master var file when it's there" >> $testname.pre
echo "[ -f $TPKG_VAR_MASTER ] && source $TPKG_VAR_MASTER" >> $testname.pre
echo "# use $TPKG_VAR_TEST for in test variable passing" >> $testname.pre
echo "[ -f $TPKG_VAR_TEST ] && source $TPKG_VAR_TEST" >> $testname.pre
out "[log] created $testname.{dsc, test, help, pre, post}"
out "[log] please create the script(s) and then run: tpkg create $testname.tpkg"
out "[log] created $testname in $dir."
cd $TPKG_CURRENT; exit 0
fi
if [ ! -e $archive ]; then
err "[fatal] Cannot find the test package: $archive. Abort."
cd $TPKG_CURRENT; exit 1
fi
## EXTRACT
if [ $goal = "extract" ]; then
dir="${testname}.dir"
if [ -d $dir ]; then
err "[fatal] Directory $dir already exists. Abort."
cd $TPKG_CURRENT; exit 1
fi
mkdir $dir
if [ ! -d $dir ]; then
err "[fatal] Failure to create $dir directory. Abort."
cd $TPKG_CURRENT; exit 1
fi
extract_tpkg_to $dir
cd $dir
trap cleanup_and_exit INT
# stop here
out "[log] extracted ${test_pkg} $dir."
cd $TPKG_CURRENT; exit 0
fi
## LIST OR DESC OR HELP
if [ $goal = "list" -o $goal = "desc" -o $goal = "help" ]; then
$0 extract "${archive}"
if [ $? -ne 0 ]; then
cd $TPKG_CURRENT; exit 1
fi
cd ${testname}.dir/
case $goal in
list*)
cat *
;;
desc*)
echo -n "$testname: "
cat $testname.dsc | awk -F': ?' '/^Description/ { print $2 }'
;;
help*)
if [ -f $testname.help ]; then
cat $testname.help
else
err "[warning] No help file found."
fi
;;
esac
cd $TPKG_CURRENT
# dir can go
rm -rf ${testname}.dir; exit 0
fi
trap cleanup_and_exit INT
# make a tmp dir during execution
if [ "$goal" != "exe" ]; then
err "[fatal] What do you mean with $goal?. Abort."
cd $TPKG_CURRENT; exit 1
fi
mktempdir "${testname}.XXXXXX"
if [ ! -d $dir ]; then
err "[fatal] Failure to create a temporary working directory. Abort."
cd $TPKG_CURRENT; exit 1
fi
## EXTRACT
extract_tpkg_to $dir
cd $dir
### EXE ###
# extract the information out of the *.dsc files
if [ ! -f $dsc_file ]; then
err "[fatal] Can't locate the description file: $dsc_file. Abort."
cleanup; exit 1
fi
SHELL=`which bash`
if [ -z ${SHELL} ]; then
SHELL=/usr/local/bin/bash
if [ ! -x $SHELL ]; then
err "[fatal] Can't find the bash shell. Abort."
cleanup; exit 1
fi
fi
# check for a .done file
if [ -f "../.done-${testname}" -a $TPKG_FORCE -ne 1 ]; then
out "[log] Found .done-${testname}. Not executing this test."
cleanup; exit 0
fi
# this is the template for .dsc files
# we need to check if all these files also exist TODO
dsc_basename=$testname
function get_field_from_dsc() # fieldname
{
grep "^$1: " $dsc_file | sed -e "s/^$1:[ ]*//" -e "s/[ ]*$//"
}
dsc_version=`get_field_from_dsc Version`
dsc_description=`get_field_from_dsc Description`
dsc_creationdate=`get_field_from_dsc CreationDate`
dsc_category=`get_field_from_dsc Category`
dsc_component=`get_field_from_dsc Component`
dsc_cmddepends=`get_field_from_dsc CmdDepends`
dsc_depends=`get_field_from_dsc Depends`
dsc_maintainer=`get_field_from_dsc Maintainer`
dsc_help=`get_field_from_dsc Help`
dsc_pre=`get_field_from_dsc Pre`
dsc_post=`get_field_from_dsc Post`
dsc_test=`get_field_from_dsc Test`
dsc_aux=`get_field_from_dsc AuxFiles`
dsc_passed=`get_field_from_dsc Passed`
dsc_failure=`get_field_from_dsc Failure`
# consistency check the lot
for i in $dsc_pre $dsc_post $dsc_test $dsc_help; do
if [ ! -z ${i} ]; then
if [ ! -f "${i}" ]; then
err "[fatal] File defined, but ${i} cannot be found. Abort."
cleanup; exit 1
fi
fi
done
for i in $dsc_pre $dsc_post $dsc_test $dsc_help; do
if [ -z ${i} ]; then
if [ -f "${i}" ]; then
err "[fatal] File not defined, but ${i} is included in the package. Abort."
cleanup; exit 1
fi
fi
done
# if we depend on another test to that one first and then return
for deptest in ${dsc_depends}; do
cd .. # go up one dir
out "[log] executing dependency test: ${TPKG_SRCDIR}/${deptest}"
${SHELL} $0 "-b ${TPKG_BASE}" exe "${TPKG_SRCDIR}/${deptest}"
test_result=$?
cd - > /dev/null # back where we belong
if [ $test_result -ne 0 ]; then
err "[fatal] Test depends on $deptest which failed. Abort."
cleanup; exit 1
fi
done
# this enhances the template from above
## Post Processing of some of these variables
# dsc_aux is a comma separated list of files, max 8 files
i=$( echo $dsc_aux | awk -F', ?' '{ print $1 "\n" $2 "\n" $3 "\n" $4 "\n" \
$5 "\n" $6 "\n" $7 "\n" $8 }' )
dsc_aux_files=($i)
dsc_aux_files_total=${#dsc_aux_files[*]}
# cmd depends
i=$( echo $dsc_cmddepends | awk -F', ?' '{ print $1 "\n" $2 "\n" $3 "\n" $4 "\n" \
$5 "\n" $6 "\n" $7 "\n" $8 }' )
dsc_cmddepends_files=($i)
dsc_cmddepends_files_total=${#dsc_cmddepends_files[*]}
for i in ${dsc_cmddepends_files[*]}; do
find_cmd $i
done
# depends can also be a comma separated list of package
# TODO
# check is the aux files are also really in the shar
for i in ${dsc_aux_files[*]}; do
if [ ! -f $i ]; then
err "[fatal] Aux. file $i must be in the archive. Abort."
cleanup; exit 1
fi
done
if [ ! -f $dsc_test ]; then
err "[fatal] Can't locate the test script: $dsc_test. Abort."
cleanup; exit 1
fi
### Actual executing of the scripts
tpkg_log "Starting test: '$dsc_basename'"
epoch # run before pre()
: > result.$dsc_basename
echo "BaseName: $dsc_basename" | write_result result.$dsc_basename
echo "Description: $dsc_description" | write_result result.$dsc_basename
echo "DateRunStart: $epoch " | write_result result.$dsc_basename
echo "--------------- Test Output ------------------" | write_result result.$dsc_basename
pre
out "[log] Executing test"
( ${SHELL} $dsc_test ${TPKG_ARGS} 2>&1 ) > result.$dsc_basename.tmp
test_result=$?
write_result result.$dsc_basename < result.$dsc_basename.tmp
rm -f result.$dsc_basename.tmp
epoch # would like to run after post, but that is not possible :-(
if [ $test_result -ne 0 ]; then
err "[warning] Test executed with errors: $test_result."
echo "!! FAILED !! !! FAILED !!" > result.$dsc_basename.tmp
echo "DateRunEnd: $epoch" >> result.$dsc_basename.tmp
err "[log] !! FAILED !!"
cat result.$dsc_basename >> result.$dsc_basename.tmp
echo "exit code: $test_result" >> result.$dsc_basename.tmp
mv result.$dsc_basename.tmp result.$dsc_basename
post;
if [ $TPKG_KEEP -eq 0 ]; then
out "[log] Removing temp directory $dir"
cleanup
else
out "[log] Keeping temp directory $dir"
fi
exit 1
fi
cp -f result.$dsc_basename result.$dsc_basename.$$
failed=-1 # -1 undef, 0 passed, 1 failed
## PASSED
[ ! -z "${dsc_passed}" ] && egrep "${dsc_passed}" result.$dsc_basename.$$ > /dev/null
if [ $? -eq 0 ]; then
err "[log] ** PASSED **"
echo "** PASSED ** ** PASSED **" > result.$dsc_basename.tmp
echo "DateRunEnd: $epoch" >> result.$dsc_basename.tmp
cat result.$dsc_basename >> result.$dsc_basename.tmp
echo "exit code: $test_result" >> result.$dsc_basename.tmp
mv result.$dsc_basename.tmp result.$dsc_basename
write_done
failed=0
fi
## FAILED
[ ! -z "${dsc_failure}" ] && egrep "${dsc_failure}" result.$dsc_basename.$$ > /dev/null
# if not found this actually means PASSED
if [ $? -eq 0 ]; then
err "[log] !! FAILED !!"
echo "!! FAILED !! !! FAILED !!" > result.$dsc_basename.tmp
echo "DateRunEnd: $epoch" >> result.$dsc_basename.tmp
cat result.$dsc_basename >> result.$dsc_basename.tmp
echo "exit code: $test_result" >> result.$dsc_basename.tmp
mv result.$dsc_basename.tmp result.$dsc_basename
failed=1
else
err "[log] ** PASSED **"
echo "** PASSED ** ** PASSED **" > result.$dsc_basename.tmp
echo "DateRunEnd: $epoch" >> result.$dsc_basename.tmp
cat result.$dsc_basename >> result.$dsc_basename.tmp
echo "exit code: $test_result" >> result.$dsc_basename.tmp
mv result.$dsc_basename.tmp result.$dsc_basename
write_done
failed=0
fi
## UNKNOWN
if [ $failed -eq -1 ]; then
# neither failed, not success, unknown
err "[log] -- UNKNOWN --"
echo "-- UNKNOWN -- -- UNKNOWN --" > result.$dsc_basename.tmp
echo "DateRunEnd: $epoch" >> result.$dsc_basename.tmp
cat result.$dsc_basename >> result.$dsc_basename.tmp
echo "exit code: $test_result" >> result.$dsc_basename.tmp
mv result.$dsc_basename.tmp result.$dsc_basename
write_done
failed=1 # not passed
fi
post
if [ $TPKG_KEEP -eq 0 ]; then
out "[log] Removing temp directory $dir"
cleanup
else
out "[log] Keeping temp directory $dir"
fi
exit $failed

331
zonemaster-ldns/ldns/test/tpkg/tpkg.1 Normal file
View File

@@ -0,0 +1,331 @@
.\" @(#)tpkg.1 0.1 12-Oct-2005 OF;
.TH tpkg 1 "12 Oct 2005"
.SH NAME
tpkg \- test package - manipulate tests
.SH SYNOPSIS
.B tpkg
[
.IR OPTIONS
]
[
.IR c|e|t|f
]
[
.IR exe|create|extract|tmpl|fake
]
test.tpkg
.B tpkg
[
.IR OPTIONS
]
[
.IR r|l|d|h|n
]
[
.IR report|clean|cleandir|list|desc|help
]
test.tpkg
.B tpkg
[
.IR OPTIONS
]
.IR clone
test1.tpkg
test2.tpkg
.SH DESCRIPTION
\fBtpkg\fR is used to manipulate test packages. A test package
is nothing more than a tar archive which holds a number of
scripts. The general idea is that by using \fBtpkg\fR you can
keep your tests organized.
.PP
All errors are send to stderr, the output of the scripts is
stored in a result. file.
.PP
When no action is given a test is \fIexecuted\fR.
.PP
When a test is executed it is first unpacked, after unpacking
the pre-script is run. If this script executes successfully
the actual test is run. The test may read the \fI.tpkg.var\fR
file to get a hold of shared variables. During the test run
a \fIresult.\fR file is written with the output of the test.
After the test the post-script is run.
.PP
All files created by \fBtpkg\fR are removed when \fBtpkg\fR \fIclean\fR is run.
.TP
\fBexe\fR -- \fBexe\fR
Execute a test. This will create a \fIresult.\fR file which looks
like this:
** PASSED ** ** PASSED **
DateRunEnd: 1000023
BaseName: svn-build
Description: Check out and build libdns/trunk
DateRunStart: 1000021
--------------- Test Output ------------------
The first line indicates if a test was failed or passed (or unknown
in some cases). The second line shows when the tests was finished
in seconds since epoch. The third line is the basename as given in the
description file.
The fourth line is the test's description, and the fifth line tells when
the test was started in seconds after the epoch.
Everything after "----- Test Output ----" is the captured output of the
test script.
.PP
Note that a test is executed it runs in a temporary directory, so anything
you reference with a relative path should have an extra '../' in front of
it.
.TP
\fBcreate\fR -- \fBc\fR
Create a tpkg file out of the test files.
.TP
\fBextract\fR -- \fBe\fR
Extract a tpkg file. A user can then edit these files and
recreate the tpkg by running \fBtpkg create\fR.
.TP
\fBtmpl\fR -- \fBt\fR
Create new test template files. A \fIdsc\fR and \fItest\fR file are created.
.TP
\fBfake\fR -- \fBf\fR
Fake the test run, but do create a \fI.done\fR, usefull when writing
tests that depend on (lenghty) other tests.
.TP
\fBlist\fR -- \fBl\fR
Print the files in the test package to standard output.
.TP
\fBreport\fR -- \fBr\fR
Create a report by gathering information from the \fIresult\fR files.
Nothing is deleted when you run \fBtpkg report\fR.
A typical example of a result. file is this:
ldns-splint: 5 s : ** PASSED ** : Run a make lint test on the ldns...
svn-build: 15 s : ** PASSED ** : Check out and build libdns/trunk
PASSED: 2 (100.00 %) FAILED: 0 (0 %) UNKNOWN: 0 (0 %)
It shows the name of the test, the time it took while executing (in
seconds), whether it passed, failed or was unknown and the description of
the test.
.PP
Essentially \fBreport\fR takes a bunch a \fIresult.\fR files and creates a
nice test report of it. When the -q switch is given only failed tests are
shown.
.TP
\fBclean\fR -- \fBclean\fR
Clean up all auxiliary files, that is remove: \fI.done*\fR, \fIresult*\fR and
the \fI.tpkg.var\fR file.
.TP
\fBcleandir\fR -- \fBcd\fR
Clean up all testname.\fBdir\fR directories.
.TP
\fBdesc\fR -- \fBd\fR
Print the description of the test to standard output.
.TP
\fBhelp\fR -- \fBh\fR
If a .help file is found its contents are displayed. This file can be
used to document some test specific things, like how to call it or
describe its arguments (if any). Together with the Description line this
should fully document the test.
.TP
\fBclone\fR -- \fBcl\fR
Clone \fItest1.tpkg\fR to \fItest2.tpkg\fR. Care is taken to also rename
the member files and to update the \fIdsc\fR file. You should always check
the new test.
.SH OPTIONS
.TP
\fB\-h\fR
Show a short help message.
.TP
\fB\-v\fR
Show the version of \fItpkg\fR.
.TP
\fB\-q\fR
Quiet, only print errors to stdout.
.TP
\fB\-l\fR
Log the name of the test to syslog when starting the test.
.TP
\fB\-p\fR \fIPRI\fR
Use \fIPRI\fR as priority, for example: daemon.notice.
.TP
\fB\-k\fR
Do not remove the test directory when doing a \fItpkg\fR \fBcreate\fR
or \fItpkg\fR \fBexe\fR.
.TP
\fB\-d\fR \fIDIR\fR
Use \fIDIR\fR as a basedirectory. This is also the place where files
like \fI.tpkg.var\fR will be kept and where the \fIresult.\fR files are written.
.TP
\fB-a\fR \fIARGS\fR
Pass the unmodified string \fIARGS\fR through to the test being run.
This way options and/or arguments can be given to the test. The pre,
test and post script will all be given this string as an argument.
.TP
\fB-n\fR \fINUM\fR
When running \fItpkg report\fR, \fItpkg\fR will return with
a non-zero exit code if less than NUM of the tests passed.
Otherwise it will return with zero.
.SH FORMAT OF THE .DSC FILE
The most important file in each test is the description
file, the one with the .dsc extension. The file is line based,
so options spanning multiple lines are not recognized and worse,
they probably make tpkg fail in unexpected ways. Also note that
currently now consistency checking is done. Non existing files listed under
AuxFiles: are not detected for instance.
.PP
A typical example looks like this:
\fBBaseName:\fR test1
\fBVersion:\fR 0.1
\fBDescription:\fR Test negative SOA TTL, added to test BUG #6
\fBCreationDate:\fR 05-10-2005
\fBMaintainer:\fR Miek Gieben
\fBCategory:\fR
\fBComponent:\fR
\fBCmdDepends:\fR
\fBDepends:\fR
\fBHelp:\fR test1.help
\fBPre:\fR test1.pre
\fBPost:\fR test1.post
\fBTest:\fR test1.test
\fBAuxFiles:\fRtest1.a, test2.b
\fBPassed:\fR
\fBFailure:\fR
.PP
Details of each line follow:
.TP
\fBBaseName\fR
The basename of the test. This is the name of the .dsc
file without the .dsc extension
.TP
\fBVersion\fR
Currently nothing is done with this. It could either
evolve in a version for the .dsc file or it could
be used to version your tests.
.TP
\fBDescription\fR
A short explanation of what this test is supposed to test.
'tpkg report' uses this. \fBtpkg\fR uses ':' (colon) as a
delimiter, so it cannot be used in the description of a test.
.TP
\fBCreationDate \fR
Automatically set by 'tpkg tpml'. The original creation date
for this test.
.TP
\fBMaintainer\fR
Who created/maintains this test.
.TP
\fBCategory\fR
Under what category does this test fall. This is not
used by tpkg, but is a hint to the users of the tests.
This could something like: 'building', 'running', etc.
.TP
\fBComponent\fR
What software component are you testing, this could be
the name of the executable. User decides what to put
here, but it should be consisted for all tests.
.TP
\fBCmdDepends\fR
Depend on these commands. If the command cannot be found ($PATH
is search), the test is aborted. Usefull to check to the environment of a
test.
.TP
\fBDepends\fR
On what other test does this test depend. The full
package name should be given, with the .tpkg extension.
Multiple tests can be specified, with whitespace between.
.TP
\fBHelp\fR
Name of a file that has a few lines of usefull information
to the user of the test. The file must have a .help suffix.
.TP
\fBPre\fR
Name of a script that should be executed before the
test is run. If the pre-script fails the test fails.
.TP
\fBPost\fR
Name of a script that should be executed after the
test has run. If the post-script fails the test fails.
.TP
\fBTest\fR
Name of the main test script.
.TP
\fBAuxfiles\fR
Other files that are needed to run this test, i.e. to
compare against. Currently this list is limited to 8
other files. The list should use commas as delimiters:
fileA, fileB, FileC, ...
.TP
\fBPassed\fR
A regular expression that is matched against the output
of the main test script. If the expression matches the
test is a success.
.TP
\fBFailure\fR
A regular expression that is matched against the output
of the main test script. If the expression matches the
test is a failure.
.PP
All files used in one test must have the same basename, otherwise 'tpkg create'
will not pick them up.
.SH FAILED OR PASSED
.PP
A test is \fIfailed\fR when:
.TP
o
The test script returns with a non-zero value.
.TP
o
The test script returns with zero, and the 'Failure' regexp matches.
.PP
A test is \fIpassed\fR when:
.TP
o
The test script returns with zero.
.TP
o
The test script returns with zero, and the 'Passed' regexp matches.
.PP
A test is \fIneither\fR failed nor passed when:
.TP
o
Test test script returns with zero, and neither the 'Passed' nor 'Failure'
regular expressions matched.
If this happens you should rewrite your test.
.SH AUTHOR
Written by Miek Gieben, NLnet Labs.
.SH REPORTING BUGS
Report bugs to <miek@nlnetlabs.nl>
.SH BUGS
As of version 1.03 the internal consistency of a package is tested whenever
a test is executed.
.SH COPYRIGHT
Copyright (C) 2005, 2006 NLnet Labs. This is free software. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Licensed under the GPL version 2.
.SH SEE ALSO
\fBREADME\fR for information about how to actually write tests.