test: Streamline test suite a little

This change aims to streamline the integrationtest suite and reduce friction when running (parts of) test suites.

Changes:
- add `test-testname` to makefile to easily run one suite
- made skipping tests first class citizen in test.sh (all available testsuites and the reasons they are skipped are now better exposed and discovered)
- suppress some output of gotest unless there is an error
- no longer build binary for examples and gotest suites
- removed .SILENT from makefile as it being applied to only some targets makes it feel weird (I just learned about this option btw, feel free to comment on this change)
- move individual tests out of `test.sh` and into `test-misc.sh`
- introduced the concept of testsuites to `test.sh`
This commit is contained in:
Johan Bloemberg
2018-02-15 17:17:25 +01:00
parent 8b9491823d
commit b34b359860
21 changed files with 184 additions and 63 deletions

View File

@@ -7,29 +7,31 @@ ROOT=$(dirname "${BASH_SOURCE}")/..
cd "${ROOT}"
. test/util.sh
tmpdir="`$mktemp --tmpdir -d tmp.XXX`" # get a dir outside of the main package
log="$tmpdir/$(basename $0 .sh).log"
failures=''
function run-test()
{
$@ || failures=$( [ -n "$failures" ] && echo "$failures\\n$@" || echo "$@" )
}
make build
base=$(go list .)
for pkg in `go list ./... | grep -v "^${base}/vendor/" | grep -v "^${base}/examples/" | grep -v "^${base}/test/" | grep -v "^${base}/old/" | grep -v "^${base}/tmp/"`; do
echo "Testing: $pkg"
echo -e "\ttesting: $pkg"
# FIXME: can we capture and output the stderr from these tests too?
run-test go test "$pkg"
run-test go test "$pkg" > "$log"
if [ "$1" = "--race" ]; then
run-test go test -race "$pkg"
run-test go test -race "$pkg" > "$log"
fi
done
make clean
if [[ -n "$failures" ]]; then
echo 'FAIL'
cat "$log"
echo 'The following `go test` runs have failed:'
echo -e "$failures"
echo
exit 1
fi
echo 'PASS'