tests/shell: add "--quick" option to skip slow tests (via NFT_TEST_SKIP_slow=y)

It's important to run (a part) of the tests in a timely manner.
Add an option to skip long running tests.

Thereby, add a more general NFT_TEST_SKIP_* mechanism.

This is related and inverse from "NFT_TEST_HAVE_json", where a test
can require [ "$NFT_TEST_HAVE_json" != n ] to run, but is skipped when
[ "$NFT_TEST_SKIP_slow" = y ].

Currently only NFT_TEST_SKIP_slow is supported. The user can set such
environment variables (or use the -Q|--quick command line option). The
configuration is printed in the test info.

Tests should check for [ "$NFT_TEST_SKIP_slow" = y ] so that the
variable has to be explicitly set to opt-out. For convenience, tests can
also add a

    # NFT_TEST_SKIP(NFT_TEST_SKIP_slow)

tag, which is evaluated by test-wrapper.sh. Or they can run a quick, reduced
part of the test, but then should still indicate to be skipped.

Mark 8 tests are as slow, that take longer than 5 seconds on my machine.
With this, a parallel wall time for the non-slow tests is only 7 seconds
(on my machine).

The ultimate point is to integrate a call to "tests/shell/run-tests.sh"
in a `make check` target. For development, you can then export
NFT_TEST_SKIP_slow=y and have a fast `make check`.

Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>

1 files changed, 28 insertions, 12 deletions

diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh
index ffd0fb3f..cd8f4805 100755
--- a/tests/shell/helpers/test-wrapper.sh
+++ b/tests/shell/helpers/test-wrapper.sh
@@ -54,23 +54,39 @@ TEST_TAGS_PARSED=0
 ensure_TEST_TAGS() {
 	if [ "$TEST_TAGS_PARSED" = 0 ] ; then
 		TEST_TAGS_PARSED=1
-		TEST_TAGS=( $(sed -n '1,10 { s/^.*\<\(NFT_TEST_REQUIRES\)\>\s*(\s*\(NFT_TEST_HAVE_[a-zA-Z0-9_]\+\)\s*).*$/\1(\2)/p }' "$1" 2>/dev/null || : ) )
+		TEST_TAGS=( $(sed -n '1,10 { s/^.*\<\(NFT_TEST_REQUIRES\|NFT_TEST_SKIP\)\>\s*(\s*\(NFT_TEST_SKIP_[a-zA-Z0-9_]\+\|NFT_TEST_HAVE_[a-zA-Z0-9_]\+\)\s*).*$/\1(\2)/p }' "$1" 2>/dev/null || : ) )
 	fi
 }
 
 rc_test=0
 
-for KEY in $(compgen -v | grep '^NFT_TEST_HAVE_') ; do
-	if [ "${!KEY}" != n ]; then
-		continue
-	fi
-	ensure_TEST_TAGS "$TEST"
-	if array_contains "NFT_TEST_REQUIRES($KEY)" "${TEST_TAGS[@]}" ; then
-		echo "Test skipped due to $KEY=n (test has \"NFT_TEST_REQUIRES($KEY)\" tag)" >> "$NFT_TEST_TESTTMPDIR/testout.log"
-		rc_test=77
-		break
-	fi
-done
+if [ "$rc_test" -eq 0 ] ; then
+	for KEY in $(compgen -v | grep '^NFT_TEST_HAVE_') ; do
+		if [ "${!KEY}" != n ]; then
+			continue
+		fi
+		ensure_TEST_TAGS "$TEST"
+		if array_contains "NFT_TEST_REQUIRES($KEY)" "${TEST_TAGS[@]}" ; then
+			echo "Test skipped due to $KEY=n (test has \"NFT_TEST_REQUIRES($KEY)\" tag)" >> "$NFT_TEST_TESTTMPDIR/testout.log"
+			rc_test=77
+			break
+		fi
+	done
+fi
+
+if [ "$rc_test" -eq 0 ] ; then
+	for KEY in $(compgen -v | grep '^NFT_TEST_SKIP_') ; do
+		if [ "${!KEY}" != y ]; then
+			continue
+		fi
+		ensure_TEST_TAGS "$TEST"
+		if array_contains "NFT_TEST_SKIP($KEY)" "${TEST_TAGS[@]}" ; then
+			echo "Test skipped due to $KEY=y (test has \"NFT_TEST_SKIP($KEY)\" tag)" >> "$NFT_TEST_TESTTMPDIR/testout.log"
+			rc_test=77
+			break
+		fi
+	done
+fi
 
 if [ "$rc_test" -eq 0 ] ; then
 	"$TEST" &>> "$NFT_TEST_TESTTMPDIR/testout.log" || rc_test=$?