tests/shell: skip tests if nft does not support JSON mode

We can build nft without JSON support, and some tests will fail without
it. Instead, they should be skipped. Also note, that the test accepts any
nft binary via the "NFT" environment variable. So it's not enough to
make the skipping dependent on build configuration, but on the currently
used $NFT variable.

Let "run-test.sh" detect and export a "NFT_TEST_HAVE_json=y|n" variable. This
is heavily inspired by Florian's feature probing patches.

Tests that require JSON can check that variable, and skip. Note that
they check in the form of [ "$NFT_TEST_HAVE_json" != n ], so the test is
only skipped, if we explicitly detect lack of support. That is, don't
check via [ "$NFT_TEST_HAVE_json" = y ].

Some of the tests still run parts of the tests that don't require JSON.
Only towards the end of such partial run, mark the test as skipped.

Some tests require JSON support throughout. For those, add  a mechanism
where tests can add a tag (in their first 10 lines):

  # NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)

This will be checked by "test-wrapper.sh", which will skip the test.
The purpose of this is to make it low-effort to skip a test and to print
the reason in the text output as

    Test skipped due to NFT_TEST_HAVE_json=n (test has "NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)" tag)

This is intentionally not shortened to NFT_TEST_REQUIRES(json), so that
we can grep for NFT_TEST_HAVE_json to find all relevant places.

Note that while NFT_TEST_HAVE_json is autodetected, the caller can also
force it by setting the environment variable. This allows to see what
would happen to such a test.

Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>

1 files changed, 36 insertions, 2 deletions

diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh
index d57108c0..51d285e3 100755
--- a/tests/shell/run-tests.sh
+++ b/tests/shell/run-tests.sh
@@ -11,6 +11,16 @@ if [[ -t 1 && -z "$NO_COLOR" ]] ; then
 	RESET=$'\e[0m'
 fi
 
+array_contains() {
+	local needle="$1"
+	local a
+	shift
+	for a; do
+		[ "$a" = "$needle" ] && return 0
+	done
+	return 1
+}
+
 _msg() {
 	local level="$1"
 	shift
@@ -160,6 +170,10 @@ usage() {
 	echo "                 kernel modules)."
 	echo "                 Parallel jobs requires unshare and are disabled with NFT_TEST_UNSHARE_CMD=\"\"."
 	echo " TMPDIR=<PATH> : select a different base directory for the result data."
+	echo
+	echo " NFT_TEST_HAVE_<FEATURE>=*|y: Some tests requires certain features or will be skipped."
+	echo "                 The features are autodetected, but you can force it by setting the variable."
+	echo "                 Supported <FEATURE>s are: ${_HAVE_OPTS[@]}."
 }
 
 NFT_TEST_BASEDIR="$(dirname "$0")"
@@ -167,6 +181,13 @@ NFT_TEST_BASEDIR="$(dirname "$0")"
 # Export the base directory. It may be used by tests.
 export NFT_TEST_BASEDIR
 
+_HAVE_OPTS=( json )
+for KEY in $(compgen -v | grep '^NFT_TEST_HAVE_' | sort) ; do
+	if ! array_contains "${KEY#NFT_TEST_HAVE_}" "${_HAVE_OPTS[@]}" ; then
+		unset "$KEY"
+	fi
+done
+
 _NFT_TEST_JOBS_DEFAULT="$(nproc)"
 [ "$_NFT_TEST_JOBS_DEFAULT" -gt 0 ] 2>/dev/null || _NFT_TEST_JOBS_DEFAULT=1
 _NFT_TEST_JOBS_DEFAULT="$(( _NFT_TEST_JOBS_DEFAULT + (_NFT_TEST_JOBS_DEFAULT + 1) / 2 ))"
@@ -367,6 +388,16 @@ if [ ${ret} -eq 126 ] || [ ${ret} -eq 127 ]; then
 	msg_error "cannot execute nft command: $NFT"
 fi
 
+NFT_REAL="${NFT_REAL-$NFT}"
+
+if [ -z "${NFT_TEST_HAVE_json+x}" ] ; then
+	NFT_TEST_HAVE_json=y
+	$NFT_TEST_UNSHARE_CMD "$NFT_REAL" -j list ruleset &>/dev/null || NFT_TEST_HAVE_json=n
+else
+	NFT_TEST_HAVE_json="$(bool_n "$NFT_TEST_HAVE_json")"
+fi
+export NFT_TEST_HAVE_json
+
 if [ "$NFT_TEST_JOBS" -eq 0 ] ; then
 	MODPROBE="$(which modprobe)"
 	if [ ! -x "$MODPROBE" ] ; then
@@ -392,8 +423,6 @@ chmod 755 "$NFT_TEST_TMPDIR"
 
 exec &> >(tee "$NFT_TEST_TMPDIR/test.log")
 
-NFT_REAL="${NFT_REAL-$NFT}"
-
 msg_info "conf: NFT=$(printf '%q' "$NFT")"
 msg_info "conf: NFT_REAL=$(printf '%q' "$NFT_REAL")"
 msg_info "conf: VERBOSE=$(printf '%q' "$VERBOSE")"
@@ -408,6 +437,11 @@ msg_info "conf: NFT_TEST_HAS_UNSHARED_MOUNT=$(printf '%q' "$NFT_TEST_HAS_UNSHARE
 msg_info "conf: NFT_TEST_KEEP_LOGS=$(printf '%q' "$NFT_TEST_KEEP_LOGS")"
 msg_info "conf: NFT_TEST_JOBS=$NFT_TEST_JOBS"
 msg_info "conf: TMPDIR=$(printf '%q' "$_TMPDIR")"
+echo
+for KEY in $(compgen -v | grep '^NFT_TEST_HAVE_' | sort) ; do
+	msg_info "conf: $KEY=$(printf '%q' "${!KEY}")"
+	export "$KEY"
+done
 
 NFT_TEST_LATEST="$_TMPDIR/nft-test.latest.$USER"