tests/shell: skip tests if nft does not support JSON mode

We can build nft without JSON support, and some tests will fail without
it. Instead, they should be skipped. Also note, that the test accepts any
nft binary via the "NFT" environment variable. So it's not enough to
make the skipping dependent on build configuration, but on the currently
used $NFT variable.

Let "run-test.sh" detect and export a "NFT_TEST_HAVE_json=y|n" variable. This
is heavily inspired by Florian's feature probing patches.

Tests that require JSON can check that variable, and skip. Note that
they check in the form of [ "$NFT_TEST_HAVE_json" != n ], so the test is
only skipped, if we explicitly detect lack of support. That is, don't
check via [ "$NFT_TEST_HAVE_json" = y ].

Some of the tests still run parts of the tests that don't require JSON.
Only towards the end of such partial run, mark the test as skipped.

Some tests require JSON support throughout. For those, add  a mechanism
where tests can add a tag (in their first 10 lines):

  # NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)

This will be checked by "test-wrapper.sh", which will skip the test.
The purpose of this is to make it low-effort to skip a test and to print
the reason in the text output as

    Test skipped due to NFT_TEST_HAVE_json=n (test has "NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)" tag)

This is intentionally not shortened to NFT_TEST_REQUIRES(json), so that
we can grep for NFT_TEST_HAVE_json to find all relevant places.

Note that while NFT_TEST_HAVE_json is autodetected, the caller can also
force it by setting the environment variable. This allows to see what
would happen to such a test.

Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>

7 files changed, 20 insertions, 1 deletions

diff --git a/tests/shell/testcases/json/0001set_statements_0 b/tests/shell/testcases/json/0001set_statements_0
index 1c72d35b..fc4941f4 100755
--- a/tests/shell/testcases/json/0001set_statements_0
+++ b/tests/shell/testcases/json/0001set_statements_0
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/0002table_map_0 b/tests/shell/testcases/json/0002table_map_0
index 4b54527b..b375e996 100755
--- a/tests/shell/testcases/json/0002table_map_0
+++ b/tests/shell/testcases/json/0002table_map_0
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/0003json_schema_version_0 b/tests/shell/testcases/json/0003json_schema_version_0
index 0ccf94c8..43f387a1 100755
--- a/tests/shell/testcases/json/0003json_schema_version_0
+++ b/tests/shell/testcases/json/0003json_schema_version_0
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/0004json_schema_version_1 b/tests/shell/testcases/json/0004json_schema_version_1
index bc451ae7..0f8d586f 100755
--- a/tests/shell/testcases/json/0004json_schema_version_1
+++ b/tests/shell/testcases/json/0004json_schema_version_1
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/0005secmark_objref_0 b/tests/shell/testcases/json/0005secmark_objref_0
index ae967435..992d1b00 100755
--- a/tests/shell/testcases/json/0005secmark_objref_0
+++ b/tests/shell/testcases/json/0005secmark_objref_0
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/0006obj_comment_0 b/tests/shell/testcases/json/0006obj_comment_0
index 76d8fe16..4c2a0e8c 100755
--- a/tests/shell/testcases/json/0006obj_comment_0
+++ b/tests/shell/testcases/json/0006obj_comment_0
@@ -1,5 +1,7 @@
 #!/bin/bash
 
+# NFT_TEST_REQUIRES(NFT_TEST_HAVE_json)
+
 set -e
 
 $NFT flush ruleset
diff --git a/tests/shell/testcases/json/netdev b/tests/shell/testcases/json/netdev
index 9f603381..dad7afcd 100755
--- a/tests/shell/testcases/json/netdev
+++ b/tests/shell/testcases/json/netdev
@@ -16,4 +16,11 @@ $NFT flush ruleset
 
 RULESET='{"nftables":[{"flush":{"ruleset":null}},{"add":{"table":{"family":"netdev","name":"test_table"}}},{"add":{"chain":{"family":"netdev","table":"test_table","name":"test_chain","type":"filter","hook":"ingress","prio":0,"dev":"d0","policy":"accept"}}}]}'
 
-$NFT -j -f - <<< $RULESET
+if [ "$NFT_TEST_HAVE_json" != n ]; then
+	$NFT -j -f - <<< $RULESET
+fi
+
+if [ "$NFT_TEST_HAVE_json" = n ]; then
+	echo "Test partially skipped due to missing JSON support."
+	exit 77
+fi