summaryrefslogtreecommitdiffstats
path: root/tests/shell
Commit message (Collapse)AuthorAgeFilesLines
...
* tests: shell: fix spurious errors in sets/0036add_set_element_expiration_0Pablo Neira Ayuso2023-09-271-3/+7
| | | | | | | | | | | | | | | A number of changes to fix spurious errors: - Add seconds as expiration, otherwise 14m59 reports 14m in minute granularity, this ensures suficient time in a very slow environment with debugging instrumentation. - Provide expected output. - Update sed regular expression to make 'ms' optional and use -E mode. Fixes: adf38fd84257 ("tests: shell: use minutes granularity in sets/0036add_set_element_expiration_0") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: shell: features: Fix table owner flag checkPhil Sutter2023-09-261-1/+1
| | | | | | | | | | | The keyword is "flags", not "flag". Resulted in a false-negative: features/table_flag_owner.nft:4:2-5: Error: syntax error, unexpected string flag owner; ^^^^ Fixes: 10373f0936cd3 ("tests: shell: skip flowtable-uaf if we lack table owner support") Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: shell: skip flowtable-uaf if we lack table owner supportFlorian Westphal2023-09-222-0/+7
| | | | Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: shell: add feature probe for sctp chunk matchingFlorian Westphal2023-09-212-10/+23
| | | | | | Skip the relavant parts of the test if nft_exthdr lacks sctp support. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: shell: add feature probe for sets with more than one elementFlorian Westphal2023-09-215-0/+23
| | | | | | | | | | Kernels < 5.11 can handle only one expression per element, e.g. its possible to attach a counter per key, or a rate limiter, or a quota, but not two at the same time. Add a probe file and skip the relevant tests if the feature is absent. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: shell: skip adding catchall elements if unuspportedFlorian Westphal2023-09-211-2/+6
| | | | | | | | | | The test fails on kernels without catchall support, so elide this small part. No need to skip the test in this case, the dump file validates that the added elements are no longer there after the timeout. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: honor NFT_TEST_FAIL_ON_SKIP variable to fail on any skipped testsThomas Haller2023-09-211-1/+7
| | | | | | | | | | | | | | | | | | The test suite should pass with various kernels and build configurations. Of course, that means, that some tests will be gracefully skipped, and we don't treat that as an overall failure. However, it should be possible to run a specific kernel (net-next?) and build configuration, where we expect that all tests pass. Add an option to fail the run, if any tests were skipped. This is to ensure that we don't have broken tests that never pass. This will make more sense with automated CI is running, to enable on a test system and ensure that at least on that system, all tests pass. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* limit: display default burst when listing rulesetPablo Neira Ayuso2023-09-206-8/+8
| | | | | | | | | | | | Default burst for limit is 5 for historical reasons but it is not displayed when listing the ruleset. Update listing to display the default burst to disambiguate. man nft(8) has been recently updated to document this, no action in this front is therefore required. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/shell: run `nft --check` on persisted dump filesThomas Haller2023-09-192-1/+34
| | | | | | | | | | | | | | | | "nft --check" will trigger a rollback in kernel. The existing dump files might hit new code paths. Take the opportunity to call the command on the existing files. And alternative would be to write a separate tests, that iterates over all files. However, then we can only run all the commands sequentially (unless we do something smart). That might be slower than the opportunity to run the checks in parallel. More importantly, it would be nice if the check for the dump file is clearly tied to the file's test. So run it right after the test, from the test wrapper. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* evaluate: perform mark datatype compatibility check from mapsPablo Neira Ayuso2023-09-192-0/+64
| | | | | | | | | | | | | Wrap datatype compatibility check into a helper function and use it for map evaluation, otherwise the following bogus error message is displayed: Error: datatype mismatch, map expects packet mark, mapping expression has type integer Add unit tests to improve coverage for this usecase. Fixes: 5d8e33ddb112 ("evaluate: relax type-checking for integer arguments in mark statements") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: expand sets and maps before evaluationPablo Neira Ayuso2023-09-194-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 3975430b12d9 ("src: expand table command before evaluation") moved ruleset expansion before evaluation, except for sets and maps. For sets and maps there is still a post_expand() phase. This patch moves sets and map expansion to allocate an independent CMD_OBJ_SETELEMS command to add elements to named set and maps which is evaluated, this consolidates the ruleset expansion to happen always before the evaluation step for all objects, except for anonymous sets and maps. This approach avoids an interference with the set interval code which detects overlaps and merges of adjacents ranges. This set interval routine uses set->init to maintain a cache of existing elements. Then, the post_expand() phase incorrectly expands set->init cache and it triggers a bogus ENOENT errors due to incorrect bytecode (placing element addition before set creation) in combination with user declared sets using the flat syntax notation. Since the evaluation step (coming after the expansion) creates implicit/anonymous sets and maps, those are not expanded anymore. These anonymous sets still need to be evaluated from set_evaluate() path and the netlink bytecode generation path, ie. do_add_set(), needs to deal with anonymous sets. Note that, for named sets, do_add_set() does not use set->init. Such content is part of the existing cache, and the CMD_OBJ_SETELEMS command is responsible for adding elements to named sets. Fixes: 3975430b12d9 ("src: expand table command before evaluation") Reported-by: Jann Haber <jannh@selfnet.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/shell: simplify collecting error result in "test-wrapper.sh"Thomas Haller2023-09-181-8/+8
| | | | | | | | | | | | | | | | | | | The previous pattern was unnecessarily confusing. The "$rc_{dump,valgrind,tainted}" variable should only remember whether that particular check failed, not the overall exit code of the test wrapper. Otherwise, if you want to know in which case the wrapper exits with code 122, you have to oddly follow the rc_valgrind variable. This change will make more sense, when we add another such variable, but which will be assigned the non-zero value at multiple places. Assigning there the exit code of the wrapper, duplicates the places where the condition maps to the exit code. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: colorize NFT_TEST_HAS_SOCKET_LIMITSThomas Haller2023-09-181-11/+18
| | | | | | | | | | | | | | NFT_TEST_HAS_SOCKET_LIMITS= is similar to NFT_TEST_HAVE_* variables and indicates a feature (or lack thereof), except that it's inverted. Maybe this should be consolidated, however, NFT_TEST_HAS_SOCKET_LIMITS= is detected in the root namespace, unlike the shell scripts from features. So it's unclear how to consolidate them best. Anyway. Still highlight a lack of the capability, as it can cause tests to be skipped and we should see that easily. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: don't show the exit status for failed testsThomas Haller2023-09-181-6/+3
| | | | | | | | | | | | | | | | | | | | Previously, for failed tests we would print the exit code W: [FAILED] 2/2 tests/shell/testcases/listing/0013objects_0: got 1 This doesn't seem very useful. For one, we have special exit codes like 0 (OK), 77 (SKIPPED), 124 (DUMP FAIL), 123 (TAINTED), 122 (VALGRIND). Any other exit code is just an arbitrary failure. We don't define any special codes, and printing them is not useful. Note that further exit codes (118 - 121) are reserved, and could be special purposed, when there is a use. You can find the real exit code from the test in the result data in the "rc-failed" file. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: set C locale in "run-tests.sh"Thomas Haller2023-09-181-2/+6
| | | | | | | | | | The tests should run always the same, regardless of the user's language settings. Set LANG=C and LC_ALL=C and unset LANGUAGE. If some part wants to test a different language, it would set it explicitly. They anyway wouldn't want to depend on something from the user's environment. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: fix preserving ruleset diff after testThomas Haller2023-09-181-0/+1
| | | | | | | | | We want to delete the file in the case when there was no diff (and we expect the file to be empty). The condition was wrong. Fixes: 55fe071cd193 ('tests/shell: cleanup result handling in "test-wrapper.sh"') Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: check diff in "maps/typeof_maps_0" and "sets/typeof_sets_0" testThomas Haller2023-09-182-31/+177
| | | | | | | | | | | These tests run different variants based on NFT_TEST_HAVE_osf support. Consequently, we cannot check the pre-generated diff. Instead, construct what we expect dynamically in the script, and compare the ruleset against that. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: implement NFT_TEST_HAVE_json feature detection as scriptThomas Haller2023-09-182-14/+10
| | | | | | | | No more need to special case the "run a script" approach for detecting the json feature. Use the new mechanism instead. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: skip reset tests if kernel lacks supportFlorian Westphal2023-09-185-4/+43
| | | | | | | | | | | | reset is implemented via flush + extra attribute, so older kernels perform a flush. This means .nft doesn't work, we need to check if the individual set contents/sets are still in place post-reset. Make this generic and permit use of feat.sh in addition to the simpler foo.nft feature files. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip test cases if ct expectation and/or timeout lacks supportFlorian Westphal2023-09-185-39/+33
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip test cases involving osf match if kernel lacks supportFlorian Westphal2023-09-183-10/+39
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip catchall tests if kernel lacks supportFlorian Westphal2023-09-186-2/+35
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip destroy tests if kernel lacks supportFlorian Westphal2023-09-186-0/+13
| | | | | | | | Destroy support was added for table/flowtable/chain etc. in a single commit, so no need to add capability tests for each destroy subtype. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip inet ingress tests if kernel lacks supportFlorian Westphal2023-09-184-7/+18
| | | | | | | Split the bridge autoremove test to a new file. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip some tests if kernel lacks netdev egress supportFlorian Westphal2023-09-183-1/+18
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip bitshift tests if kernel lacks supportFlorian Westphal2023-09-1811-0/+27
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip inner matching tests if unsupportedFlorian Westphal2023-09-182-0/+9
| | | | | Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip map query if kernel lacks supportFlorian Westphal2023-09-182-8/+38
| | | | | | | | | | | | | | On recent kernels one can perform a lookup in a map without a destination register (i.e., treat the map like a set -- pure existence check). Add a feature probe and work around the missing feature in typeof_maps_add_delete: do the test with a simplified ruleset, Indicate skipped even though a reduced test was run (earlier errors cause a failure) to not trigger dump validation error. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: skip netdev_chain_0 if kernel requires netdev deviceFlorian Westphal2023-09-182-0/+9
| | | | | | | | | | | | | | This test case only works on kernel 6.4+. Add feature probe for this and tag the test accordingly using the scheme added by Thomas Haller in "tests/shell: skip tests if nft does not support JSON mode" so that run-test.sh skips it if kernel requires a device. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: add and use chain binding feature probeFlorian Westphal2023-09-184-7/+62
| | | | | | | | | | Alter 30s-stress to suppress anon chains when its unuspported. Note that 30s-stress is optionally be run standalone, so also update the test script. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: cleanup creating dummy interfaces in testsThomas Haller2023-09-186-31/+32
| | | | | | | | | | | | | | | | | | | | | | | In "tests/shell/testcases/chains/netdev_chain_0", calling "trap ... EXIT" multiple times does not work. Fix it, by calling one cleanup function. Note that we run in separate namespaces, so the cleanup is usually not necessary. Still do it, we might want to run without unshare (via NFT_TEST_UNSHARE_CMD=""). Without unshare, it's important that the cleanup always works. In practice it might not, for example, "trap ... EXIT" does not run for SIGTERM. A leaked interface might break the follow up test and tests interfere with each other. Try to workaround that by first trying to delete the interface. Also failures to create the interfaces are not considered fatal. I don't understand under what circumstances this might fail, note that there are other tests that create dummy interface and don't "exit 77" on failure. We want to know when something odd is going on. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: suggest 4Mb /proc/sys/net/core/{wmem_max,rmem_max} for rootlessThomas Haller2023-09-181-3/+3
| | | | | | | | | | | | | | | | 2Mb was not enough to pass "tests/shell/testcases/sets/0030add_many_elements_interval_0" in an unprivileged/rootless namespace. Instead, bump the suggestion to 4Mb, which lets the test pass. Note that the 4Mb are only the recommended value when running the test as rootless, and is used to autodetect NFT_TEST_HAS_SOCKET_LIMITS=y. You can set whatever values are suitable for your environment, and explicitly indicate whether the limits are appropriate or not via NFT_TEST_HAS_SOCKET_LIMITS=n|y. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: shell: add sample ruleset reproducerFlorian Westphal2023-09-182-0/+501
| | | | | | | | | | | | Changes on kernel side no longer permit transactions that reference a chain after it is bound. This test case breaks when run with nftables 1.0.6 and earlier. Keep this as a test case in tree to catch any future problems in this area. Link: https://lore.kernel.org/netfilter-devel/20230911213750.5B4B663206F5@dd20004.kasserver.com/ Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: colorize NFT_TEST_SKIP_/NFT_TEST_HAVE_ in test outputThomas Haller2023-09-161-2/+10
| | | | | | | | | Having a "SKIP" option as "y" or a "HAVE" option as "n", is note worthy because tests may be skipped based on that. Colorize, to make it easier to see in the test output. Signed-off-by: Thomas Haller <thaller@redhat.com>
* tests/shell: add feature probing via "features/*.nft" filesThomas Haller2023-09-161-0/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running selftests on older kernels makes some of them fail very early because some tests use features that are not available on older kernels, e.g. -stable releases. Known examples: - inner header matching - anonymous chains - elem delete from packet path Also, some test cases might fail because a feature isn't compiled in, such as netdev chains. This adds a feature-probing mechanism to shell tests. Simply drop a 'nft -f' compatible file with a .nft suffix into "tests/shell/features". "run-tests.sh" will load it via `nft --check` and will export NFT_TEST_HAVE_${feature}=y|n Here ${feature} is the basename of the .nft file without file extension. It must be all lower-case. This extends the existing NFT_TEST_HAVE_json= feature detection. Similarly, NFT_TEST_REQUIRES(NFT_TEST_HAVE_*) tags work to easily skip a test. The test script that cannot fully work without the feature should either skip the test entirely (NFT_TEST_REQUIRES(NFT_TEST_HAVE_*)), or run a reduced/modified test. If a modified test was run and passes, it is still a good idea to mark the overall result as skipped (exit 77) instead of claiming success to the modified test. We want to know when not the full test was running, while we want to test as much as we can. This patch is based on Florian's feature probing patch. Originally-by: Florian Westphal <fw@strlen.de> Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: shell: fix dump validation messageFlorian Westphal2023-09-162-13/+0
| | | | | | | | | | | | This test output depends on CONFIG_HZ: - update @y { ip saddr timeout 1d2h3m4s8ms } + update @y { ip saddr timeout 1d2h3m4s10ms } The dump record is with HZ=1000, on HZ=250 we get failure. Remove the dump file for now. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: honor CLICOLOR_FORCE to force coloring in run-tests.shThomas Haller2023-09-151-5/+10
| | | | | | | | | | | | | We honor NO_COLOR= to disable coloring, let's also honor CLICOLOR_FORCE= to enable it. The purpose will be for `make` calling the script and redirecting to a file, while enabling colors. See-also: https://bixense.com/clicolors/ Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: accept $NFT_TEST_TMPDIR_TAG for the result directoryThomas Haller2023-09-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We allow the user to set "$TMPDIR" to affect where the "nft-test.*" directory is created. However, we don't allow the user to specify the exact location, so the user doesn't really know which directory was created. One remedy is that the test will also create the symlink "$TMPDIR/nft-test.latest.$USER" to point to the last test result. However, if you run multiple tests in parallel, that is not reliable to find the test results. Accept $NFT_TEST_TMPDIR_TAG and use it as part of the generated filename. That way, the caller can set it to a unique tag, and find the directory later based on that. For example export TMPDIR=/tmp export NFT_TEST_TMPDIR_TAG=".$(uuidgen)" ./tests/shell/run-tests.sh ls -lad "$TMPDIR/nft-test."*"$NFT_TEST_TMPDIR_TAG"*/ will work reliably -- as long as the tag is chosen uniquely. The reason to not allow the user to specify the directory name directly, is because we want that tests results follow the well-known pattern "/tmp/nft-test*". Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: exit 77 from "run-tests.sh" if all tests were skippedThomas Haller2023-09-151-1/+7
| | | | | | | | | | | | | | If there are multiple tests and some of them pass and some are skipped, the overall result should be success (zero). Because likely the user just selected a bunch of tests (or all of them). So skipping some tests does not mean that the entire run is not a success. However, if all tests are skipped, then mark the overall result as skipped too. The more common case is if you only run one single test, then we want to know, that the test didn't run. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: in find_tests() use C locale for sorting tests namesThomas Haller2023-09-151-1/+1
| | | | | | | | It makes more sense, that the sort order does not depend on the user's locale. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: special handle base path starting with "./"Thomas Haller2023-09-151-1/+3
| | | | | | | | | | | | | | | | | | When we auto detect the tests with `tests/shell/run-tests.sh -L`, then commonly the NFT_TEST_BASEDIR starts with a redundant "./". That's a bit ugly. Instead, special handle that case and remove the prefix. The effect is that `tests/shell/run-tests.sh -L` shows tests/shell/testcases/bitwise/0040mark_binop_0 instead of ./tests/shell/testcases/bitwise/0040mark_binop_0 Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: add missing nft/nodump files for testsThomas Haller2023-09-153-0/+29
| | | | | | | | | | Three tests didn't have a nft/nodump file, because previously I only generated files on Fedora kernel, where those tests are failing. Generate them on CentOS-Stream-9 with kernel 5.14.0-354.el9.x86_64. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: drop unstable dump for "transactions/0051map_0" testThomas Haller2023-09-152-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The file "tests/shell/testcases/transactions/dumps/0051map_0.nft" gets generated differently on Fedora 38 (6.4.14-200.fc38.x86_64) and CentOS-Stream-9 (5.14.0-354.el9.x86_64). It's not stable. diff --git c/tests/shell/testcases/transactions/dumps/0051map_0.nft w/tests/shell/testcases/transactions/dumps/0051map_0.nft index 59d69df70e61..fa7df9f93757 100644 --- c/tests/shell/testcases/transactions/dumps/0051map_0.nft +++ w/tests/shell/testcases/transactions/dumps/0051map_0.nft @@ -1,7 +1,11 @@ table ip x { + chain w { + } + chain m { } chain y { + ip saddr vmap { 1.1.1.1 : jump w, 2.2.2.2 : accept, 3.3.3.3 : goto m } } } Drop it. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: remove spurious .nft dump filesThomas Haller2023-09-152-16/+0
| | | | | | | | | | | These are left-over dumps ([1]), or dumps generated with the wrong name ([2]). Remove the files. [1] commit eb14363d44ce ('tests: shell: move chain priority and policy to chain folder') [2] commit b4775dec9f80 ('src: ingress inet support') Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: add option to shuffle execution order of testsThomas Haller2023-09-151-0/+18
| | | | | | | | | | | | The user can set NFT_TEST_SHUFFLE_TESTS=y|n to have the tests shuffled randomly. The purpose of shuffling is to find tests that depend on each other, or would break when run in unexpected order. If unspecified, by default tests are shuffled if no tests are selected on the command line. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: add "random-source.sh" helper for random-source for sort/shufThomas Haller2023-09-152-1/+41
| | | | | | | | | | | | | | | | | Commands `sort` and `shuf` have a "--random-source" argument. That's useful for generating stable, reproducible "random" output. However, we want to do this based on a fixed seed, while the "--random-source" expects a stream of randomness. Add a helper script for that. Also, use the stable randomness for shuf in the test "tests/shell/testcases/sets/automerge_0". See-also: https://www.gnu.org/software/coreutils/manual/html_node/Random-sources.html#Random-sources Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: export NFT_TEST_RANDOM_SEED variable for testsThomas Haller2023-09-151-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | Let "run-tests.sh" export a NFT_TEST_RANDOM_SEED variable, set to a decimal, random integer (in the range of 0 to 0x7FFFFFFF). The purpose is to provide a seed to tests for randomization. Randomizing tests is very useful to increase the coverage while not testing all combinations (which might not be practical). The point of NFT_TEST_RANDOM_SEED is that the user can set the environment variable so that the same series of random events is used. That is useful for reproducing an issue, that is known to happen with a certain seed. - by default, if the user leaves NFT_TEST_RANDOM_SEED unset or empty, the script generates a number using $SRANDOM. - if the user sets NFT_TEST_RANDOM_SEED to an integer it is taken as is (modulo 0x80000000). - otherwise, calculate a number by hashing the value of $NFT_TEST_RANDOM_SEED. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: ensure vgdb-pipe files are deleted from "nft-valgrind-wrapper.sh"Thomas Haller2023-09-142-3/+11
| | | | | | | | | | | | | When the valgrind process gets killed, those files can be left over. They are located in the original $TMPDIR (usually /tmp). They should be cleaned up. I tried to cleanup the files from withing "nft-valgrind-wrapper.sh" itself via a `trap`, but it doesn't work. Instead, let "run-tests.sh" delete all files with a matching pattern. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: kill running child processes when aborting "run-tests.sh"Thomas Haller2023-09-141-5/+26
| | | | | | | | | | | | | | When aborting "run-tests.sh", child processes were left running. Kill them. It's surprisingly complicated to get this somewhat right. Do it by enabling monitor mode for each test call, so that they run in separate process groups and we can kill the entire group. Note that we cannot just `kill -- -$$`, because it's not clear who is in this process group. Also, we don't want to kill the `tee` process which handles our logging. Signed-off-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/shell: typeof_integer/raw: prefer @nh for payload matchingFlorian Westphal2023-09-106-12/+12
| | | | | | | | | | @ih fails on kernels where payload expression doesn't support the 'inner' base offset. This test isn't about inner headers, so just use @nh which is universally available. Signed-off-by: Florian Westphal <fw@strlen.de>