summaryrefslogtreecommitdiffstats
path: root/iptables-test.py
Commit message (Collapse)AuthorAgeFilesLines
* tests: iptables-test: Increase non-fast mode strictnessPhil Sutter2024-02-011-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | The simple search for the rule in save output accepted arbitrary leading and trailing rule parts. This was partly desired as it allowed to omit the leading '-A' flag or ignore the mandatory '-j CONTINUE' in ebtables rules, though it could hide bugs. Introduction of fast mode mitigated this due to the way how it searches for multiple rules at the same time, but there are cases which fast mode does not support yet (e.g. test cases containing variant-specific rule output). Given save output format will never contain the rule in first or last line, so enclosing the searched rule in newline characters is sufficient to make the search apply to full lines only. The only drawback is having to add '-A' and '-j CONTINUE' parts if needed. The hidden bugs this revealed were: - Long --nflog-prefix strings are not cut to 64 chars with iptables-nft - The TCPMSS rule supposed to fail with legacy only must specify an expected save output Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Use difflib if dumps differPhil Sutter2023-12-211-5/+7
| | | | | | | Improve log readability by printing a unified diff of the expected vs. actual iptables-save output. Signed-off-by: Phil Sutter <phil@nwl.cc>
* Revert --compat option related commitsPhil Sutter2023-09-011-15/+4
| | | | | | | | | | | | | | | | | | | This reverts the following commits: b14c971db6db0 ("tests: Test compat mode") 11c464ed015b5 ("Add --compat option to *tables-nft and *-nft-restore commands") ca709b5784c98 ("nft: Introduce and use bool nft_handle::compat") 402b9b3c07c81 ("nft: Pass nft_handle to add_{target,action}()") This implementation of a compatibility mode implements rules using xtables extensions if possible and thus relies upon existence of those in kernel space. Assuming no viable replacement for the internal mechanics of this mode will be found in foreseeable future, it will effectively block attempts at deprecating and removing of these xtables extensions in favor of nftables expressions and thus hinder upstream's future plans for iptables. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: Test compat modePhil Sutter2023-08-111-4/+15
| | | | | | | | | | Extend iptables-test.py by a third mode, which is using xtables-nft-multi and passing --compat to all calls creating rules. Also add a shell testcase asserting the effectiveness of --compat by comparing debug (-vv) output. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Fix command segfault reportsPhil Sutter2023-08-101-3/+6
| | | | | | | Logging produced a stack trace due to undefined variable 'cmd'. Fixes: 0e80cfea3762b ("tests: iptables-test: Implement fast test mode") Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test.py: make explicit use of python3Arturo Borrero Gonzalez2023-03-151-1/+1
| | | | | | | | | | | In most distros 'python' means python2, which is not available anywhere. This is a problem when, for example, building the Debian package. This script is called as part of the build but 'python' is not available. Mention python3 explictly. The script runs just fine in python3. Signed-off-by: Arturo Borrero Gonzalez <arturo@netfilter.org> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: Adjust testsuite return codes to automake guidelinesPhil Sutter2022-12-141-1/+1
| | | | | | | | | | | | | As per the manual[1]: "When no test protocol is in use, an exit status of 0 from a test script will denote a success, an exit status of 77 a skipped test, an exit status of 99 a hard error, and any other exit status will denote a failure." [1] https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Cover for obligatory -j CONTINUE in ebtablesPhil Sutter2022-10-261-0/+3
| | | | | | | | | Unlike iptables, ebtables includes the default rule target in output. Instead of adding it to every rule in ebtables tests, add special casing to the testscript checking if the expected rule output contains a target already and adding the default one if not. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Implement fast test modePhil Sutter2022-10-261-1/+159
| | | | | | | | | | | | | | | | | | | | | | | | | Implement a faster mode of operation for suitable test files: 1) Collect all rules to add and all expected output in lists 2) Any supposedly failing rules are checked immediately like in slow mode. 3) Create and load iptables-restore input from the list in (1) 5) Construct the expected iptables-save output from (1) and check it in a single search 5) If any of the steps above fail, fall back to slow mode for verification and detailed error analysis. Fast mode failures are not fatal, merely warn about them. To keep things simple (and feasible), avoid complicated test files involving external commands, multiple tables or variant-specific results. Aside from speeding up testsuite run-time, rule searching has become more strict since EOL char is practically part of the search string. This revealed many false positives where the expected string was actually a substring of the printed rule. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Test both variants by defaultPhil Sutter2022-10-011-33/+51
| | | | | | | | | Via '--legacy' and '--nftables' flags one may choose the variant to test. Change the default (none of them given) from legacy to both, by effectively running twice. Prefix the summary line with the tested variant for clarity and print a total count line as well. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Pass netns to execute_cmd()Phil Sutter2022-10-011-24/+16
| | | | | | | | | | | | | | | The command to run might have to be prefixed. Once if the command is 'iptables' (or related) to define the variant, once if '-N' was given to run the command inside the netns. Doing both prefixing inside execute_cmd() avoids a potential conflict and thus simplifies things: The "external command" and "external iptables call" lines become identical in handling, there is no need for a separate prefix char anymore. As a side-effect, this commit also fixes for delete_rule() calls in error case ignoring the netns value. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Simplify execute_cmd() callingPhil Sutter2022-10-011-3/+3
| | | | | | Default 'lineno' parameter to zero, Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Simplify '-N' option a bitPhil Sutter2022-10-011-7/+10
| | | | | | | | Instead of hard-coding, store the netns name in args.netns if the flag was given. The value defaults to None, so existing 'if netns' checks are still valid. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: support explicit variant test resultJeremy Sowden2022-02-131-5/+13
| | | | | | | | Now that there are more than two test results, add support for explicitly indicating which result to expect if the variants differ. Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: add `NOMATCH` test resultJeremy Sowden2022-02-131-11/+25
| | | | | | | | | | | | | Currently, there are two supported test results: `OK` and `FAIL`. It is expected that either the iptables command fails, or it succeeds and dumping the rule has the correct output. However, it is possible that the command may succeed but the output may not be correct. Add a `NOMATCH` result to cover this outcome. Make a few white-space improvements at the same time. Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: iptables-test: rename variableJeremy Sowden2022-02-131-10/+10
| | | | | | | | "Splitted" hasn't been current since the seventeenth century. Replace it with "tokens". Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: iptables-test: Support variant deviationPhil Sutter2022-02-101-0/+25
| | | | | | | | | | | | | | | | Some test results are not consistent between variants: * CLUSTERIP is not supported with nft_compat, so all related tests fail with iptables-nft. * iptables-legacy mandates TCPMSS be combined with SYN flag match, iptables-nft does not care. (Or precisely, xt_TCPMSS.ko can't validate match presence.) Introduce an optional fourth test spec field to specify the variant it applies to. Consequently, the opposite result is expected with the other variant. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: correct misspelt variableJeremy Sowden2022-01-161-9/+9
| | | | | | | "EXECUTEABLE" -> "EXECUTABLE" Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: iptables-test: Fix conditional colors on stderrPhil Sutter2021-09-221-4/+5
| | | | | | | | | | Štěpán's patch to make colored output depend on whether output is a TTY clashed with my change to print errors to stderr instead of stdout. Fix this by telling maybe_colored() if it should print colors or not as only caller knows where output is sent to. Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test.py: print with color escapes only when stdout isattyŠtěpán Němec2021-09-131-10/+13
| | | | | | | | When the output doesn't go to a terminal (typical case: log files), the escape sequences are just noise. Signed-off-by: Štěpán Němec <snemec@redhat.com> Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Exit non-zero on errorPhil Sutter2021-09-131-1/+2
| | | | | | If any test fails, return a non-zero exit code. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Print errors to stderrPhil Sutter2021-09-131-4/+5
| | | | | | | No big deal, just pass the extra parameter to the four error print calls. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: iptables-test: Fix missing chain casePhil Sutter2021-09-131-2/+5
| | | | | | | | | If a chain line was really missing, Python complained about reference before assignment of 'chain_array' variable. While being at it, reuse print_error() function for reporting and allow to continue with the next input file instead of exiting. Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test: Make netns spawning more robustPhil Sutter2021-08-111-7/+30
| | | | | | | On systems without unshare Python module, try to call unshare binary with oneself as parameters. Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test.py: Try to unshare netns by defaultPhil Sutter2020-11-171-0/+7
| | | | | | | | If user did not explicitly requst to "test netnamespace path", try an import of 'unshare' module and call unshare() to avoid killing the local host's network by accident. Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test.py: Accept multiple test files on commandlinePhil Sutter2020-11-171-2/+2
| | | | | | | | This allows to call the script like so: | # ./iptables-test.py -n extensions/libebt_*.t Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test: Don't choke on empty linesPhil Sutter2020-05-111-1/+1
| | | | | | | | | | | | | | | | | | | | The script code wasn't expecting empty lines: | Traceback (most recent call last): | File "./iptables-test.py", line 380, in <module> | main() | File "./iptables-test.py", line 370, in main | file_tests, file_passed = run_test_file(filename, args.netns) | File "./iptables-test.py", line 265, in run_test_file | if item[1] == "=": | IndexError: list index out of range Fix this by ignoring empty lines or those consisting of whitespace only. While being at it, remove the empty line from libxt_IDLETIMER.t which exposed the problem. Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test.py: Fix --host modePhil Sutter2020-02-241-3/+2
| | | | | | | | | In some cases, the script still called repo binaries. Avoid this when in --host mode to allow testing without the need to compile sources in beforehand. Fixes: 1b5d762c1865e ("iptables-test: Support testing host binaries") Signed-off-by: Phil Sutter <phil@nwl.cc>
* iptables-test: Run tests in lexical orderPhil Sutter2019-10-171-2/+6
| | | | | | | | To quickly see if a given test was run or not, sort the file list. Also filter non-test files right when preparing the list. Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* iptables-test: Support testing host binariesPhil Sutter2019-09-151-2/+6
| | | | | | | | Introduce --host parameter to run the testsuite against host's binaries instead of built ones. Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Florian Westphal <fw@strlen.de>
* iptables-tests: fix python3Shekhar Sharma2019-06-261-11/+11
| | | | | | | | | This converts the iptables-test.py file to run on both python2 and python3. The error regarding out.find() has been fixed by using method .encode('utf-8') in its argument. Signed-off-by: Shekhar Sharma <shekhar250198@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* arptables: add basic test infra for arptables-nftFlorian Westphal2018-11-121-1/+10
| | | | Signed-off-by: Florian Westphal <fw@strlen.de>
* iptables-tests: add % to run iptables commandsPablo Neira Ayuso2018-11-031-0/+8
| | | | | | | Lines starting by % allows you to run iptables commands, use it for rateest test. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* iptables-tests: do not append xtables-multi to external commandsPablo Neira Ayuso2018-11-031-1/+1
| | | | | | | | | Lines starting by @ can be used to invoke an external command of any kind. Do not add xtables-multi here since we may want to execute a non-iptables command. Fixes: 9ff99156b63e ("iptables-test: fix netns test") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: add basic ebtables test supportFlorian Westphal2018-11-031-3/+10
| | | | | | | now that we have ebtables-save, lets add test cases for ebtables-nft as well. Signed-off-by: Florian Westphal <fw@strlen.de>
* iptables-test: fix netns testTaehee Yoo2018-11-011-13/+9
| | | | | | | | | | | | | | | | The libxt_rateest test always fails because dependent command is not executed in netns. (@iptables -I INPUT -j RATEEST --rateest-name RE1 --rateest-interval \ 250.0ms --rateest-ewmalog 500.0ms) After this path, adding netns command is executed first. Then test commands are executed. Fixes: 0123183f43a9 ("iptables-test: add -N option to exercise netns removal path") Reported-by: Pablo Neira Ayuso <pablo@netfilter.org> Signed-off-by: Taehee Yoo <ap420073@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* iptables-test: add -N option to exercise netns removal pathPablo Neira Ayuso2018-10-201-6/+32
| | | | | | | We are getting bug reports lately from the netns path, add a new option to exercise this path. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* xtables: rename xt-multi binaries to -nft, -legacyFlorian Westphal2018-06-271-2/+4
| | | | | | | | | | | | | | This adds a clear distinction between old iptables (formerly xtables-multi, now xtables-legacy-multi) and new iptables (formerly xtables-compat-multi, now xtables-nft-multi). Users will get the ip/ip6tables names via symbolic links, having a distinct name postfix for the legacy/nft variants helps to make a clear distinction, as iptables-nft will always use nf_tables and iptables-legacy always uses get/setsockopt wheres "iptables" could be symlinked to either -nft or -legacy. Signed-off-by: Florian Westphal <fw@strlen.de>
* iptables-test: fix bug with rateestFlorian Westphal2018-05-071-2/+5
| | | | | | | | | | | | | | rateest test needs to insert rateest targets with @progname syntax. However, this used the system-installed binary rather than the one from git. Morever, it did not respect --nftables switch, i.e. add occurs with iptables so iptables-compat won't find it. Fixes: d7ac61b58e78 ("add nft switch and test binaries from git") Signed-off-by: Florian Westphal <fw@strlen.de>
* iptables-test: add nft switch and test binaries from gitFlorian Westphal2018-04-281-3/+14
| | | | | | | | | instead of testing the system-wide installed iptables, test the version from git instead. Also add a --nftables switch to feed the test cases to xt-compat. Signed-off-by: Florian Westphal <fw@strlen.de>
* add iptables unit test infrastructurePablo Neira Ayuso2013-10-071-0/+311
This patch adds a python script to verify unit test cases. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>