summaryrefslogtreecommitdiffstats
path: root/tests/py/nft-test.py
Commit message (Collapse)AuthorAgeFilesLines
* tests: py: Warn if recorded JSON output matches the inputPhil Sutter2024-04-121-0/+2
| | | | | | Actively support spring-cleaning by nagging callers. Signed-off-by: Phil Sutter <phil@nwl.cc>
* evaluate: reset statement length context before evaluating statementPablo Neira Ayuso2023-12-081-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch consolidates ctx->stmt_len reset in stmt_evaluate() to avoid this problem. Note that stmt_evaluate_meta() and stmt_evaluate_ct() already reset it after the statement evaluation. Moreover, statement dependency can be generated while evaluating a meta and ct statement. Payload statement dependency already manually stashes this before calling stmt_evaluate(). Add a new stmt_dependency_evaluate() function to stash statement length context when evaluating a new statement dependency and use it for all of the existing statement dependencies. Florian also says: 'meta mark set vlan id map { 1 : 0x00000001, 4095 : 0x00004095 }' will crash. Reason is that the l2 dependency generated here is errounously expanded to a 32bit-one, so the evaluation path won't recognize this as a L2 dependency. Therefore, pctx->stacked_ll_count is 0 and __expr_evaluate_payload() crashes with a null deref when dereferencing pctx->stacked_ll[0]. nft-test.py gains a fugly hack to tolerate '!map typeof vlan id : meta mark'. For more generic support we should find something more acceptable, e.g. !map typeof( everything here is a key or data ) timeout ... tests/py update and assert(pctx->stacked_ll_count) by Florian Westphal. Fixes: edecd58755a8 ("evaluate: support shifts larger than the width of the left operand") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: add map supportPablo Neira Ayuso2023-09-271-4/+66
| | | | | | | | | | Add basic map support to this infrastructure, eg. !map1 ipv4_addr : mark;ok Adding elements to map is still not supported. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Don't colorize output if stderr is redirectedPhil Sutter2022-04-081-1/+1
| | | | | | | | Cover for calls with '2>/tmp/log' and avoid printing escape sequences to that file. One could still keep colored output on stdout, but that required a printing routine for non-errors. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests/py: Avoid duplicate records in *.got filesPhil Sutter2021-11-301-10/+19
| | | | | | | | | | If payloads don't contain family-specific bits, they may sit in a single *.payload file for all tested families. In such case, nft-test.py will consequently write dissenting payloads into a single *.got file. To avoid the duplicate entries, check if a matching record exists already before writing it out. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests/py: Make netns spawning more robustPhil Sutter2021-08-111-6/+34
| | | | | | | On systems without unshare Python module, try to call unshare binary with oneself as parameters. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests/py: fix error messageFlorian Westphal2021-06-021-1/+5
| | | | | | | This should say "should have failed" only if the set add operation was supposed to fail, not when its supposed to work. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/py: Write dissenting payload into the right filePhil Sutter2021-02-041-1/+4
| | | | | | | | | | | | The testsuite supports diverging payloads depending on table family. This is necessary since for some families, dependency matches are created. If a payload mismatch happens, record it into a "got"-file which matches the family-specific payload file, not the common one. This eases use of diff-tools a lot as the extra other families' payloads confuse the tools. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: py: add netdev folder and reject.t icmp casesJose M. Guisado Gomez2020-11-021-1/+1
| | | | | | | | | | Add unit tests for the use of reject with icmp inside netdev family. reject.t from inet family couldn't be reused because it was using meta nfproto which is not supported inside netdev. Signed-off-by: Jose M. Guisado Gomez <guigom@riseup.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: flush log file output before running each commandPablo Neira Ayuso2020-09-141-0/+2
| | | | | | | If nft crashes or hits an assertion, the last command run shows in the /tmp/nftables-test.log file. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: Run in separate network namespace, don't break connectivityStefano Brivio2020-06-151-0/+6
| | | | | | | | | | | | | It might be convenient to run tests from a development branch that resides on another host, and if we break connectivity on the test host as tests are executed, we can't run them this way. If kernel implementation (CONFIG_NET_NS), unshare(1), or Python bindings for unshare() are not available, warn and continue. Suggested-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Support testing host binariesPhil Sutter2020-02-071-4/+18
| | | | | | | Support -H/--host option to use host's libnftables.so.1. Alternatively users may specify a custom library path via -l/--library option. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests/py: Set a fixed timezone in nft-test.pyPhil Sutter2019-11-291-0/+1
| | | | | | | | | | | | Payload generated for 'meta time' matches depends on host's timezone and DST setting. To produce constant output, set a fixed timezone in nft-test.py. Choose UTC-2 since most payloads are correct then, adjust the remaining two tests. Fixes: 0518ea3f70d8c ("tests: add meta time test cases") Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Ander Juaristi <a@juaristi.eus> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Fix test script for Python3 tempfilePhil Sutter2019-10-291-4/+4
| | | | | | | | | | | | | When instantiating a temporary file using tempfile's TemporaryFile() constructor, the resulting object's 'name' attribute is of type int. This in turn makes print_msg() puke while trying to concatenate string and int using '+' operator. Fix this by using format strings consequently, thereby cleaning up code a bit. Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: make sure i is definedFlorian Westphal2019-08-161-0/+1
| | | | | | | | | | | The test script can die in case there are severe problems, such as rlen being 0 -- in that case i is undefined and script evaluation is aborted. Found during nft development, no existing test case shows this problem. Signed-off-by: Florian Westphal <fw@strlen.de>
* src: add ct expectations supportStéphane Veyret2019-07-161-0/+4
| | | | | | | This modification allow to directly add/list/delete expectations. Signed-off-by: Stéphane Veyret <sveyret@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: fix python3Shekhar Sharma2019-07-011-33/+33
| | | | | | | This converts the nft-test.py file to run on both py2 and py3. Signed-off-by: Shekhar Sharma <shekhar250198@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Support JSON validationPhil Sutter2019-05-311-1/+24
| | | | | | | | | | Introduce a new flag -s/--schema to nft-test.py which enables validation of any JSON input and output against our schema. Make use of traceback module to get more details if validation fails. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Fix error messages in chain_delete()Phil Sutter2019-04-301-2/+2
| | | | | | | | | | Adding string and chain object is an illegal operation in Python. Instead concatenate with cmd string since that contains all required information already. Fixes: 820fd08b5f1d4 ("tests/py: Review print statements in nft-test.py") Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* py: Adjust Nftables class to output flags changesPhil Sutter2018-10-311-18/+11
| | | | | | | | | | | | | Introduce setter/getter methods for each introduced output flag. Ignore NFT_CTX_OUTPUT_NUMERIC_ALL for now since it's main purpose is for internal use. Adjust the script in tests/py accordingly: Due to the good defaults, only numeric proto output has to be selected - this is not a must, but allows for the test cases to remain unchanged. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Check differing rule output for sanityPhil Sutter2018-09-111-0/+24
| | | | | | | | | If an added rule's listing differs from the input (either expected or not), reinsert that output and check payload again to make sure the asymmetry doesn't lead to (internal) changes in ruleset. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Make nft-test.py a little more robustPhil Sutter2018-09-031-2/+9
| | | | | | | | | When adding a new test, missing payload file causes nft-test.py to choke due to accessing undeclared variables. Fix this by making sure relevant variables are declared outside of try-catch blocks. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: add ct timeout testsHarsha Sharma2018-08-311-0/+4
| | | | | | | Add test for adding ct timeout objects and assigning it via rule. Signed-off-by: Harsha Sharma <harshasharmaiitr@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* JSON: Review verdict statement and expressionPhil Sutter2018-08-301-1/+3
| | | | | | | | | | | | Change jump and goto verdicts to become extensible by dedicating an object for the target parameter. While being at it, drop break and queue verdict expressions since they don't seem to exist, no idea where I got those from in the first place. For queue, there is a dedicated expression at least. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Fix coloring of differencesPhil Sutter2018-08-301-17/+14
| | | | | | | This was surprisingly hard to get right, but this should do the trick. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: fix import when run from other directoryEric Leblond2018-06-211-2/+2
| | | | | Signed-off-by: Eric Leblond <eric@regit.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: minor cleaningEric Leblond2018-06-201-4/+5
| | | | | | | Move import and use explicit parameter in object creation. Signed-off-by: Eric Leblond <eric@regit.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* JSON: Sort out rule position and handles in generalPhil Sutter2018-06-011-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | First of all, don't print the position property when listing rules. This was there only because libnftnl JSON output has it too, but since the preferred way to *add* a rule at some location is via 'handle' keyword, keeping "position" in output would be non-intuitive. Changing "position" property name to "handle" instead is also a no-go since that would clash with the real rule handle. Secondly, turn all handle output on regardless of octx->handle setting. For a programmatic API like JSON, this should be fine. Thirdly, fix rule locations when parsing JSON: Respect "handle" property for CMD_INSERT and CMD_ADD and ignore "pos" at all (actually even a typo, should have read "position"). Also support "index" property recently added to standard syntax. Finally, adjust nft-test.py for the above changes: There is no "position" property to drop from rule output, and "handle" property will always be present. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Make diff functions use print_* functionsMáté Eckl2018-05-171-6/+4
| | | | | Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: print path of the logfileMáté Eckl2018-05-171-1/+4
| | | | | | | | | | It is good to know that a log is generated even without browsing the nft-test.py source code. Also print_info function is introduced. Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: print_msg refactorMáté Eckl2018-05-171-6/+8
| | | | | | | | | | | | | | | The errstr attribute was hard-coded to "ERROR:" errstr has been moved in the parameter list. As print_msg is only used from the other print_* this is not an issue, and as there is a print_error function, I don't think that strerr should default to "ERROR:". Also this kind of messages now get written to stderr. This can be beneficial if someone wants to redirect output to a file. Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: specify python version in nft-test.pyMáté Eckl2018-05-171-1/+1
| | | | | | | | /usr/bin/python is linked to different main version of python in different distributions (eg. 2 on debian, 3 on arch linux). Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/py: Support testing JSON input and output as wellPhil Sutter2018-05-111-5/+181
| | | | | | | | | | | | | | | | | | This extends nft-test.py by optional JSON testing capabilities, activated via '-j'/'--enable-json' parameter). JSON testing happens for all rules which are supposed to work: After a rule has been added and the existing tests (payload, ruleset listing output) have been performed, basically the same test is done again using a recorded JSON equivalent and (if necessary) a recorded listing output. The code tries to ease new test case creation overhead by auto-generating JSON equivalent input via listing the (non-JSON) rule in JSON format. Also, differing netlink debug and listing output are stored in *.got files to assist in analyzing/fixing failing test cases. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Highlight offending parts in differences warningsPhil Sutter2018-05-111-1/+34
| | | | | | | | Print the non-equal parts of the two rules in yellow when printing the differences warning. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Don't read expected payload for each tablePhil Sutter2018-05-111-10/+12
| | | | | | | | | When testing rule adding to different table families, expected payload was read for each tested family again. Instead, read it just once and just try to read a family-specific payload for each tested family. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Reduce indenting level in nft-test.pyPhil Sutter2018-05-111-61/+65
| | | | | Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: allow to specify sets with a timeoutFlorian Westphal2018-05-091-8/+17
| | | | | | | | | | | Not usable yet, as the set timeout netlink output isn't captured so far, but it adds groundwork to add this as a follow-up. Set definition syntax changes a little, if you want to add multiple elements they now have to be separated by "," just like in nftables. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/py: Fix differences printing for rules with setsPhil Sutter2018-04-141-2/+2
| | | | | | | | If a rule contained a set, it would always print the tested rule as differing from the output instead of any given output (if defined). Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Allow passing multiple files to nft-test.pyPhil Sutter2018-04-111-5/+6
| | | | | | | | | | This allows to run the testsuite over a subset of test cases, e.g. like so: | ./nft-test.py bridge/*.t Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Review print statements in nft-test.pyPhil Sutter2018-04-111-41/+39
| | | | | | | | | | | | | | | | Make use of format strings as they are easier to read than manual string concatenation. Also use class Table's __str__ method instead of printing the 'name' attribute. This changes the output in that table names are prepended by their family, but the extra information may come in handy when analyzing issues. Since class Chain's __str__ method returns just the 'name' attribute content, it may be used as synonym. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Use libnftables instead of calling nft binaryPhil Sutter2018-04-111-81/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a simple nftables Python class in py/nftables.py which gives access to libnftables API via ctypes module. nft-test.py is extended to make use of the above class instead of calling nft binary. Since command line formatting had to be touched anyway, this patch also streamlines things a bit by introducing __str__ methods to classes Table and Chain and making extensive use of format strings instead of onerously adding all string parts together. Since the called commands don't see a shell anymore, all shell meta character escaping done in testcases is removed. The visible effects of this change are: * Four new warnings in ip/flowtable.t due to changing objref IDs (will be addressed later in a patch to libnftnl). * Reported command line in warning and error messages changed slightly for obvious reasons. * Reduction of a full test run's runtime by a factor of four. Status diff after running with 'time': < 83 test files, 77 files passed, 1724 unit tests, 0 error, 33 warning < 87.23user 696.13system 15:11.82elapsed 85%CPU (0avgtext+0avgdata 9604maxresident)k < 8inputs+36800outputs (0major+35171235minor)pagefaults 0swaps > 83 test files, 77 files passed, 1724 unit tests, 4 error, 33 warning > 6.80user 30.18system 3:45.86elapsed 16%CPU (0avgtext+0avgdata 14064maxresident)k > 0inputs+35808outputs (0major+2874minor)pagefaults 0swaps Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* nft-test: check start of rule with sets tooFlorian Westphal2018-04-011-0/+4
| | | | | | | | | | | | | | | We special case rules with sets ({}) so set members are sorted properly. But we failed to check start of rule: input: meta mark { 1, 2 } bar expect: meta mark { 1, 2 } bar We made sure 'bar' is equal in both input and expected output, but we did not check start (before {). This causes a gazillion of warnings, will fix in followup commit. Signed-off-by: Florian Westphal <fw@strlen.de>
* src: Spelling fixesVille Skyttä2018-02-151-1/+1
| | | | | Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: trivial: Fix error messagePhil Sutter2017-12-091-1/+1
| | | | | | | | The error message for failed chain creation quotes the chain's name but lacked the closing tick. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Florian Westphal <fw@strlen.de>
* netlink_linearize: skip set element expression in set statement keyAnders K. Pedersen2017-10-061-2/+7
| | | | | | | | | | | | | | | | | | | | Before this patch the following fails: # nft add rule ip6 filter x \ set add ip6 saddr . ip6 daddr @test nft: netlink_linearize.c:648: netlink_gen_expr: Assertion `dreg < ctx->reg_low' failed. Aborted This is was previously fixed for flow statements in fbea4a6f4449 ("netlink_linearize: skip set element expression in flow table key"), and this patch implements the same change for set statements by using the set element key in netlink_gen_set_stmt(). nft-test.py is updated to support set types with concatenated data types in order to support testing of this. Signed-off-by: Anders K. Pedersen <akp@cohaesio.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Fail test forcefully when bug is not fixedShyam Saini2017-06-261-1/+1
| | | | | | | | | | | | | | | | | | | | When we have "fail" in the test cases then py test doesn't complain anything, but the test should complain if the fix is not applied. Before applying 986dea8a4a9d ("evaluate: avoid reference to multiple src data in statements which set values"), nft throws following error message and exits with error code 134. $ nft add rule x y tcp dport set { 0 , 1 } BUG: unknown expression type set reference nft: netlink_linearize.c:696: netlink_gen_expr: Assertion `0' failed. Aborted This commit enforces nft-test.py to throw error message when the fix is not applied. Signed-off-by: Shyam Saini <mayhs11saini@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: flush stdout on each test runPablo Neira Ayuso2017-05-291-0/+2
| | | | | | Useful when redirecting output to file and tail -f it. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: add insert-failure testFlorian Westphal2017-03-161-3/+14
| | | | | | | It should not be possible to add a ip6 restricted helper to ip family. Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: add ct helper testsFlorian Westphal2017-03-161-1/+10
| | | | | | | needs minor tweak to nft-test.py so we don't zap the ';' withhin the {}. Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Add suport for stateful objects in python testsElise Lennion2017-01-271-0/+131
| | | | | | | | | | | | | | This allows to write pytests using the new stateful objects. To add an object use the symbol '%', followed by the name, type and specifications (currently used in quota): %cnt1 type counter;ok # Adds the counter cnt1 to all tables %qt1 type quota over 25 mbytes;ok # Adds the quota qt1 to all tables Signed-off-by: Elise Lennion <elise.lennion@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>