summaryrefslogtreecommitdiffstats
path: root/tests/py/nft-test.py
Commit message (Collapse)AuthorAgeFilesLines
* tests: py: add netdev folder and reject.t icmp casesJose M. Guisado Gomez2020-11-021-1/+1
| | | | | | | | | | Add unit tests for the use of reject with icmp inside netdev family. reject.t from inet family couldn't be reused because it was using meta nfproto which is not supported inside netdev. Signed-off-by: Jose M. Guisado Gomez <guigom@riseup.net> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: flush log file output before running each commandPablo Neira Ayuso2020-09-141-0/+2
| | | | | | | If nft crashes or hits an assertion, the last command run shows in the /tmp/nftables-test.log file. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: Run in separate network namespace, don't break connectivityStefano Brivio2020-06-151-0/+6
| | | | | | | | | | | | | It might be convenient to run tests from a development branch that resides on another host, and if we break connectivity on the test host as tests are executed, we can't run them this way. If kernel implementation (CONFIG_NET_NS), unshare(1), or Python bindings for unshare() are not available, warn and continue. Suggested-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Support testing host binariesPhil Sutter2020-02-071-4/+18
| | | | | | | Support -H/--host option to use host's libnftables.so.1. Alternatively users may specify a custom library path via -l/--library option. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests/py: Set a fixed timezone in nft-test.pyPhil Sutter2019-11-291-0/+1
| | | | | | | | | | | | Payload generated for 'meta time' matches depends on host's timezone and DST setting. To produce constant output, set a fixed timezone in nft-test.py. Choose UTC-2 since most payloads are correct then, adjust the remaining two tests. Fixes: 0518ea3f70d8c ("tests: add meta time test cases") Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Ander Juaristi <a@juaristi.eus> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Fix test script for Python3 tempfilePhil Sutter2019-10-291-4/+4
| | | | | | | | | | | | | When instantiating a temporary file using tempfile's TemporaryFile() constructor, the resulting object's 'name' attribute is of type int. This in turn makes print_msg() puke while trying to concatenate string and int using '+' operator. Fix this by using format strings consequently, thereby cleaning up code a bit. Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: make sure i is definedFlorian Westphal2019-08-161-0/+1
| | | | | | | | | | | The test script can die in case there are severe problems, such as rlen being 0 -- in that case i is undefined and script evaluation is aborted. Found during nft development, no existing test case shows this problem. Signed-off-by: Florian Westphal <fw@strlen.de>
* src: add ct expectations supportStéphane Veyret2019-07-161-0/+4
| | | | | | | This modification allow to directly add/list/delete expectations. Signed-off-by: Stéphane Veyret <sveyret@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: fix python3Shekhar Sharma2019-07-011-33/+33
| | | | | | | This converts the nft-test.py file to run on both py2 and py3. Signed-off-by: Shekhar Sharma <shekhar250198@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Support JSON validationPhil Sutter2019-05-311-1/+24
| | | | | | | | | | Introduce a new flag -s/--schema to nft-test.py which enables validation of any JSON input and output against our schema. Make use of traceback module to get more details if validation fails. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Fix error messages in chain_delete()Phil Sutter2019-04-301-2/+2
| | | | | | | | | | Adding string and chain object is an illegal operation in Python. Instead concatenate with cmd string since that contains all required information already. Fixes: 820fd08b5f1d4 ("tests/py: Review print statements in nft-test.py") Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* py: Adjust Nftables class to output flags changesPhil Sutter2018-10-311-18/+11
| | | | | | | | | | | | | Introduce setter/getter methods for each introduced output flag. Ignore NFT_CTX_OUTPUT_NUMERIC_ALL for now since it's main purpose is for internal use. Adjust the script in tests/py accordingly: Due to the good defaults, only numeric proto output has to be selected - this is not a must, but allows for the test cases to remain unchanged. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Check differing rule output for sanityPhil Sutter2018-09-111-0/+24
| | | | | | | | | If an added rule's listing differs from the input (either expected or not), reinsert that output and check payload again to make sure the asymmetry doesn't lead to (internal) changes in ruleset. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Make nft-test.py a little more robustPhil Sutter2018-09-031-2/+9
| | | | | | | | | When adding a new test, missing payload file causes nft-test.py to choke due to accessing undeclared variables. Fix this by making sure relevant variables are declared outside of try-catch blocks. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: add ct timeout testsHarsha Sharma2018-08-311-0/+4
| | | | | | | Add test for adding ct timeout objects and assigning it via rule. Signed-off-by: Harsha Sharma <harshasharmaiitr@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* JSON: Review verdict statement and expressionPhil Sutter2018-08-301-1/+3
| | | | | | | | | | | | Change jump and goto verdicts to become extensible by dedicating an object for the target parameter. While being at it, drop break and queue verdict expressions since they don't seem to exist, no idea where I got those from in the first place. For queue, there is a dedicated expression at least. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Fix coloring of differencesPhil Sutter2018-08-301-17/+14
| | | | | | | This was surprisingly hard to get right, but this should do the trick. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: fix import when run from other directoryEric Leblond2018-06-211-2/+2
| | | | | Signed-off-by: Eric Leblond <eric@regit.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: minor cleaningEric Leblond2018-06-201-4/+5
| | | | | | | Move import and use explicit parameter in object creation. Signed-off-by: Eric Leblond <eric@regit.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* JSON: Sort out rule position and handles in generalPhil Sutter2018-06-011-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | First of all, don't print the position property when listing rules. This was there only because libnftnl JSON output has it too, but since the preferred way to *add* a rule at some location is via 'handle' keyword, keeping "position" in output would be non-intuitive. Changing "position" property name to "handle" instead is also a no-go since that would clash with the real rule handle. Secondly, turn all handle output on regardless of octx->handle setting. For a programmatic API like JSON, this should be fine. Thirdly, fix rule locations when parsing JSON: Respect "handle" property for CMD_INSERT and CMD_ADD and ignore "pos" at all (actually even a typo, should have read "position"). Also support "index" property recently added to standard syntax. Finally, adjust nft-test.py for the above changes: There is no "position" property to drop from rule output, and "handle" property will always be present. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Make diff functions use print_* functionsMáté Eckl2018-05-171-6/+4
| | | | | Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: print path of the logfileMáté Eckl2018-05-171-1/+4
| | | | | | | | | | It is good to know that a log is generated even without browsing the nft-test.py source code. Also print_info function is introduced. Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: print_msg refactorMáté Eckl2018-05-171-6/+8
| | | | | | | | | | | | | | | The errstr attribute was hard-coded to "ERROR:" errstr has been moved in the parameter list. As print_msg is only used from the other print_* this is not an issue, and as there is a print_error function, I don't think that strerr should default to "ERROR:". Also this kind of messages now get written to stderr. This can be beneficial if someone wants to redirect output to a file. Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: specify python version in nft-test.pyMáté Eckl2018-05-171-1/+1
| | | | | | | | /usr/bin/python is linked to different main version of python in different distributions (eg. 2 on debian, 3 on arch linux). Signed-off-by: Máté Eckl <ecklm94@gmail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/py: Support testing JSON input and output as wellPhil Sutter2018-05-111-5/+181
| | | | | | | | | | | | | | | | | | This extends nft-test.py by optional JSON testing capabilities, activated via '-j'/'--enable-json' parameter). JSON testing happens for all rules which are supposed to work: After a rule has been added and the existing tests (payload, ruleset listing output) have been performed, basically the same test is done again using a recorded JSON equivalent and (if necessary) a recorded listing output. The code tries to ease new test case creation overhead by auto-generating JSON equivalent input via listing the (non-JSON) rule in JSON format. Also, differing netlink debug and listing output are stored in *.got files to assist in analyzing/fixing failing test cases. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Highlight offending parts in differences warningsPhil Sutter2018-05-111-1/+34
| | | | | | | | Print the non-equal parts of the two rules in yellow when printing the differences warning. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Don't read expected payload for each tablePhil Sutter2018-05-111-10/+12
| | | | | | | | | When testing rule adding to different table families, expected payload was read for each tested family again. Instead, read it just once and just try to read a family-specific payload for each tested family. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Reduce indenting level in nft-test.pyPhil Sutter2018-05-111-61/+65
| | | | | Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: allow to specify sets with a timeoutFlorian Westphal2018-05-091-8/+17
| | | | | | | | | | | Not usable yet, as the set timeout netlink output isn't captured so far, but it adds groundwork to add this as a follow-up. Set definition syntax changes a little, if you want to add multiple elements they now have to be separated by "," just like in nftables. Signed-off-by: Florian Westphal <fw@strlen.de>
* tests/py: Fix differences printing for rules with setsPhil Sutter2018-04-141-2/+2
| | | | | | | | If a rule contained a set, it would always print the tested rule as differing from the output instead of any given output (if defined). Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Allow passing multiple files to nft-test.pyPhil Sutter2018-04-111-5/+6
| | | | | | | | | | This allows to run the testsuite over a subset of test cases, e.g. like so: | ./nft-test.py bridge/*.t Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Review print statements in nft-test.pyPhil Sutter2018-04-111-41/+39
| | | | | | | | | | | | | | | | Make use of format strings as they are easier to read than manual string concatenation. Also use class Table's __str__ method instead of printing the 'name' attribute. This changes the output in that table names are prepended by their family, but the extra information may come in handy when analyzing issues. Since class Chain's __str__ method returns just the 'name' attribute content, it may be used as synonym. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: Use libnftables instead of calling nft binaryPhil Sutter2018-04-111-81/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a simple nftables Python class in py/nftables.py which gives access to libnftables API via ctypes module. nft-test.py is extended to make use of the above class instead of calling nft binary. Since command line formatting had to be touched anyway, this patch also streamlines things a bit by introducing __str__ methods to classes Table and Chain and making extensive use of format strings instead of onerously adding all string parts together. Since the called commands don't see a shell anymore, all shell meta character escaping done in testcases is removed. The visible effects of this change are: * Four new warnings in ip/flowtable.t due to changing objref IDs (will be addressed later in a patch to libnftnl). * Reported command line in warning and error messages changed slightly for obvious reasons. * Reduction of a full test run's runtime by a factor of four. Status diff after running with 'time': < 83 test files, 77 files passed, 1724 unit tests, 0 error, 33 warning < 87.23user 696.13system 15:11.82elapsed 85%CPU (0avgtext+0avgdata 9604maxresident)k < 8inputs+36800outputs (0major+35171235minor)pagefaults 0swaps > 83 test files, 77 files passed, 1724 unit tests, 4 error, 33 warning > 6.80user 30.18system 3:45.86elapsed 16%CPU (0avgtext+0avgdata 14064maxresident)k > 0inputs+35808outputs (0major+2874minor)pagefaults 0swaps Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* nft-test: check start of rule with sets tooFlorian Westphal2018-04-011-0/+4
| | | | | | | | | | | | | | | We special case rules with sets ({}) so set members are sorted properly. But we failed to check start of rule: input: meta mark { 1, 2 } bar expect: meta mark { 1, 2 } bar We made sure 'bar' is equal in both input and expected output, but we did not check start (before {). This causes a gazillion of warnings, will fix in followup commit. Signed-off-by: Florian Westphal <fw@strlen.de>
* src: Spelling fixesVille Skyttä2018-02-151-1/+1
| | | | | Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: trivial: Fix error messagePhil Sutter2017-12-091-1/+1
| | | | | | | | The error message for failed chain creation quotes the chain's name but lacked the closing tick. Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Florian Westphal <fw@strlen.de>
* netlink_linearize: skip set element expression in set statement keyAnders K. Pedersen2017-10-061-2/+7
| | | | | | | | | | | | | | | | | | | | Before this patch the following fails: # nft add rule ip6 filter x \ set add ip6 saddr . ip6 daddr @test nft: netlink_linearize.c:648: netlink_gen_expr: Assertion `dreg < ctx->reg_low' failed. Aborted This is was previously fixed for flow statements in fbea4a6f4449 ("netlink_linearize: skip set element expression in flow table key"), and this patch implements the same change for set statements by using the set element key in netlink_gen_set_stmt(). nft-test.py is updated to support set types with concatenated data types in order to support testing of this. Signed-off-by: Anders K. Pedersen <akp@cohaesio.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Fail test forcefully when bug is not fixedShyam Saini2017-06-261-1/+1
| | | | | | | | | | | | | | | | | | | | When we have "fail" in the test cases then py test doesn't complain anything, but the test should complain if the fix is not applied. Before applying 986dea8a4a9d ("evaluate: avoid reference to multiple src data in statements which set values"), nft throws following error message and exits with error code 134. $ nft add rule x y tcp dport set { 0 , 1 } BUG: unknown expression type set reference nft: netlink_linearize.c:696: netlink_gen_expr: Assertion `0' failed. Aborted This commit enforces nft-test.py to throw error message when the fix is not applied. Signed-off-by: Shyam Saini <mayhs11saini@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: flush stdout on each test runPablo Neira Ayuso2017-05-291-0/+2
| | | | | | Useful when redirecting output to file and tail -f it. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: add insert-failure testFlorian Westphal2017-03-161-3/+14
| | | | | | | It should not be possible to add a ip6 restricted helper to ip family. Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: add ct helper testsFlorian Westphal2017-03-161-1/+10
| | | | | | | needs minor tweak to nft-test.py so we don't zap the ';' withhin the {}. Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Add suport for stateful objects in python testsElise Lennion2017-01-271-0/+131
| | | | | | | | | | | | | | This allows to write pytests using the new stateful objects. To add an object use the symbol '%', followed by the name, type and specifications (currently used in quota): %cnt1 type counter;ok # Adds the counter cnt1 to all tables %qt1 type quota over 25 mbytes;ok # Adds the quota qt1 to all tables Signed-off-by: Elise Lennion <elise.lennion@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Use stateless option on testsElise Lennion2017-01-181-1/+1
| | | | | | | | | | | To don't trigger false errors because of unrelated traffic on the tested machine. Tests, which have rules with counter and 'ok' result, are updated to avoid new Warnings. Signed-off-by: Elise Lennion <elise.lennion@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* Revert "tests: py: nft-tests.py: Add function for loading and removing ↵Florian Westphal2016-11-171-32/+0
| | | | | | | | | | | | | | | | | kernel modules" Its not needed at the moment, all tests that need a network interface (meta iif/oif) use the lo interface. For iifname/oifname the network device doesn't have to exist. We can revisit this in case we need it in the future. (seems commit causes issues on kernels built without dummy module). Reported-by: Arturo Borrero Gonzalez <arturo@debian.org> Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Manuel Messner <mm@skelett.io>
* tests: py: nft-tests.py: Add function for loading and removing kernel modulesManuel Johannes Messner2016-09-061-0/+32
| | | | | | | | Some tests use the dummy kernel module. This commit adds a function to automatically load that module and remove it afterwards. Signed-off-by: Manuel Johannes Messner <manuel.johannes.messner@hs-furtwangen.de> Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: allow to run tests with other nft binariesArturo Borrero2016-04-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | Allow to run tests with other nft binaries by reading a 'NFT' environment variable, allowing arbitrary locations for the nft binary. This is what the tests/shell/run-tests.sh script does. Among other thing, this allow us to properly hook this testsuite from the Debian CI environment (https://ci.debian.net) where we can perform tests for packages 'as installed'. Examples: # run with default config (ie src/nft) % ./nft-test.py # run with installed binary (ie /usr/sbin/nft) % NFT=/usr/sbin/nft ./nft-test.py Signed-off-by: Arturo Borrero Gonzalez <arturo.borrero.glez@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: explicitly indication of set type and flags from test definitionsPablo Neira Ayuso2016-04-271-21/+57
| | | | | | | | | This patch adds explicit set type in test definitions, as well as flags. This has triggered a rework that starts by introducing a Set class to make this whole code more extensible and maintainable. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* nft-test: don't zap remainder of rule after handling a setFlorian Westphal2016-03-101-6/+12
| | | | | | | | | | | | | | | | | | | | | | Don't delete the part after the set, i.e. given chain input { type filter hook input priority 0; policy accept; vlan id { 1, 2, 4, 100, 4095} vlan pcp 1-3 } don't remove the vlan pcp 1-3 part. This exposes following bug: bridge/vlan.t: WARNING: line: 32: 'nft add rule --debug=netlink bridge test-bridge input vlan id { 1, 2, 4, 100, 4095 } vlan pcp 1-3': 'vlan id { 1, 2, 4, 100, 4095 } vlan pcp 1-3' mismatches 'vlan id { 4, 1, 2, 4095, 100} vlan pcp 0-0' We do not shift the range, so on reverse translation we get a 0-0 output. The bug will be fixes in a followup commit. Signed-off-by: Florian Westphal <fw@strlen.de> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: modify supported test file syntaxPablo M. Bermudo Garay2016-01-121-26/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Until now, the syntax to represent tables and chains in test files was: *ip;test-ip4 *ip6;test-ip6 *inet;test-inet :input;type filter hook input priority 0 Where lines starting with * are tables and lines starting with : are chains. This commit change the test script to deal with new syntax: :input;type filter hook input priority 0 *ip;test-ip4;input *ip6;test-ip6;input *inet;test-inet;input Now the chains should be included before tables. Also, lines defining tables have a new third part (delimited by semicolon) where the chains needed by the table are declared. If table needs to include more than one chain, those must be separated by commas: :input;type filter hook input priority 0 :forward;type filter hook forward priority 0 :output;type filter hook output priority 0 *arp;test-arp;input,forward,output This new syntax allow to include in the same test file chains not supported by all families of tables tested. Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests/py: convert chains and tables to objectsPablo M. Bermudo Garay2016-01-121-58/+72
| | | | | | | | Now these concepts are represented by objects instead of lists or sparse parameters. Signed-off-by: Pablo M. Bermudo Garay <pablombg@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>