summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* src: restore interval sets work with string datatypesPablo Neira Ayuso2022-04-132-6/+8
| | | | | | | | | | | Switch byteorder of string datatypes to host byteorder. Partial revert of ("src: make interval sets work with string datatypes") otherwise new interval code complains with conflicting intervals. testcases/sets/sets_with_ifnames passes fine again. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: support to partial deletion with automergePablo Neira Ayuso2022-04-135-3/+257
| | | | | | | | | | | | | | | | | | | | | | | | Splice the existing set element cache with the elements to be deleted and merge sort it. The elements to be deleted are identified by the EXPR_F_REMOVE flag. The set elements to be deleted is automerged in first place if the automerge flag is set on. There are four possible deletion scenarios: - Exact match, eg. delete [a-b] and there is a [a-b] range in the kernel set. - Adjust left side of range, eg. delete [a-b] from range [a-x] where x > b. - Adjust right side of range, eg. delete [a-b] from range [x-b] where x < a. - Split range, eg. delete [a-b] from range [x-y] where x < a and b < y. Update nft_evaluate() to use the safe list variant since new commands are dynamically registered to the list to update ranges. This patch also restores the set element existence check for Linux kernels <= 5.7. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: allow for zero length rangesPablo Neira Ayuso2022-04-131-1/+1
| | | | | | | | | Allow for ranges such as, eg. 30-30. This is required by the new intervals.c code, which normalize constant, prefix set elements to all ranges. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: add support to automerge with kernel elementsPablo Neira Ayuso2022-04-137-40/+168
| | | | | | | | | | | | | | | | | | Extend the interval codebase to support for merging elements in the kernel with userspace element updates. Add a list of elements to be purged to cmd and set objects. These elements representing outdated intervals are deleted before adding the updated ranges. This routine splices the list of userspace and kernel elements, then it mergesorts to identify overlapping and contiguous ranges. This splice operation is undone so the set userspace cache remains consistent. Incrementally update the elements in the cache, this allows to remove dd44081d91ce ("segtree: Fix add and delete of element in same batch"). Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* mnl: update mnl_nft_setelem_del() to allow for more reusePablo Neira Ayuso2022-04-133-5/+6
| | | | | | Pass handle and element list as parameters to allow for code reuse. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: remove rbtree datastructurePablo Neira Ayuso2022-04-134-488/+0
| | | | | | Not used by anyone anymore, remove it. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: replace interval segment tree overlap and automergePablo Neira Ayuso2022-04-1310-667/+477
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a rewrite of the segtree interval codebase. This patch now splits the original set_to_interval() function in three routines: - add set_automerge() to merge overlapping and contiguous ranges. The elements, expressed either as single value, prefix and ranges are all first normalized to ranges. This elements expressed as ranges are mergesorted. Then, there is a linear list inspection to check for merge candidates. This code only merges elements in the same batch, ie. it does not merge elements in the kernela and the userspace batch. - add set_overlap() to check for overlapping set elements. Linux kernel >= 5.7 already checks for overlaps, older kernels still needs this code. This code checks for two conflict types: 1) between elements in this batch. 2) between elements in this batch and kernelspace. The elements in the kernel are temporarily merged into the list of elements in the batch to check for this overlaps. The EXPR_F_KERNEL flag allows us to restore the set cache after the overlap check has been performed. - set_to_interval() now only transforms set elements, expressed as range e.g. [a,b], to individual set elements using the EXPR_F_INTERVAL_END flag notation to represent e.g. [a,b+1), where b+1 has the EXPR_F_INTERVAL_END flag set on. More relevant updates: - The overlap and automerge routines are now performed in the evaluation phase. - The userspace set object representation now stores a reference to the existing kernel set object (in case there is already a set with this same name in the kernel). This is required by the new overlap and automerge approach. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: add EXPR_F_KERNEL to identify expression in the kernelPablo Neira Ayuso2022-04-133-1/+7
| | | | | | This allows to identify the set elements that reside in the kernel. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* segtree: add support for get element with sets that contain ifnamesFlorian Westphal2022-04-132-15/+65
| | | | | | | | | | | | | | | | nft get element inet filter s { bla, prefixfoo } table inet filter { set s { type ifname flags interval elements = { "prefixfoo*", "bla" } } Also add test cases for this. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* segtree: use correct byte order for 'element get'Florian Westphal2022-04-131-1/+2
| | | | | | | | Fails when the argument / set contains strings: we need to use host byte order if element has string base type. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: add testcases for interface names in setsFlorian Westphal2022-04-132-0/+111
| | | | | | | | | | | | | Add initial test case, sets with names and interfaces, anonymous and named ones. Check match+no-match. netns with ppp1 and ppq veth, send packets via both interfaces. Rule counters should have incremented on the three rules. (that match on set that have "abcdef1" or "abcdef*" strings in them). Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* segtree: add string "range" reversal supportFlorian Westphal2022-04-131-6/+41
| | | | | | | | | | | | | | | | | | | | Previous commits allows to use set key as a range, i.e. key ifname flags interval elements = { eth* } and then have it match on any interface starting with 'eth'. Listing is broken however, we need to reverse-translate the (128bit) number back to a string. 'eth*' is stored as interval 00687465 0000000 .. 00697465 0000000, i.e. "eth-eti", this adds the needed endianess fixups. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: make interval sets work with string datatypesFlorian Westphal2022-04-132-6/+32
| | | | | | | | | | | | | | | | | | | Allows to interface names in interval sets: table inet filter { set s { type ifname flags interval elements = { eth*, foo } } Concatenations are not yet supported, also, listing is broken, those strings will not be printed back because the values will remain in big-endian order. Followup patch will extend segtree to translate this back to host byte order. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: string prefix expression must retain original lengthFlorian Westphal2022-04-131-1/+3
| | | | | | | | | | | | | | | | To make something like "eth*" work for interval sets (match eth0, eth1, and so on...) we must treat the string as a 128 bit integer. Without this, segtree will do the wrong thing when applying the prefix, because we generate the prefix based on 'eth*' as input, with a length of 3. The correct import needs to be done on "eth\0\0\0\0\0\0\0...", i.e., if the input buffer were an ipv6 address, it should look like "eth\0::", not "::eth". Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* segtree: split prefix and range creation to a helper functionFlorian Westphal2022-04-131-43/+52
| | | | | | | No functional change intended. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: keep prefix expression lengthFlorian Westphal2022-04-132-0/+2
| | | | | | | | | | | | | Else, range_expr_value_high() will see a 0 length when doing: mpz_init_bitmask(tmp, expr->len - expr->prefix_len); This wasn't a problem so far because prefix expressions generated from "string*" were never passed down to the prefix->range conversion functions. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: make byteorder conversion on string base type a no-opFlorian Westphal2022-04-131-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prerequisite for support of interface names in interval sets: table inet filter { set s { type ifname flags interval elements = { "foo" } } chain input { type filter hook input priority filter; policy accept; iifname @s counter } } Will yield: "Byteorder mismatch: meta expected big endian, got host endian". This is because of: /* Data for range lookups needs to be in big endian order */ if (right->set->flags & NFT_SET_INTERVAL && byteorder_conversion(ctx, &rel->left, BYTEORDER_BIG_ENDIAN) < 0) It doesn't make sense to me to add checks to all callers of byteorder_conversion(), so treat this similar to EXPR_CONCAT and turn TYPE_STRING byteorder change into a no-op. Signed-off-by: Florian Westphal <fw@strlen.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Add meta time tests without 'meta' keywordMartin Gignac2022-04-103-0/+52
| | | | | | | | | v1.0.2 of 'nft' fails on 'time < "2022-07-01 11:00:00"' but succeeds when 'meta' is specified ('meta time < "2022-07-01 11:00:00"'). This extends coverage by testing 'time' without 'meta'. Signed-off-by: Martin Gignac <martin.gignac@gmail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* tests: py: Don't colorize output if stderr is redirectedPhil Sutter2022-04-081-1/+1
| | | | | | | | Cover for calls with '2>/tmp/log' and avoid printing escape sequences to that file. One could still keep colored output on stdout, but that required a printing routine for non-errors. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: monitor: Hide temporary file names from error outputPhil Sutter2022-04-081-2/+2
| | | | | | | | Make error output deterministic by passing input to nft via stdin. This way error messages will contain "/dev/stdin" instead of the temporary file name. Signed-off-by: Phil Sutter <phil@nwl.cc>
* tests: py: extend meta time coveragePablo Neira Ayuso2022-04-083-0/+52
| | | | | | Add meta time tests using < and > operands. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* meta: fix compiler warning in date_type_parse()Lukas Straub2022-04-051-4/+6
| | | | | | | | | | | | | After commit 0210097879 ("meta: time: use uint64_t instead of time_t") there is a compiler warning due to comparison of the return value from parse_iso_date with -1, which is now implicitly cast to uint64_t. Fix this by making parse_iso_date take a pointer to the tstamp and return bool instead. Fixes: 0210097879 ("meta: time: use uint64_t instead of time_t") Signed-off-by: Lukas Straub <lukasstraub2@web.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* meta: time: use uint64_t instead of time_tLukas Straub2022-04-051-1/+1
| | | | | | | | | | | | | time_t may be 32 bit on some platforms and thus can't fit a timestamp with nanoseconds resolution. This causes overflows and ultimatively breaks meta time expressions on such platforms. Fix this by using uint64_t instead. Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1567 Fixes: f8f32deda31df ("meta: Introduce new conditions 'time', 'day' and 'hour'") Signed-off-by: Lukas Straub <lukasstraub2@web.de> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* include: add missing `#include`Jeremy Sowden2022-04-051-0/+1
| | | | | | | datatype.h uses bool and so should include <stdbool.h>. Signed-off-by: Jeremy Sowden <jeremy@azazel.net> Signed-off-by: Florian Westphal <fw@strlen.de>
* examples: add .gitignore fileJeremy Sowden2022-04-051-0/+5
| | | | Signed-off-by: Florian Westphal <fw@strlen.de>
* tests: py: add inet/vmap testsPablo Neira Ayuso2022-03-294-0/+222
| | | | | | | Add a few tests with concatenations including raw and integer type expressions. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: Restore optimization for raw payload expressionsPablo Neira Ayuso2022-03-293-3/+63
| | | | | | | | This patch reverts d0f14b5337e7 ("optimize: do not merge raw payload expressions") after adding support for concatenation with variable length TYPE_INTEGER. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: allow to use integer type header fields via typeof set declarationPablo Neira Ayuso2022-03-294-7/+57
| | | | | | | | | | | | | | | Header fields such as udp length cannot be used in concatenations because it is using the generic integer_type: test.nft:3:10-19: Error: can not use variable sized data types (integer) in concat expressions typeof udp length . @th,32,32 ^^^^^^^^^^~~~~~~~~~~~~ This patch slightly extends ("src: allow to use typeof of raw expressions in set declaration") to set on NFTNL_UDATA_SET_KEY_PAYLOAD_LEN in userdata if TYPE_INTEGER is used. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: allow to use typeof of raw expressions in set declarationPablo Neira Ayuso2022-03-2912-33/+205
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the dynamic datatype to allocate an instance of TYPE_INTEGER and set length and byteorder. Add missing information to the set userdata area for raw payload expressions which allows to rebuild the set typeof from the listing path. A few examples: - With anonymous sets: nft add rule x y ip saddr . @ih,32,32 { 1.1.1.1 . 0x14, 2.2.2.2 . 0x1e } - With named sets: table x { set y { typeof ip saddr . @ih,32,32 elements = { 1.1.1.1 . 0x14 } } } Incremental updates are also supported, eg. nft add element x y { 3.3.3.3 . 0x28 } expr_evaluate_concat() is used to evaluate both set key definitions and set key values, using two different function might help to simplify this code in the future. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* expression: typeof verdict needs verdict datatypePablo Neira Ayuso2022-03-293-0/+7
| | | | | | | | | | | | | | | | | Otherwise listing breaks showing [invalid type] notice. # nft list ruleset table inet x { map y { typeof ip saddr : verdict elements = { 1.1.1.1 : 0x1010101 [invalid type] } } } Update tests to cover this usecase. Fixes: 4ab1e5e60779 ("src: allow use of 'verdict' in typeof definitions") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: copy field_count for anonymous object maps as wellFlorian Westphal2022-03-213-11/+39
| | | | | | | | | | without this test fails with: W: [FAILED] tests/shell/testcases/maps/anon_objmap_concat: got 134 BUG: invalid range expression type concat nft: expression.c:1452: range_expr_value_low: Assertion `0' failed. Signed-off-by: Florian Westphal <fw@strlen.de>
* rule: Avoid segfault with anonymous chainsPablo Neira Ayuso2022-03-171-0/+3
| | | | | | | | | | | | | Phil Sutter says: "When trying to add a rule which contains an anonymous chain to a non-existent chain, string_misspell_update() is called with a NULL string because the anonymous chain has no name. Avoid this by making the function NULL-pointer tolerant." Fixes: c330152b7f777 ("src: support for implicit chain bindings") Reported-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: init cmd pointer for new on-stack contextFlorian Westphal2022-03-042-0/+7
| | | | | | | else, this will segfault when trying to print the "table 'x' doesn't exist" error message. Signed-off-by: Florian Westphal <fw@strlen.de>
* optimize: do not assume log prefixPablo Neira Ayuso2022-03-043-3/+14
| | | | | | ... log prefix might not be present in log statements. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: do not merge unsupported statement expressionsPablo Neira Ayuso2022-03-031-0/+21
| | | | | | Only value, range, prefix, set and list are supported at this stage. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: incorrect assert() for unexpected expression typePablo Neira Ayuso2022-03-031-3/+3
| | | | | | | assert(1) is noop, this should be assert(0) instead. Fixes: 561aa3cfa8da ("optimize: merge verdict maps with same lookup key") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: more robust statement merge with vmapPablo Neira Ayuso2022-03-033-3/+18
| | | | | | | | | | | | | | Check expressions that are expected on the rhs rather than using a catch-all default case. Actually, lists and sets need to be their own routine, because this needs the set element key expression to be merged. This is a follow up to 99eb46969f3d ("optimize: fix vmap with anonymous sets"). Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: fix vmap with anonymous setsPablo Neira Ayuso2022-03-033-2/+14
| | | | | | | | | | | | | | | | The following example ruleset crashes: table inet a { chain b { tcp dport { 1 } accept tcp dport 2-3 drop } } because handling for EXPR_SET is missing. Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* scanner: Fix for ipportmap nat statementsPhil Sutter2022-03-021-1/+1
| | | | | | | | Due to lookahead, "addr" keyword is still found in IP/IP6 scope, not STMT_NAT one. Fixes: a67fce7ffe7e4 ("scanner: nat: Move to own scope") Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: dup, fwd, tproxy: Move to own scopesPhil Sutter2022-03-013-7/+16
| | | | | | With these three scopes in place, keyword 'to' may be isolated. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: meta: Move to own scopePhil Sutter2022-03-013-7/+10
| | | | | | | This allows to isolate 'length' and 'protocol' keywords shared by other scopes as well. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: at: Move to own scopePhil Sutter2022-03-013-10/+15
| | | | | | | Modification of raw TCP option rule is a bit more complicated to avoid pushing tcp_hdr_option_type into the introduced scope by accident. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: nat: Move to own scopePhil Sutter2022-03-013-15/+20
| | | | | | | | | | Unify nat, masquerade and redirect statements, they widely share their syntax. Note the workaround of adding "prefix" to SCANSTATE_IP. This is required to fix for 'snat ip prefix ...' style expressions. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: policy: move to own scopePhil Sutter2022-03-013-6/+11
| | | | | | Isolate 'performance' and 'memory' keywords. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: flags: move to own scopePhil Sutter2022-03-013-20/+26
| | | | | | This isolates at least 'constant', 'dynamic' and 'all' keywords. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: reject: Move to own scopePhil Sutter2022-03-013-4/+9
| | | | | | Two more keywords isolated. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: import, export: Move to own scopesPhil Sutter2022-03-013-7/+15
| | | | | | | In theory, one could use a common scope for both import and export commands, their parameters are identical. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: reset: move to own ScopePhil Sutter2022-03-013-6/+11
| | | | | | Isolate two more keywords shared with list command. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: monitor: Move to own ScopePhil Sutter2022-03-013-7/+14
| | | | | | Some keywords are shared with list command. Signed-off-by: Phil Sutter <phil@nwl.cc>
* scanner: rt: Extend scope over rt0, rt2 and srhPhil Sutter2022-03-012-9/+9
| | | | | | | These are technically all just routing headers with different types, so unify them under the same scope. Signed-off-by: Phil Sutter <phil@nwl.cc>