summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* libnftables: release top level scopePablo Neira Ayuso2022-06-231-0/+2
| | | | | | | | | | Otherwise bogus variable redefinition are reported via -o/--optimize: redefinition.conf:5:8-21: Error: redefinition of symbol 'interface_inet' define interface_inet = enp5s0 ^^^^^^^^^^^^^^ Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: limit statement is not supported yetPablo Neira Ayuso2022-06-231-12/+0
| | | | | | | | Revert support for limit statement, the limit statement is stateful and it applies a ratelimit per rule, transformation for merging rules with the limit statement needs to use anonymous sets with statements. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: assume verdict is same when rules have no verdictPablo Neira Ayuso2022-06-231-1/+2
| | | | Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: only merge OP_IMPLICIT and OP_EQ relationalPablo Neira Ayuso2022-06-231-0/+10
| | | | | | Add test to cover this case. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add unsupported statementPablo Neira Ayuso2022-06-231-4/+52
| | | | | | | | | | | | | | | | Do not try to merge rules with unsupported statements. This patch adds a dummy unsupported statement which is included in the statement collection and the rule vs statement matrix. When looking for possible rule mergers, rules using unsupported statements are discarded, otherwise bogus rule mergers might occur. Note that __stmt_type_eq() already returns false for unsupported statements. Add a test using meta mark statement, which is not yet supported. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add hash expression supportPablo Neira Ayuso2022-06-231-0/+12
| | | | | | Extend expr_cmp() to compare hash expressions used in relational. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add numgen expression supportPablo Neira Ayuso2022-06-231-0/+8
| | | | | | Extend expr_cmp() to compare numgen expressions used in relational. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add binop expression supportPablo Neira Ayuso2022-06-231-0/+2
| | | | | | | Do recursive call using left expression in the binop expression tree to search for the primary expression. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add fib expression supportPablo Neira Ayuso2022-06-231-0/+6
| | | | | | Extend expr_cmp() to compare fib expressions used in relational. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add xfrm expression supportPablo Neira Ayuso2022-06-231-0/+6
| | | | | | Extend expr_cmp() to compare xfrm expressions used in relational. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: add osf expression supportPablo Neira Ayuso2022-06-231-0/+6
| | | | | | Extend expr_cmp() to compare osf expressions used in relational. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: fix verdict map mergingPablo Neira Ayuso2022-06-231-4/+7
| | | | | | | | | | Skip comparison when collecting the statement and building the rule vs statement matrix. Compare verdict type when merging rules. When infering rule mergers, honor the STMT_VERDICT with map (ie. vmap). Fixes: 561aa3cfa8da ("optimize: merge verdict maps with same lookup key") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: fix reject statementPablo Neira Ayuso2022-06-231-3/+16
| | | | | | | | Add missing code to the statement collection routine. Compare reject expressions when available. Add tests/shell. Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: remove comment after mergingPablo Neira Ayuso2022-06-231-0/+5
| | | | | | | | | Remove rule comment after merging rules, let the user decide if they want to reintroduce the comment in the ruleset file. Update optimizations/merge_stmt test. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: do not print stateful informationPablo Neira Ayuso2022-06-231-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Do not print stateful information such as counters which are likely set to zero. Before this patch: Merging: packets.conf:10:3-29: ip protocol 4 counter drop packets.conf:11:3-29: ip protocol 41 counter drop packets.conf:12:3-29: ip protocol 47 counter drop into: ip protocol { 4, 41, 47 } counter packets 0 bytes 0 drop ^^^^^^^^^^^^^^^^^ After: Merging: packets.conf:10:3-29: ip protocol 4 counter drop packets.conf:11:3-29: ip protocol 41 counter drop packets.conf:12:3-29: ip protocol 47 counter drop into: ip protocol { 4, 41, 47 } counter drop Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: do not merge rules with set reference in rhsPablo Neira Ayuso2022-06-231-0/+10
| | | | | | | Otherwise set reference ends up included in an anonymous set, as an element, which is not supported. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: do not compare relational expression rhs when collecting statementsPablo Neira Ayuso2022-06-231-18/+21
| | | | | | | | | | | When building the statement matrix, do not compare expression right hand side, otherwise bogus mismatches might occur. The fully compared flag is set on when comparing rules to look for possible mergers. Fixes: 3f36cc6c3dcd ("optimize: do not merge unsupported statement expressions") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: Do not sort cached set elements over and over againPhil Sutter2022-06-192-25/+23
| | | | | | | | | | | | | | | | | | | | When adding element(s) to a non-empty set, code merged the two lists and sorted the result. With many individual 'add element' commands this causes substantial overhead. Make use of the fact that existing_set->init is sorted already, sort only the list of new elements and use list_splice_sorted() to merge the two sorted lists. Add set_sort_splice() and use it for set element overlap detection and automerge. A test case adding ~25k elements in individual commands completes in about 1/4th of the time with this patch applied. Joint work with Pablo. Fixes: 3da9643fb9ff9 ("intervals: add support to automerge with kernel elements") Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: do not empty cache for mapsPablo Neira Ayuso2022-06-191-3/+6
| | | | | | | | Translate set element to range and sort in maps for the NFT_SET_MAP case, which does not support for automerge yet. Fixes: 81e36530fcac ("src: replace interval segment tree overlap and automerge") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: do not report exact overlaps for new elementsPablo Neira Ayuso2022-06-191-2/+1
| | | | | | | | | | | | | | | | | | | | Two new elements that represent an exact overlap should not trigger an error. add table t add set t s { type ipv4_addr; flags interval; } add element t s { 1.0.1.0/24 } ... add element t s { 1.0.1.0/24 } result in a bogus error. # nft -f set.nft set.nft:1002:19-28: Error: conflicting intervals specified add element t s { 1.0.1.0/24 } ^^^^^^^^^^ Fixes: 3da9643fb9ff ("intervals: add support to automerge with kernel elements") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* rule: collapse set element commandsPablo Neira Ayuso2022-06-192-3/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Robots might generate a long list of singleton element commands such as: add element t s { 1.0.1.0/24 } ... add element t s { 1.0.2.0/23 } collapse them into one single command before the evaluation step, ie. add element t s { 1.0.1.0/24, ..., 1.0.2.0/23 } this speeds up overlap detection and set element automerge operations in this worst case scenario. Since 3da9643fb9ff9 ("intervals: add support to automerge with kernel elements"), the new interval tracking relies on mergesort. The pattern above triggers the set sorting for each element. This patch adds a list to cmd objects that store collapsed commands. Moreover, expressions also contain a reference to the original command, to uncollapse the commands after the evaluation step. These commands are uncollapsed after the evaluation step to ensure error reporting works as expected (command and netlink message are mapped 1:1). For the record: - nftables versions <= 1.0.2 did not perform any kind of overlap check for the described scenario above (because set cache only contained elements in the kernel in this case). This is a problem for kernels < 5.7 which rely on userspace to detect overlaps. - the overlap detection could be skipped for kernels >= 5.7. - The extended netlink error reporting available for set elements since 5.19-rc might allow to remove the uncollapse step, in this case, error reporting does not rely on the netlink sequence to refer to the command triggering the problem. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* Revert "scanner: flags: move to own scope"Florian Westphal2022-06-102-26/+21
| | | | | | | | | | | | | | | | | | | | | | | Excess nesting of scanner scopes is very fragile and error prone: rule `iif != lo ip daddr 127.0.0.1/8 counter limit rate 1/second log flags all prefix "nft_lo4 " drop` fails with `Error: No symbol type information` hinting at `prefix` Problem is that we nest via: counter limit log flags By the time 'prefix' is scanned, state is still stuck in 'counter' due to this nesting. Working around "prefix" isn't enough, any other keyword, e.g. "level" in 'flags all level debug' will be parsed as 'string' too. So, revert this. Fixes: a16697097e2b ("scanner: flags: move to own scope") Reported-by: Christian Göttsche <cgzones@googlemail.com> Signed-off-by: Florian Westphal <fw@strlen.de>
* evaluate: reset ctx->set after set interval evaluationPablo Neira Ayuso2022-06-011-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise bogus error reports on set datatype mismatch might occur, such as: Error: datatype mismatch, expected Internet protocol, expression has type IPv4 address meta l4proto { tcp, udp } th dport 443 dnat to 10.0.0.1 ~~~~~~~~~~~~ ^^^^^^^^^^^^ with an unrelated set declaration. table ip test { set set_with_interval { type ipv4_addr flags interval } chain prerouting { type nat hook prerouting priority dstnat; policy accept; meta l4proto { tcp, udp } th dport 443 dnat to 10.0.0.1 } } This bug has been introduced in the evaluation step. Reported-by: Roman Petrov <nwhisper@gmail.com> Fixes: 81e36530fcac ("src: replace interval segment tree overlap and automerge)" Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: segfault when releasing unsupported statementPablo Neira Ayuso2022-06-011-1/+1
| | | | | | | | | Call xfree() instead since stmt_alloc() does not initialize the statement type fields. Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1609 Fixes: ea1f1c9ff608 ("optimize: memleak in statement matrix") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* nft: simplify chain lookup in do_list_chainChander Govindarajan2022-05-311-6/+2
| | | | | | | | use the chain_cache_find function for faster lookup of chain instead of iterating over all chains in table Signed-off-by: ChanderG <mail@chandergovind.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: fix compilation --with-mini-gmpPablo Neira Ayuso2022-05-301-6/+6
| | | | | | | | | | | | Use pr_gmp_debug() instead to compile with minigmp. intervals.c: In function ‘set_delete’: intervals.c:489:25: warning: implicit declaration of function ‘gmp_printf’; did you mean ‘gmp_vfprintf’? [-Wimplicit-function-declaration] 489 | gmp_printf("remove: [%Zx-%Zx]\n", | ^~~~~~~~~~ | gmp_vfprintf Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* json: update json output ordering to place rules after chainsChander Govindarajan2022-05-241-2/+5
| | | | | | | | | | | | | | | | | | Currently the json output of `nft -j list ruleset` interleaves rules with chains. As reported in this bug: https://bugzilla.netfilter.org/show_bug.cgi?id=1580 the json cannot be fed into `nft -j -f <file>` since rules may reference chains that are created later Instead create rules after all chains are output. Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1580 Signed-off-by: ChanderG <mail@chandergovind.org> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* netlink_delinearize: release last register on exitPablo Neira Ayuso2022-05-161-1/+1
| | | | | | | | | | | | | | netlink_release_registers() does not release the expression in the last 32-bit register. struct netlink_parse_ctx { ... struct expr *registers[MAX_REGS + 1]; This array is MAX_REGS + 1 (verdict register + 16 32-bit registers). Fixes: 371c3a0bc3c2 ("netlink_delinearize: release expressions in context registers") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* segtree: add pretty-print support for wildcard strings in concatenated setsFlorian Westphal2022-05-091-2/+36
| | | | | | | | | | For concat ranges, something like 'ppp*' is translated as a range from 'ppp\0\0\0...' to 'ppp\ff\ff\ff...'. In order to display this properly, check for presence of string base type and convert to symbolic expression, with appended '*' character. Signed-off-by: Florian Westphal <fw@strlen.de>
* netlink: swap byteorder for host-endian concat dataFlorian Westphal2022-05-091-0/+4
| | | | | | | | | | | All data must be passed in network byte order, else matching won't work respectively kernel will reject the interval because it thinks that start is after end This is needed to allow use of 'ppp*' in interval sets with concatenations. Signed-off-by: Florian Westphal <fw@strlen.de>
* intervals: deletion should adjust range not yet in the kernelPablo Neira Ayuso2022-05-071-3/+0
| | | | | | | | | | | | | | | Do not remove the range if it does not exists yet in the kernel, adjust it instead. Uncovered by use-after-free error. ==276702==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d00190663c at pc 0x7ff310ab526f bp 0x7fffeb76f750 sp 0x7fffeb76f748 READ of size 4 at 0x60d00190663c thread T0 #0 0x7ff310ab526e in __adjust_elem_right .../nftables/src/intervals.c:300 #1 0x7ff310ab59a7 in adjust_elem_right .../nftables/src/intervals.c:311 #2 0x7ff310ab6daf in setelem_adjust .../nftables/src/intervals.c:354 #3 0x7ff310ab783a in setelem_delete .../nftables/src/intervals.c:411 #4 0x7ff310ab80e6 in __set_delete .../nftables/src/intervals.c:451 Fixes: 3e8d934e4f72 ("intervals: support to partial deletion with automerge") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: memleak in statement matrixPablo Neira Ayuso2022-05-041-0/+1
| | | | | | | Release clone object in case this statement is not supported. Fixes: 743b0e81371f ("optimize: do not clone unsupported statement") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: merge nat rules with same selectors into mapPablo Neira Ayuso2022-05-031-10/+194
| | | | | | | | | | | | | | | | | | | | | | | Verdict and nat are mutually exclusive, no need to support for this combination. # cat ruleset.nft table ip x { chain y { type nat hook postrouting priority srcnat; policy drop; ip saddr 1.1.1.1 tcp dport 8000 snat to 4.4.4.4:80 ip saddr 2.2.2.2 tcp dport 8001 snat to 5.5.5.5:90 } } # nft -o -c -f ruleset.nft Merging: ruleset.nft:4:3-52: ip saddr 1.1.1.1 tcp dport 8000 snat to 4.4.4.4:80 ruleset.nft:5:3-52: ip saddr 2.2.2.2 tcp dport 8001 snat to 5.5.5.5:90 into: snat to ip saddr . tcp dport map { 1.1.1.1 . 8000 : 4.4.4.4 . 80, 2.2.2.2 . 8001 : 5.5.5.5 . 90 } Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: do not clone unsupported statementPablo Neira Ayuso2022-05-031-1/+1
| | | | | | | | Skip unsupported statements when building the statement matrix, otherwise clone remains uninitialized. Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* optimize: incorrect logic in verdict comparisonPablo Neira Ayuso2022-05-031-4/+6
| | | | | | | | Keep inspecting rule verdicts before assuming they are equal. Update existing test to catch this bug. Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: fix always-true assertionsFlorian Westphal2022-04-262-3/+4
| | | | | | | assert(1) is a no-op, this should be assert(0). Use BUG() instead. Add missing CATCHALL to avoid BUG(). Signed-off-by: Florian Westphal <fw@strlen.de>
* intervals: set on EXPR_F_KERNEL flag for new elements in set cachePablo Neira Ayuso2022-04-181-0/+3
| | | | | | | | | So follow up command in this batch that update the set assumes this element is already in the kernel. Fixes: 3da9643fb9ff ("intervals: add support to automerge with kernel elements") Fixes: 3ed9fadaab95 ("intervals: build list of elements to be added from cache") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: allow use of base integer types as set keys in concatenationsFlorian Westphal2022-04-181-7/+17
| | | | | | | | | | | | | | | | "typeof ip saddr . ipsec in reqid" won't work because reqid uses integer type, i.e. dtype->size is 0. With "typeof", the size can be derived from the expression length, via set->key. This computes the concat length based either on dtype->size or expression length. It also updates concat evaluation to permit a zero datatype size if the subkey expression has nonzero length (i.e., typeof was used). Signed-off-by: Florian Westphal <fw@strlen.de>
* intervals: build list of elements to be added from cachePablo Neira Ayuso2022-04-181-40/+30
| | | | | | | | | | | Loop over the set cache and add elements that have no EXPR_F_KERNEL, meaning that these are new elements in the set that have resulted from adjusting/split existing ranges. This fixes several partial deletions of the same interval in one command. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: fix deletion of multiple ranges with automergePablo Neira Ayuso2022-04-181-12/+22
| | | | | | | | | Iterate over the list of elements to be deleted, then splice one EXPR_F_REMOVE element at a time to update the list of existing sets incrementally. Fixes: 3e8d934e4f722 ("intervals: support to partial deletion with automerge") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: add elements with EXPR_F_KERNEL to purge list onlyPablo Neira Ayuso2022-04-181-3/+7
| | | | | | | | Do not add elements to purge list which are not in the kernel, otherwise, bogus ENOENT is reported. Fixes: 3e8d934e4f722 ("intervals: support to partial deletion with automerge") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* netlink: remove unused argument from helper functionFlorian Westphal2022-04-181-3/+3
| | | | Signed-off-by: Florian Westphal <fw@strlen.de>
* intervals: Simplify element sanity checksPhil Sutter2022-04-141-16/+10
| | | | | | | | | | Since setelem_delete() assigns to 'prev' pointer only if it doesn't have EXPR_F_REMOVE flag set, there is no need to check that flag in called functions. Fixes: 3e8d934e4f722 ("intervals: support to partial deletion with automerge") Signed-off-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: unset EXPR_F_KERNEL for adjusted elementsPablo Neira Ayuso2022-04-131-3/+3
| | | | | | | | | | | | | | | | This element is adjusted, reset the EXPR_F_KERNEL flag, this is a new element and the old is purged from the kernel. The existing list of elements in the kernel is spliced to the elements to be removed, then merge-sorted. The EXPR_F_REMOVE flag specifies that this element represents a deletion. The EXPR_F_REMOVE and EXPR_F_KERNEL allows to track objects: whether element is in the kernel (EXPR_F_KERNEL), element is new (no flag) or element represents a removal (EXPR_F_REMOVE). Reported-by: Phil Sutter <phil@nwl.cc> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: restore interval sets work with string datatypesPablo Neira Ayuso2022-04-132-6/+8
| | | | | | | | | | | Switch byteorder of string datatypes to host byteorder. Partial revert of ("src: make interval sets work with string datatypes") otherwise new interval code complains with conflicting intervals. testcases/sets/sets_with_ifnames passes fine again. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: support to partial deletion with automergePablo Neira Ayuso2022-04-133-3/+254
| | | | | | | | | | | | | | | | | | | | | | | | Splice the existing set element cache with the elements to be deleted and merge sort it. The elements to be deleted are identified by the EXPR_F_REMOVE flag. The set elements to be deleted is automerged in first place if the automerge flag is set on. There are four possible deletion scenarios: - Exact match, eg. delete [a-b] and there is a [a-b] range in the kernel set. - Adjust left side of range, eg. delete [a-b] from range [a-x] where x > b. - Adjust right side of range, eg. delete [a-b] from range [x-b] where x < a. - Split range, eg. delete [a-b] from range [x-y] where x < a and b < y. Update nft_evaluate() to use the safe list variant since new commands are dynamically registered to the list to update ranges. This patch also restores the set element existence check for Linux kernels <= 5.7. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: allow for zero length rangesPablo Neira Ayuso2022-04-131-1/+1
| | | | | | | | | Allow for ranges such as, eg. 30-30. This is required by the new intervals.c code, which normalize constant, prefix set elements to all ranges. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* intervals: add support to automerge with kernel elementsPablo Neira Ayuso2022-04-134-39/+129
| | | | | | | | | | | | | | | | | | Extend the interval codebase to support for merging elements in the kernel with userspace element updates. Add a list of elements to be purged to cmd and set objects. These elements representing outdated intervals are deleted before adding the updated ranges. This routine splices the list of userspace and kernel elements, then it mergesorts to identify overlapping and contiguous ranges. This splice operation is undone so the set userspace cache remains consistent. Incrementally update the elements in the cache, this allows to remove dd44081d91ce ("segtree: Fix add and delete of element in same batch"). Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* mnl: update mnl_nft_setelem_del() to allow for more reusePablo Neira Ayuso2022-04-132-4/+4
| | | | | | Pass handle and element list as parameters to allow for code reuse. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: remove rbtree datastructurePablo Neira Ayuso2022-04-132-389/+0
| | | | | | Not used by anyone anymore, remove it. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>