| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
nomerge_vmap gains a nodump file, the test uses --check.
Signed-off-by: Florian Westphal <fw@strlen.de>
|
|
|
|
|
|
|
|
| |
The code obviously intended to do this already but got the array length
check wrong.
Fixes: e70354f53e9f6 ("libnftables: Implement JSON output support")
Signed-off-by: Phil Sutter <phil@nwl.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-o/--optimize results in EEXIST error when merging two rules that lead
to ambiguous set/map, for instance:
table ip x {
chain v4icmp {}
chain v4icmpc {}
chain y {
ip protocol icmp jump v4icmp
ip protocol icmp goto v4icmpc
}
}
which is not possible because duplicated keys are not possible in
set/map. This is how it shows when running a test:
Merging:
testcases/sets/dumps/sets_with_ifnames.nft:56:3-30: ip protocol icmp jump v4icmp
testcases/sets/dumps/sets_with_ifnames.nft:57:3-31: ip protocol icmp goto v4icmpc
into:
ip protocol vmap { icmp : jump v4icmp, icmp : goto v4icmpc }
internal:0:0-0: Error: Could not process rule: File exists
Add a new step to compare rules that are candidate to be merged to
detect colissions in set/map keys in order to skip them in the next
final merging step.
Add tests/shell unit to improve coverage.
Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following rules:
udp dport 137 ct state new,untracked accept
udp dport 138 ct state new,untracked accept
results in:
nft: src/optimize.c:670: __merge_concat: Assertion `0' failed.
The logic to expand to the new,untracked list in the concatenation is
missing.
Fixes: 187c6d01d357 ("optimize: expand implicit set element when merging into concatenation")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Check if right hand side of relational is a bitmask, ie.
relational
/ \
... or
/ \
value or
/ \
value value
then, if left hand side is a binop expression, compare left and right
hand sides (not only left hand of this binop expression) to check for
redundant matches in consecutive rules, ie.
relational
/ \
and ...
/ \
payload value
before this patch, only payload in the binop expression was compared.
This allows to compact several rules matching tcp flags in a set/map, eg.
# nft -c -o -f ruleset.nft
Merging:
ruleset.nft:7:17-76: tcp flags & (fin | syn | rst | ack | urg) == fin | ack | urg
ruleset.nft:8:17-70: tcp flags & (fin | syn | rst | ack | urg) == fin | ack
ruleset.nft:9:17-64: tcp flags & (fin | syn | rst | ack | urg) == fin
ruleset.nft:10:17-70: tcp flags & (fin | syn | rst | ack | urg) == syn | ack
ruleset.nft:11:17-64: tcp flags & (fin | syn | rst | ack | urg) == syn
ruleset.nft:12:17-70: tcp flags & (fin | syn | rst | ack | urg) == rst | ack
ruleset.nft:13:17-64: tcp flags & (fin | syn | rst | ack | urg) == rst
ruleset.nft:14:17-70: tcp flags & (fin | syn | rst | ack | urg) == ack | urg
ruleset.nft:15:17-64: tcp flags & (fin | syn | rst | ack | urg) == ack
into:
tcp flags & (fin | syn | rst | ack | urg) == { fin | ack | urg, fin | ack, fin, syn | ack, syn, rst | ack, rst, ack | urg, ack }
Merging:
ruleset.nft:17:17-61: tcp flags & (ack | urg) == ack jump ack_chain
ruleset.bft:18:17-61: tcp flags & (ack | urg) == urg jump urg_chain
into:
tcp flags & (ack | urg) vmap { ack : jump ack_chain, urg : jump urg_chain }
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
| |
do not merge raw payload expressions with different length.
Other expression rely on key comparison which is assumed to have the
same length already.
Fixes: 60dcc01d6351 ("optimize: add __expr_cmp()")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Do not hit assert():
nft: optimize.c:486: rule_build_stmt_matrix_stmts: Assertion `k >= 0' failed.
variables are not supported by -o/--optimize at this stage.
Fixes: 9be404a153bc ("optimize: ignore existing nat mapping")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
The counter statement that is zapped from the rule needs to be cloned
before inserting it into each set element.
Fixes: 686ab8b6996e ("optimize: do not remove counter in verdict maps")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Update tests which contain unsupported features in older kernels.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
Ordering of 'nft -j list ruleset' output has changed, Regenerate
existing json-nft dumps. No functional change intended, merely the
position of chain objects should have moved up in the "nftables" array.
Signed-off-by: Phil Sutter <phil@nwl.cc>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Those are expected to be stable, so add them.
Some are not 100% correct, as "typeof" is misprinted as "type" (json output
and input parser lack support for this), but for these files the "type"
is valid too.
This will allow better validation once proper "typeof" support is
added to json.c and json-parser.c.
Signed-off-by: Florian Westphal <fw@strlen.de>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The problem with single line output as produced by 'nft -j list ruleset'
is its incompatibility to unified diff format as any change in this
single line will produce a diff which contains the old and new lines in
total. This is not just unreadable but will blow up patches which may
exceed mailinglists' mail size limits.
Convert them all at once by feeding their contents to
tests/shell/helpers/json-pretty.sh.
Signed-off-by: Phil Sutter <phil@nwl.cc>
|
|
|
|
|
|
|
|
|
| |
The "handle" in JSON output is not stable. Sanitize/normalize to zero.
Adjust the sanitize code, and regenerate the .json-nft files.
Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Detach nat inet from existing tests not to reduce test coverage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Generate and add ".json-nft" files. These files contain the output of
`nft -j list ruleset` after the test. Also, "test-wrapper.sh" will
compare the current ruleset against the ".json-nft" files and test them
with "nft -j --check -f $FILE`. These are useful extra tests, that we
almost get for free.
Note that for some JSON dumps, `nft -f --check` fails (or prints
something). For those tests no *.json-nft file is added. The bugs needs
to be fixed first.
An example of such an issue is:
$ DUMPGEN=all ./tests/shell/run-tests.sh tests/shell/testcases/maps/nat_addr_port
which gives a file "rc-failed-chkdump" with
Command `./tests/shell/../../src/nft -j --check -f "tests/shell/testcases/maps/dumps/nat_addr_port.json-nft"` failed
>>>>
internal:0:0-0: Error: Invalid map type 'ipv4_addr . inet_service'.
internal:0:0-0: Error: Parsing command array at index 3 failed.
internal:0:0-0: Error: unqualified type integer specified in map definition. Try "typeof expression" instead of "type datatype".
<<<<
Tests like "tests/shell/testcases/nft-f/0012different_defines_0" and
"tests/shell/testcases/nft-f/0024priority_0" also don't get a .json-nft
dump yet, because their output is not stable. That needs fixing too.
Cc: Pablo Neira Ayuso <pablo@netfilter.org>
Cc: Florian Westphal <fw@strlen.de>
Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
|
|
|
|
|
|
|
| |
One without pipapo support and another with not to harm existing
coverage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
| |
Split this to move set stateful expression support into a separated test
not to harm existing coverage.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Skip tests that require stateful expressions in sets.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Skip tests that require reject at prerouting hook.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Skip tests that require net/netfilter/nft_set_pipapo support.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The file "optimizations/dumps/single_anon_set.nft.input" was laying
around, and it was unclear how it was used.
Let's extend "check-patch.sh" to flag all unused files. But the script
cannot understand how "single_anon_set.nft.input" is used (aside allow
listing it).
Instead, inline the script to keep it inside the test (script).
We still write the data to a separate file and don't use `nft -f -`
(because reading stdin uses a different code path we want to cover).
Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Several tests didn't have a ".nft" dump file committed. Generate one and
commit it to git.
While not all tests have a stable ruleset to compare, many have. Commit
the .nft files for the tests where the output appears to be stable.
This was generated by running `./tests/shell/run-tests.sh -g` twice, and
commit the files that were identical both times. Note that 7 tests on my
machine fail, so those are skipped.
Also skip the files
tests/shell/testcases/maps/dumps/0004interval_map_create_once_0.nft
tests/shell/testcases/nft-f/dumps/0011manydefines_0.nft
tests/shell/testcases/sets/dumps/0011add_many_elements_0.nft
tests/shell/testcases/sets/dumps/0030add_many_elements_interval_0.nft
tests/shell/testcases/sets/dumps/0068interval_stack_overflow_0.nft
Those files are larger than 100KB, and I don't think we want to blow up
the git repository this way. Even if they are only text files and
compress well.
Signed-off-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Extend e0aace943412 ("libnftables: Drop cache in error case") to also
drop the cache with -c/--check, this is a dry run mode and kernel does
not get any update.
This fixes a bug with -o/--optimize, which first runs in an implicit
-c/--check mode to validate that the ruleset is correct, then it
provides the proposed optimization. In this case, if the cache is not
emptied, old objects in the cache refer to scanner data that was
already released, which triggers BUG like this:
BUG: invalid input descriptor type 151665524
nft: erec.c:161: erec_print: Assertion `0' failed.
Aborted
This bug was triggered in a ruleset that contains a set for geoip
filtering. This patch also extends tests/shell to cover this case.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add counter to set element instead of dropping it:
# nft -c -o -f test.nft
Merging:
test.nft:6:3-50: ip saddr 1.1.1.1 ip daddr 2.2.2.2 counter accept
test.nft:7:3-48: ip saddr 1.1.1.2 ip daddr 3.3.3.3 counter drop
into:
ip daddr . ip saddr vmap { 2.2.2.2 . 1.1.1.1 counter : accept, 3.3.3.3 . 1.1.1.2 counter : drop }
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
fee6bda06403 ("evaluate: remove anon sets with exactly one element")
introduces an optimization to remove use of sets with single element.
Skip this optimization if set element contains stateful statements.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The redirect and masquerade statements can be handled as verdicts:
- if redirect statement specifies no ports.
- masquerade statement, in any case.
Exceptions to the rule: If redirect statement specifies ports, then nat
map transformation can be used iif both statements specify ports.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1668
Fixes: 0a6dbfce6dc3 ("optimize: merge nat rules with same selectors into map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Infer family from key in nat mapping, otherwise nat mapping via merge
breaks since family is not specified.
Merging:
fw-test-bug2.nft:4:9-78: iifname enp2s0 ip daddr 72.2.3.66 tcp dport 53122 dnat to 10.1.1.10:22
fw-test-bug2.nft:5:9-77: iifname enp2s0 ip daddr 72.2.3.66 tcp dport 443 dnat to 10.1.1.52:443
fw-test-bug2.nft:6:9-75: iifname enp2s0 ip daddr 72.2.3.70 tcp dport 80 dnat to 10.1.1.52:80
into:
dnat ip to iifname . ip daddr . tcp dport map { enp2s0 . 72.2.3.66 . 53122 : 10.1.1.10 . 22, enp2s0 . 72.2.3.66 . 443 : 10.1.1.52 . 443, enp2s0 . 72.2.3.70 . 80 : 10.1.1.52 . 80 }
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1657
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
User might be already using a nat mapping in their ruleset, use the
unsupported statement when collecting statements in this case.
# nft -c -o -f ruleset.nft
nft: optimize.c:443: rule_build_stmt_matrix_stmts: Assertion `k >= 0' failed.
Aborted
The -o/--optimize feature only cares about linear rulesets at this
stage, but do not hit assert() in this case.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1656
Fixes: 0a6dbfce6dc3 ("optimize: merge nat rules with same selectors into map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Select the merge criteria based on the statements that are used
in the candidate rules, instead of using the list of statements
in the given chain.
Update tests to include a rule with a verdict, which triggers
the bug described in the bugzilla ticket.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1657
Fixes: 0a6dbfce6dc3 ("optimize: merge nat rules with same selectors into map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
# nft -c -o -f ruleset.nft
Merging:
ruleset.nft:3:3-53: meta pkttype broadcast udp dport { 67, 547 } accept
ruleset.nft:4:17-58: meta pkttype multicast udp dport 1900 drop
into:
meta pkttype . udp dport vmap { broadcast . { 67, 547 } : accept, multicast . 1900 : drop }
ruleset.nft:3:38-39: Error: invalid data type, expected concatenation of (packet type, internet network service)
meta pkttype broadcast udp dport { 67, 547 } accept
^^
Similar to 187c6d01d357 ("optimize: expand implicit set element when
merging into concatenation") but for verdict maps.
Reported-by: Simon G. Trajkovski <neur0armitage@proton.me>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following ruleset fails to be merged using set + concatenation:
meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.3.0/24 accept
meta iifname eth1 ip saddr 1.1.1.2 ip daddr 2.2.4.0-2.2.4.10 accept
hitting the following assertion:
nft: optimize.c:585: __merge_concat_stmts: Assertion `0' failed.
Abort
This patch also updates tests/shell.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
| |
Use datatype_equal(), otherwise dynamically allocated datatype fails
to fulfill the datatype pointer check, triggering the assertion:
nft: evaluate.c:1249: expr_evaluate_binop: Assertion `expr_basetype(left) == expr_basetype(right)' failed.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1636
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Generalize the existing code to deal with implicit sets. When merging a
ruleset like the following:
udp dport 128 iifname "foo" #1
udp dport { 67, 123 } iifname "bar" #2
into a concatenation of statements, the following expansion need to
be done for rule #2:
67 . "bar"
123 . "bar"
The expansion logic consists of cloning the existing concatenation being
built and then append each element in the implicit set. A list of
ongoing concatenations being built is maintained, so further expansions
are also supported.
Extend test to cover for this use-case.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1628
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Rules that are equal need to have at least one mergeable statement.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Existing concatenation cannot be merge at this stage, skip them
otherwise this assertion is hit:
nft: optimize.c:434: rule_build_stmt_matrix_stmts: Assertion `k >= 0' failed
Extend existing test to cover this.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
| |
Otherwise bogus variable redefinition are reported via -o/--optimize:
redefinition.conf:5:8-21: Error: redefinition of symbol 'interface_inet'
define interface_inet = enp5s0
^^^^^^^^^^^^^^
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
| |
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Add test to cover this case.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
Just run -o/--optimize on a ruleset.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not try to merge rules with unsupported statements. This patch adds a
dummy unsupported statement which is included in the statement
collection and the rule vs statement matrix.
When looking for possible rule mergers, rules using unsupported
statements are discarded, otherwise bogus rule mergers might occur.
Note that __stmt_type_eq() already returns false for unsupported
statements.
Add a test using meta mark statement, which is not yet supported.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
Add missing code to the statement collection routine. Compare reject
expressions when available. Add tests/shell.
Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
| |
Remove rule comment after merging rules, let the user decide if they want
to reintroduce the comment in the ruleset file.
Update optimizations/merge_stmt test.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
| |
Otherwise set reference ends up included in an anonymous set, as an
element, which is not supported.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
| |
Call xfree() instead since stmt_alloc() does not initialize the
statement type fields.
Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1609
Fixes: ea1f1c9ff608 ("optimize: memleak in statement matrix")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Verdict and nat are mutually exclusive, no need to support for this
combination.
# cat ruleset.nft
table ip x {
chain y {
type nat hook postrouting priority srcnat; policy drop;
ip saddr 1.1.1.1 tcp dport 8000 snat to 4.4.4.4:80
ip saddr 2.2.2.2 tcp dport 8001 snat to 5.5.5.5:90
}
}
# nft -o -c -f ruleset.nft
Merging:
ruleset.nft:4:3-52: ip saddr 1.1.1.1 tcp dport 8000 snat to 4.4.4.4:80
ruleset.nft:5:3-52: ip saddr 2.2.2.2 tcp dport 8001 snat to 5.5.5.5:90
into:
snat to ip saddr . tcp dport map { 1.1.1.1 . 8000 : 4.4.4.4 . 80, 2.2.2.2 . 8001 : 5.5.5.5 . 90 }
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
Keep inspecting rule verdicts before assuming they are equal. Update
existing test to catch this bug.
Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
This patch reverts d0f14b5337e7 ("optimize: do not merge raw payload
expressions") after adding support for concatenation with variable
length TYPE_INTEGER.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
... log prefix might not be present in log statements.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Check expressions that are expected on the rhs rather than using a
catch-all default case.
Actually, lists and sets need to be their own routine, because this
needs the set element key expression to be merged.
This is a follow up to 99eb46969f3d ("optimize: fix vmap with anonymous
sets").
Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The following example ruleset crashes:
table inet a {
chain b {
tcp dport { 1 } accept
tcp dport 2-3 drop
}
}
because handling for EXPR_SET is missing.
Fixes: 1542082e259b ("optimize: merge same selector with different verdict into verdict map")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|