summaryrefslogtreecommitdiffstats
path: root/src/cache.c
Commit message (Collapse)AuthorAgeFilesLines
* intervals: add support to automerge with kernel elementsPablo Neira Ayuso2022-04-131-0/+6
| | | | | | | | | | | | | | | | | | Extend the interval codebase to support for merging elements in the kernel with userspace element updates. Add a list of elements to be purged to cmd and set objects. These elements representing outdated intervals are deleted before adding the updated ranges. This routine splices the list of userspace and kernel elements, then it mergesorts to identify overlapping and contiguous ranges. This splice operation is undone so the set userspace cache remains consistent. Incrementally update the elements in the cache, this allows to remove dd44081d91ce ("segtree: Fix add and delete of element in same batch"). Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: silence compiler warningsFlorian Westphal2022-01-151-2/+2
| | | | | | | | | | | | cache.c:504:22: warning: ‘chain’ may be used uninitialized in this function [-Wmaybe-uninitialized] cache.c:504:22: warning: ‘table’ may be used uninitialized in this function [-Wmaybe-uninitialized] erec.c:128:16: warning: ‘line’ may be used uninitialized in this function [-Wmaybe-uninitialized] optimize.c:524:9: warning: ‘line’ may be used uninitialized in this function [-Wmaybe-uninitialized] Fixes: 8ad4056e9182 ("erec: expose print_location() and line_location()") Fixes: afbd102211dc ("src: do not use the nft_cache_filter object from mnl.c") Fixes: fb298877ece2 ("src: add ruleset optimization infrastructure") Signed-off-by: Florian Westphal <fw@strlen.de>
* src: 'nft list chain' prints anonymous chains correctlyPablo Neira Ayuso2022-01-151-0/+37
| | | | | | | | | If the user is requesting a chain listing, e.g. nft list chain x y and a rule refers to an anonymous chain that cannot be found in the cache, then fetch such anonymous chain and its ruleset. Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1577 Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add helper function to fill up the rule cachePablo Neira Ayuso2022-01-151-16/+25
| | | | | | | Add a helper function to dump the rules and add them to the corresponding chain. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: do not set error code twicePablo Neira Ayuso2022-01-151-21/+7
| | | | | | | The 'ret' variable is already set to a negative value to report an error, do not set it again to a negative value. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: do not use the nft_cache_filter object from mnl.cPablo Neira Ayuso2022-01-151-1/+8
| | | | | | Pass the table and chain strings to mnl_nft_rule_dump() instead. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: Support filtering for a specific flowtablePhil Sutter2021-12-031-13/+42
| | | | | | | | | | Extend nft_cache_filter to hold a flowtable name so 'list flowtable' command causes fetching the requested flowtable only. Dump flowtables just once instead of for each table, merely assign fetched data to tables inside the loop. Signed-off-by: Phil Sutter <phil@nwl.cc>
* cache: Filter set list on server sidePhil Sutter2021-12-031-26/+37
| | | | | | | | | Fetch either all tables' sets at once, a specific table's sets or even a specific set if needed instead of iterating over the list of previously fetched tables and fetching for each, then ignoring anything returned that doesn't match the filter. Signed-off-by: Phil Sutter <phil@nwl.cc>
* cache: Filter chain list on kernel sidePhil Sutter2021-12-031-19/+19
| | | | | | | | | | | | | When operating on a specific chain, add payload to NFT_MSG_GETCHAIN so kernel returns only relevant data. Since ENOENT is an expected return code, do not treat this as error. While being at it, improve code in chain_cache_cb() a bit: - Check chain's family first, it is a less expensive check than comparing table names. - Do not extract chain name of uninteresting chains. Signed-off-by: Phil Sutter <phil@nwl.cc>
* cache: Filter rule list on kernel sidePhil Sutter2021-12-031-20/+3
| | | | | | | | | | | Instead of fetching all existing rules in kernel's ruleset and filtering in user space, add payload to the dump request specifying the table and chain to filter for. Since list_rule_cb() no longer needs the filter, pass only netlink_ctx to the callback and drop struct rule_cache_dump_ctx. Signed-off-by: Phil Sutter <phil@nwl.cc>
* cache: Filter tables on kernel sidePhil Sutter2021-12-031-8/+1
| | | | | | | | | | | Instead of requesting a dump of all tables and filtering the data in user space, construct a non-dump request if filter contains a table so kernel returns only that single table. This should improve nft performance in rulesets with many tables present. Signed-off-by: Phil Sutter <phil@nwl.cc>
* cache: do not skip populating anonymous set with -tPablo Neira Ayuso2021-11-181-4/+7
| | | | | | | | | | | | | | | | | | | --terse does not apply to anonymous set, add a NFT_CACHE_TERSE bit to skip named sets only. Moreover, prioritize specific listing filter over --terse to avoid a bogus: netlink: Error: Unknown set '__set0' in lookup expression when invoking: # nft -ta list set inet filter example Extend existing test to improve coverage. Fixes: 9628d52e46ac ("cache: disable NFT_CACHE_SETELEM_BIT on --terse listing only") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: filter out rules by chainPablo Neira Ayuso2021-11-111-3/+81
| | | | | | | | | | | | | | | | | | | | With an autogenerated ruleset with ~20k chains. # time nft list ruleset &> /dev/null real 0m1,712s user 0m1,258s sys 0m0,454s Speed up listing of a specific chain: # time nft list chain nat MWDG-UGR-234PNG3YBUOTS5QD &> /dev/null real 0m0,542s user 0m0,251s sys 0m0,292s Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: missing family in cache filteringPablo Neira Ayuso2021-11-111-4/+8
| | | | | | | | Check family when filtering out listing of tables and sets. Fixes: 3f1d3912c3a6 ("cache: filter out tables that are not requested") Fixes: 635ee1cad8aa ("cache: filter out sets and maps that are not requested") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: do not populate cache if it is going to be flushedPablo Neira Ayuso2021-11-111-2/+69
| | | | | | | Skip set element netlink dump if set is flushed, this speeds up set flush + add element operation in a batch file for an existing set. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: move list filter under structPablo Neira Ayuso2021-11-111-11/+11
| | | | | | | Wrap the table and set fields for list filtering to prepare for the introduction element filters. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: disable NFT_CACHE_SETELEM_BIT on --terse listing onlyPablo Neira Ayuso2021-10-281-2/+2
| | | | | | | Instead of NFT_CACHE_SETELEM which also disables set dump. Fixes: 6bcd0d576a60 ("cache: unset NFT_CACHE_SETELEM with --terse listing") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: ensure evaluate_cache_list flags are set correctlyChris Arges2021-10-271-0/+1
| | | | | | | | | This change ensures that when listing rulesets with the terse flag that the terse flag is maintained. Fixes: 6bcd0d576a60 ("cache: unset NFT_CACHE_SETELEM with --terse listing") Signed-off-by: Chris Arges <carges@cloudflare.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: honor table in set filteringPablo Neira Ayuso2021-10-271-1/+2
| | | | | | | | Check if table mismatch, in case the same set name is used in different tables. Fixes: 635ee1cad8aa ("cache: filter out sets and maps that are not requested") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: honor filter in set listing commandsPablo Neira Ayuso2021-10-271-0/+2
| | | | | | | | Fetch table, set and set elements only for set listing commands, e.g. nft list set inet filter ipv4_bogons. Fixes: 635ee1cad8aa ("cache: filter out sets and maps that are not requested") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: always set on NFT_CACHE_REFRESH for listingPablo Neira Ayuso2021-10-271-6/+7
| | | | | | | | This flag forces a refresh of the cache on list commands, several object types are missing this flag, this fixes nft --interactive mode. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: unset NFT_CACHE_SETELEM with --terse listingPablo Neira Ayuso2021-10-021-3/+12
| | | | | | Skip populating the set element cache in this case to speed up listing. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: filter out sets and maps that are not requestedPablo Neira Ayuso2021-09-301-2/+19
| | | | | | | Do not fetch set content for list commands that specify a set name. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: filter out tables that are not requestedPablo Neira Ayuso2021-09-301-10/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Do not fetch table content for list commands that specify a table name, e.g. # nft list table filter This speeds up listing of a given table by not populating the cache with tables that are not needed. - Full ruleset (huge with ~100k lines). # sudo nft list ruleset &> /dev/null real 0m3,049s user 0m2,080s sys 0m0,968s - Listing per table is now faster: # nft list table nat &> /dev/null real 0m1,969s user 0m1,412s sys 0m0,556s # nft list table filter &> /dev/null real 0m0,697s user 0m0,478s sys 0m0,220s Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1326 Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: finer grain cache population for list commandsPablo Neira Ayuso2021-09-291-2/+23
| | | | | | Skip full cache population for list commands to speed up listing. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: set on cache flags for nested notationPablo Neira Ayuso2021-09-291-0/+10
| | | | | | | | | Set on the cache flags for the nested notation too, this is fixing nft -f with two files, one that contains the set declaration and another that adds a rule that refers to such set. Closes: https://bugzilla.netfilter.org/show_bug.cgi?id=1474 Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: provide a empty list for flowtables and objects when request failsPablo Neira Ayuso2021-08-261-16/+20
| | | | | | | | | Old kernels do not support for dumping the flowtable and object lists, provide an empty list instead to unbreak the cache initialization. Fixes: 560963c4d41e ("cache: add hashtable cache for flowtable") Fixes: 45a84088ecbd ("cache: add hashtable cache for object") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: skip set element netlink dump for add/delete element commandPablo Neira Ayuso2021-08-261-2/+14
| | | | | | | | Add NFT_CACHE_SETELEM_MAYBE to dump the set elements conditionally, only in case that the set interval flag is set on. Reported-by: Cristian Constantin <const.crist@googlemail.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: check errno before invoking cache_release()Marco Oliverio2021-05-141-2/+4
| | | | | | | | | | | | if genid changes during cache_init(), check_genid() sets errno to EINTR to force a re-init of the cache. cache_release() may inadvertly change errno by calling free(). Indeed free() may invoke madvise() that changes errno to ENOSYS on system where kernel is configured without support for this syscall. Signed-off-by: Marco Oliverio <marco.oliverio@tanaza.com> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: remove flowtable from cache on delete flowtable commandPablo Neira Ayuso2021-05-021-0/+5
| | | | | | | Update the cache to remove this flowtable from the evaluation phase. Add flowtable_cache_del() function for this purpose. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* evaluate: remove chain from cache on delete chain commandPablo Neira Ayuso2021-05-021-0/+5
| | | | | | | Update the cache to remove this chain from the evaluation phase. Add chain_cache_del() function for this purpose. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add hashtable cache for tablePablo Neira Ayuso2021-05-021-6/+42
| | | | | | | | | | | | Add a hashtable for fast table lookups. Tables that reside in the cache use the table->cache_hlist and table->cache_list heads. Table that are created from command line / ruleset are also added to the cache. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: missing table cache for several policy objectsPablo Neira Ayuso2021-05-021-0/+4
| | | | | | Populate the cache with tables for several policy objects types. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add set_cache_del() and use itPablo Neira Ayuso2021-05-021-0/+5
| | | | | | | | Update set_cache_del() from the monitor path to remove sets in the cache. Fixes: df48e56e987f ("cache: add hashtable cache for sets") Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add hashtable cache for flowtablePablo Neira Ayuso2021-05-021-2/+86
| | | | | | | | | | Add flowtable hashtable cache. Actually I am not expecting that many flowtables to benefit from the hashtable to be created by streamline this code with tables, chains, sets and policy objects. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add hashtable cache for objectPablo Neira Ayuso2021-05-021-2/+96
| | | | | | | | | | | | | | | | | | | | This patch adds a hashtable for object lookups. This patch also splits table->objs in two: - Sets that reside in the cache are stored in the new tables->cache_obj and tables->cache_obj_ht. - Set that defined via command line / ruleset file reside in tables->obj. Sets in the cache (already in the kernel) are not placed in the table->objs list. By keeping separated lists, objs defined via command line / ruleset file can be added to cache. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: consolidate object cache infrastructurePablo Neira Ayuso2021-05-021-12/+36
| | | | | | | This patch consolidates the object cache infrastructure. Update set and chains to use it. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: consolidate nft_cache infrastructurePablo Neira Ayuso2021-05-021-17/+18
| | | | | | | | - prepend nft_ prefix to nft_cache API and internal functions - move declarations to cache.h (and remove redundant declarations) - move struct nft_cache definition to cache.h Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: pass chain name to chain_cache_find()Pablo Neira Ayuso2021-05-021-5/+4
| | | | | | | | You can identify chains through the unique handle in deletions, update this interface to take a string instead of the handle to prepare for the introduction of 64-bit handle chain lookups. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: bail out if chain list cannot be fetched from kernelPablo Neira Ayuso2021-04-031-1/+1
| | | | | | Do not report success if chain cache list cannot be built. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: add hashtable cache for setsPablo Neira Ayuso2021-04-031-3/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a hashtable for set lookups. This patch also splits table->sets in two: - Sets that reside in the cache are stored in the new tables->cache_set and tables->cache_set_ht. - Set that defined via command line / ruleset file reside in tables->set. Sets in the cache (already in the kernel) are not placed in the table->sets list. By keeping separated lists, sets defined via command line / ruleset file can be added to cache. Adding 10000 sets, before: # time nft -f x real 0m6,415s user 0m3,126s sys 0m3,284s After: # time nft -f x real 0m3,949s user 0m0,743s sys 0m3,205s Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: check for NULL chain in cache_init()Pablo Neira Ayuso2021-04-031-0/+5
| | | | | | | | | | | | | Another process might race to add chains after chain_cache_init(). The generation check does not help since it comes after cache_init(). NLM_F_DUMP_INTR only guarantees consistency within one single netlink dump operation, so it does not help either (cache population requires several netlink dump commands). Let's be safe and do not assume the chain exists in the cache when populating the rule cache. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: statify chain_cache_dump()Pablo Neira Ayuso2021-04-031-1/+2
| | | | | | Only used internally in cache.c Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: split chain list in tablePablo Neira Ayuso2021-04-031-3/+3
| | | | | | | | | | | | | | | | | | | This patch splits table->lists in two: - Chains that reside in the cache are stored in the new tables->cache_chain and tables->cache_chain_ht. The hashtable chain cache allows for fast chain lookups. - Chains that defined via command line / ruleset file reside in tables->chains. Note that chains in the cache (already in the kernel) are not placed in the table->chains. By keeping separated lists, chains defined via command line / ruleset file can be added to cache. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: rename chain_htable to cache_chain_htPablo Neira Ayuso2021-04-031-3/+3
| | | | | | Rename the hashtable chain that is used for fast cache lookups. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: move remaining cache functions in rule.c to cache.cPablo Neira Ayuso2021-03-111-2/+205
| | | | | | Move all the cache logic to src/cache.c Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: add chain hashtable cachePablo Neira Ayuso2020-08-261-0/+92
| | | | | | | | | | | | | | | | | | | | | | | | This significantly improves ruleset listing time with large rulesets (~50k rules) with _lots_ of non-base chains. # time nft list ruleset &> /dev/null Before this patch: real 0m11,172s user 0m6,810s sys 0m4,220s After this patch: real 0m4,747s user 0m0,802s sys 0m3,912s This patch also removes list_bindings from netlink_ctx since there is no need to keep a temporary list of chains anymore. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: cache gets out of sync in interactive modePablo Neira Ayuso2020-08-061-0/+2
| | | | | | | | | | | Since 94a945ffa81b ("libnftables: Get rid of explicit cache flushes"), the cache logic checks for the generation number to refresh the cache. This breaks interactive mode when listing stateful objects though. This patch adds a new flag to force a cache refresh when the user requests a ruleset listing. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* src: rename CMD_OBJ_SETELEM to CMD_OBJ_ELEMENTSPablo Neira Ayuso2020-05-141-3/+3
| | | | | | | | The CMD_OBJ_ELEMENTS provides an expression that contains the list of set elements. This leaves room to introduce CMD_OBJ_SETELEMS in a follow up patch. Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
* cache: Fix for doubled output after reset commandPhil Sutter2020-01-161-1/+3
| | | | | | | | | | | | | | | Reset command causes a dump of the objects to reset and adds those to cache. Yet it ignored if the object in question was already there and up to now CMD_RESET was flagged as NFT_CACHE_FULL. Tackle this from two angles: First, reduce cache requirements of reset command to the necessary bits which is table cache. This alone would suffice if there wasn't interactive mode (and other libnftables users): A cache containing the objects to reset might be in place already, so add dumped objects to cache only if they don't exist already. Signed-off-by: Phil Sutter <phil@nwl.cc> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>