| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The output buffer for date-times is of sufficient size provided that we
don't get oversized integer values for any of the fields, which is a
reasonable assumption. However, the compiler complains about possible
truncation, e.g.:
ulogd_output_JSON.c:314:65: warning: `%06u` directive output may be truncated writing between 6 and 10 bytes into a region of size between 0 and 18
ulogd_output_JSON.c:313:25: note: `snprintf` output between 27 and 88 bytes into a destination of size 38
Fix the warnings by increasing the buffer size.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The compiler has two sets of complaints. Firstly, `t->tm_gmtoffset` is
a `long int`, but it is being passed to `abs`, which leads to warnings
such as:
ulogd_output_JSON.c:308:34: warning: absolute value function `abs` given an argument of type `long int` but has parameter of type `int` which may cause truncation of value
Secondly, it can't verify that the hour value derived from the offset
will in fact fit into `%02d`, thus:
ulogd_output_JSON.c:306:37: warning: `%02d` directive output may be truncated writing between 2 and 6 bytes into a region of size 5
To remedy these, we now mod the offset by 86,400 and assign it to an `int`
before deriving the hour and minute values.
We also change the format-specifier for the hour value to `%+03d` which
causes a sign to be printed even if the value is positive, thus allowing
us not to specify the sign explicitly and to drop the `abs` call for the
hour value.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
| |
`sqlite3_createstmt` returns non-zero on error, but the return-value was
being ignored. Change the calling code to check the return-value, log
an error message and propagate the error.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we derive a field-name by replacing all the underscores in a
DB column-name with full-stops and use the field-name to find the
matching input-key. However, every time we create a new insert SQL
statement, we derive the column-names by copying the field-names to a
buffer, replacing all the full-stops with underscores, and then
appending the buffer containing the column-name to the one containing
the statments.
Apart from the inefficiency, `strncpy` is used to do the copies, which
leads gcc to complain:
ulogd_output_SQLITE3.c:234:17: warning: `strncpy` output may be truncated copying 31 bytes from a string of length 31
Instead, leave the underscores in the field-name, but copy it once to a
buffer in which the underscores are replaced and use this to find the
input-key.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we copy the column-name to a buffer, iterate over it to
replace the underscores with full-stops, using `strchr` from the start
of the buffer on each iteration, then copy the buffer to the field's
`name` member.
Apart from the inefficiency, `strncpy` is used to do the copies, which
leads gcc to complain:
ulogd_output_SQLITE3.c:341:17: warning: `strncpy` output may be truncated copying 31 bytes from a string of length 31
Furthermore, the buffer is not initialized, which means that there is
also a possible buffer overrun if the column-name is too long, since
`strncpy` will not append a NUL.
Instead, copy the column-name directly to the field using `snprintf`,
and run `strchr` from the last underscore on each iteration.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`sqlite3_createstmt` contains a variable `stmt_pos` which points to the
end of the SQL already written, where the next chunk should be appended.
Currently, this is assigned after every write:
sprintf(stmt_pos, ...);
stmt_pos = priv->stmt + strlen(priv->stmt);
However, since `sprintf` returns the number of bytes written, increment
`stmt_pos` by the return-value of `sprintf` in order to avoid the
repeated `strlen` calls.
Pablo mangled this original patch to add this chunk at the end of this
patch (originally submitted as a conversion to use strcpy).
+ for (i = 0; i < cols - 1; i++)
+ stmt_pos += sprintf(stmt_pos, "?,");
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
| |
When mapping DB column names to input-keys, if we cannot find a key to
match a column, the newly allocated `struct field` is leaked. Free it,
and log an error message.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
| |
There is a an off-by-one error in the size of some of the buffers used
to hold key-names. The maximum length of a name is `ULOGD_MAX_KEYLEN`,
and so declare the buffers with size `ULOGD_MAX_KEYLEN + 1`.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In `open_db_pgsql`, we test whether various config-settings are defined
by comparing their string values to `NULL`. However, the `u.string`
member of `struct config_entry` is an array, not a pointer, so it is
never `NULL`. Instead, check whether the string is empty.
Use a pointer to the end of the `connstr` buffer and `sprintf`, rather
than repeated `strcat`s.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we copy the column-name to a buffer, iterate over it to
replace the underscores with full-stops, using `strchr` from the start
of the buffer on each iteration, then copy the buffer to the input-key's
`name` member.
Apart from the inefficiency, `strncpy` is used to do the copies, which
leads gcc to complain:
ulogd_output_PGSQL.c:204:17: warning: `strncpy` output may be truncated copying 31 bytes from a string of length 31
Furthermore, the buffer is not initialized, which means that there is
also a possible buffer overrun if the column-name is too long, since
`strncpy` will not append a NUL.
Instead, copy the column-name directly to the input-key using
`snprintf`, and run `strchr` from the last underscore on each iteration.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we copy the column-name to a buffer, iterate over it to
replace the underscores with full-stops, using `strchr` from the start
of the buffer on each iteration, then copy the buffer to the input-key's
`name` member.
Apart from the inefficiency, `strncpy` is used to do the copies, which
leads gcc to complain:
ulogd_output_MYSQL.c:149:17: warning: `strncpy` output may be truncated copying 31 bytes from a string of length 31
Furthermore, the buffer is not initialized, which means that there is
also a possible buffer overrun if the column-name is too long, since
`strncpy` will not append a NUL.
Instead, copy the column-name directly to the input-key using
`snprintf`, and run `strchr` from the last underscore on each iteration.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
| |
In `open_db_dbi`, we test whether various config-settings are defined
by comparing their string values to `NULL`. However, the `u.string`
member of `struct config_entry` is an array, not a pointer, so it is
never `NULL`. Instead, check whether the string is empty.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
On error, `dbi_conn_quote_string_copy` returns zero. In this case, we
need to set `*dst` to NUL. Handle a return-value of `2` as normal
below. `1` is never returned.
Replace `strncpy` with `memcpy`: using `strncpy` is nearly always a
mistake, and we don't need its special behaviour here.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, we copy the column-name to a buffer, iterate over it to
replace the underscores with full-stops, using `strchr` from the start
of the buffer on each iteration, iterate over it a second time to
lower-case all letters, and finally copy the buffer to the input-key's
`name` member.
In addition to being inefficient, `strncpy` is used to do the copies,
which leads gcc to complain:
ulogd_output_DBI.c:160:17: warning: `strncpy` output may be truncated copying 31 bytes from a string of length 31
Furthermore, the buffer is not initialized, which means that there is
also a possible buffer overrun if the column-name is too long, since
`strncpy` will not append a NUL.
Instead, copy the column-name directly to the input-key using
`snprintf`, and then iterate over it once to replace underscores and
lower-case letters.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The DBI output plugin uses some libdbi functions which have been
deprecated in favour of re-entrant equivalents. Switch to the
re-entrant functions.
Remove superfluous `init` declaration.
Add destructor to clean up DBI instance on exit.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
There are a number of places where we `malloc` some memory and then
`memset` it to zero. Use `calloc` instead.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
This patch enables to show "ct" as well as "raw" if output type is
ULOGD_DTYPE_RAW and "ct" input exists.
Signed-off-by: Ken-ichirou MATSUZAWA <chamas@h4.dion.ne.jp>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
| |
There are a couple of logging calls which use the wrong specifiers for
their integer arguments. Change the specifiers to match the arguments.
Use the correct type for the variable holding the return-value of
`send(2)`.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
| |
make distcheck reports ipfix.h is not included in the tarball file.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, make enters all sub-directories containing source-code, even
if they only contain optional targets which are not configured to be
built. Instead, change the Makefiles so that the sub-directories are
optional, rather than the targets.
Group sub-directory definitions consistently at the top of the Makefiles
that contain them.
Trim a few leading and trailing blank lines.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
| |
Move `${regular_CFLAGS}` from configure.ac to Make_global.am, renaming
it to `AM_CFLAGS`. Add `AM_CPPFGLAGS` to include
`$(top_srcdir)/include`. Include the new file in the Makefiles that
require it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
| |
plugin input type ULOGD_DTYPE_RAW was missing
Signed-off-by: Ken-ichirou MATSUZAWA <chamas@h4.dion.ne.jp>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds the ability to send template records
to the remote collector.
In addition, it also introduces a new
configuration parameter 'send_template', which tells when template
records should be sent. It accepts the following string values:
- "once": Send the template record only the first time (might be coalesced
with data records).
- "always": Send the template record always, with every data record that is sent
to the collector (multiple data records might be sent together).
- "never": Assume the collector knows the schema already. Do not send template records.
If omitted, the default value for 'send_template' is "once".
Signed-off-by: Ander Juaristi <a@juaristi.eus>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds an IPFIX output plugin to ulogd2. It generates NetFlow/IPFIX
traces and sends them to a remote server (collector) via TCP or UDP.
Based on original work by Holger Eitzenberger <holger@eitzenberger.org>.
How to test this
----------------
I am currently testing this with the NFCT input and Wireshark.
Place the following in ulogd.conf:
# this will print all flows on screen
loglevel=1
# load NFCT and IPFIX plugins
plugin="/lib/ulogd/ulogd_inpflow_NFCT.so"
plugin="/lib/ulogd/ulogd_output_IPFIX.so"
stack=ct1:NFCT,ipfix1:IPFIX
[ct1]
netlink_socket_buffer_size=217088
netlink_socket_buffer_maxsize=1085440
accept_proto_filter=tcp,sctp
[ipfix1]
oid=1
host="127.0.0.1"
#port=4739
#send_template="once"
I am currently testing it by launching a plain NetCat listener on port
4739 (the default for IPFIX) and then running Wireshark and see that it
dissects the IPFIX/NetFlow traffic correctly (obviously this relies on
the Wireshark NetFlow dissector being correct).
First:
nc -vvvv -l 127.0.0.1 4739
Then:
sudo ulogd -vc ulogd.conf
Signed-off-by: Ander Juaristi <a@juaristi.eus>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
|
|
|
|
| |
Extend the JSON output plugin so that the generated JSON stream can be
sent to a remote host via TCP/UDP or to a local unix socket.
Signed-off-by: Andreas Jaggi <andreas.jaggi@waterwave.ch>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
|
|
|
|
| |
Signed-off-by: Kaarle Ritvanen <kaarle.ritvanen@datakunkku.fi>
|
|
|
|
|
|
|
| |
This option was left behind when the code was rewritten and is no longer
functional or useful. Remove it entirely.
Signed-off-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While this is not strictly needed for ISO 8601, this is helpful since
otherwise, the receiver can't assume anything about the
timezone.
This uses a GNU extension but as ulogd is quite Linux-specific, this
shouldn't be a problem. The POSIX variables (tzname and daylight) are
quite difficult to use because daylight handling is incomplete (daylight
don't say if DST is now in effect, it just says it is sometimes in
effect).
A timezone offset is used instead of a timezone since it is usually
easier to parse (strptime in glibc is not able to parse a timezone name)
and don't require an up-to-date TZ database.
Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While Logstash is quite flexible in the JSON messages received, the
canonical format it "expects" is the JSON Event v1 format. The timestamp
should be keyed by `@timestamp` and there should be a `@version` key
whose value is 1. All other keys are free.
There is no formal specification of this format. It is however described
here:
https://github.com/elastic/logstash/blob/1.5/lib/logstash/event.rb#L26-L47
It's useful to respect this format as it allows a user to use a less
capable receiver. The new format is enabled only when `eventv1=1` is set
in plugin configuration.
Signed-off-by: Vincent Bernat <Vincent.Bernat@exoscale.ch>
|
|
|
|
| |
Signed-off-by: Felix Janda <felix.janda@posteo.de>
|
|
|
|
| |
Signed-off-by: Jimmy Jones <jimmyjones2@gmx.co.uk>
|
|
|
|
|
|
|
|
|
| |
This patch updates the JSON output plugin to have it use the
timestamp of the packet if available. The date format used
for the timestamp is now using ISO 8601 to have an easy
import in most software (tested with logstash and splunk).
Signed-off-by: Eric Leblond <eric@regit.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch introduces a new JSON output plugin. This
patch displays CIM field name instead of ulogd key valu
if this CIM field is available.
The module does not display binary address but uses the
string version of them. So a complete stack is for example:
stack=log2:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,mac2str1:HWHDR,json1:JSON
If boolean_label is set to 1, then the numeric_label put on packet
by the input plugin is coding the decision on packet. If 0, then
packet has been blocked and if non null it has been accepted.
|
|
|
|
|
|
| |
Some counter have been recently switch to u64. This has caused warnings
relative to format string. This patch uses PRIu64 macro to fix these
warnings.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Kernel and libnetfilter_conntrack counters are 64bit,
so use 64bit too in ulogd instead of 32bit.
Worked fine on little endian systems but big endian systems
had zero counter...
Didn't test ipfix output, but RFC allows template with
either 32 or 64 counters, so should be safe.
Signed-off-by: Ulrich Weber <uw@xyne.com>
Signed-off-by: Eric Leblond <eric@regit.org>
|
|
|
|
|
|
|
|
|
| |
This patch adds a configuration variable for PostgreSQL output.
Named connstring it stores the character string that will be
used to connect to the PostgreSQL server. This allows the user
to use all options available like TLS parameters for example.
Signed-off-by: Eric Leblond <eric@regit.org>
|
|
|
|
| |
Nullify sqlite3 handler at deinit.
|
|
|
|
| |
Nullify mysql handler at deinit.
|
|
|
|
| |
Clean postgresql handler at deinit.
|
| |
|
|
|
|
|
|
|
|
| |
It seems a cast of time_t is needed for i386 system to avoid a
crash. I've added a cast to uint64_t that should be ok on all
Linux system.
Reported-by: netfilter@openenterprise.co.uk
|
|
|
|
|
| |
clang is complaining about missing = being a gnu extension. This
patch adds equal sign to fix the warning.
|
|
|
|
| |
This patch also update some copyright and licence declaration.
|
|
|
|
|
|
|
| |
Graphite is a web application which provide real-time visualization
and storage of numeric time-series data. This patch adds a module
named GRAPHITE which sends NFACCT accounting data to a graphite
server.
|
|
|
|
|
| |
Rename internal keyname ip6.payload_len to remove "_"
to facilitate this.
|
|
|
|
| |
Add 'schema' variable to look into corresponding schema.
|
|
|
|
|
|
|
|
| |
The flags retrieved from `pkg-config --cflags ...` are generally only
preprocessor flags (mostly -I to point to the directories), since
anything else would inconvenience downstream users.
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes this error:
make[3]: Entering directory "/home/jengelh/code/ulogd2/output"
CC ulogd_output_XML.lo
ulogd_output_XML.c:31:49: fatal error:
libnetfilter_acct/libnetfilter_acct.h: No such file or directory
(Note that pkgconfig-provided cflags are actually cppflags, so
I add ${LIBNETFILTER_ACCT_CFLAGS} to AM_CPPFLAGS.)
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
|
|
|
|
|
|
| |
Inclusion of libnetfilter_acct.h is not enough as the integer type
definition are not included in the header. So if NFCT is disable, the
compilation fails.
|
| |
|