Why is syslog-ng truncating output?

I'm sending syslog from a firewall to syslog-ng 3.5.6, which is running on a CentOS 7 server. I can view the logs coming in via tcpdump, and everything looks good. However, once syslog-ng processes the logs and sends them to the output file, they are truncated. I have other syslog sources coming in that are not truncated.

Here's an example of the tcpdump:

192.168.1.1.10002 > 192.168.2.1.514: SYSLOG, length: 694
        Facility authpriv (10), Severity notice (5)
        Msg: 1 2019-11-08T19:30:08 192.168.1.1 CP-GW - Log [[email protected] Action="accept" UUid="{0x5dc5c542}" rule="117" rule_uid="{C94E97R1}" rule_name="Test" src="192.168.1.38" dst="192.168.160.1" proto="17" user="test administrator (test_administrator)(+)Jtest.Administrator (FI) @ TEST - TEST - TEST (Jtest_administrator)(+)Testl.Administ..." src_user_name="test administrator (test_administrator)(+)Jtest.Administrator (FN) @ TEST - TEST - TEST (Jtest_administrator)(+)Testl.Administ..." src_machine_name="[email protected]" snid="76d73747" product="VPN-1 & FireWall-1" service="53" s_port="64642" product_family="Network"]
19:30:09.083410 IP (tos 0x0, ttl 55, id 0, offset 0, flags [DF], proto UDP (17), length 722)

Here's what the output file looks like:

Nov  8 19:30:08 192.168.1.1 CP-GW:

Here's the syslog-ng conf.d file:

destination d_c2 {
  file ("/var/log/fw-test/${SOURCEIP}/${C_YEAR}-${C_MONTH}-${C_DAY}T${C_HOUR}.log"
    owner(root) group(splunk)
    perm(0755) dir_perm(0755)
    create_dirs(yes)
  );
};

filter f_c2 {
  netmask(192.168.1.1);
};

log {
  source(s_udp514);
  filter(f_c2);
  destination(d_c2);
};

And the options section of syslog-ng.conf:

options {
    flush_lines (100);
    time_reopen (10);
    log_fifo_size (1000);
    chain_hostnames (yes);
    use_dns (no);
    use_fqdn (no);
    create_dirs (no);
    keep_hostname (yes);
    log-msg-size(1048576);
};

Any ideas why the output is being truncated?

Thanks in advance

Solution

That is the correct behavior, since your message does not contain anything apart from the header and structured-data.

The input is in IETF syslog format (RFC 5424):

SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG]
HEADER = PRI VERSION SP ISOTIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID

MSG here is optional, and it's empty in your case:

1 2019-11-08T19:30:08 192.168.1.1 CP-GW - Log [[email protected] Action="accept" UUid="{0x5dc5c542}" rule="117" rule_uid="{C94E97R1}" rule_name="Test" src="192.168.1.38" dst="192.168.160.1" proto="17" user="test administrator (test_administrator)(+)Jtest.Administrator (FI) @ TEST - TEST - TEST (Jtest_administrator)(+)Testl.Administ..." src_user_name="test administrator (test_administrator)(+)Jtest.Administrator (FN) @ TEST - TEST - TEST (Jtest_administrator)(+)Testl.Administ..." src_machine_name="[email protected]" snid="76d73747" product="VPN-1 & FireWall-1" service="53" s_port="64642" product_family="Network"] here would come the message body

The output is in the old BSD syslog format (RFC 3164), it doesn't contain any structured-data:

Nov  8 19:30:08 192.168.1.1 CP-GW:

You can avoid losing sdata information by creating a template that contains all the important fields, or by using a different output format, such as the original IETF syslog format:

file ("/var/log/fw-test/${SOURCEIP}/${C_YEAR}-${C_MONTH}-${C_DAY}T${C_HOUR}.log"
  owner(root) group(splunk) perm(0755) dir_perm(0755) create_dirs(yes)
  flags(syslog-protocol)
);