Logging large messages with node-bunyan-syslog

Question

I'm using node-bunyan-syslog to provide syslog streaming for my node application's Bunyan logger, which seems to be a commonly used library for syslog logging in node applications. Here's my basic logger implementation:

var bunyan = require('bunyan');
var syslog = require('bunyan-syslog');

var syslogStream = syslog.createBunyanStream({
    host: 'my.remote.host',
    port: 514,
    type: 'udp'
});

module.exports = bunyan.createLogger({
    name: 'alert-creator-logger',
    streams: [
        {
            level: 'debug',
            type: 'raw',
            stream: syslogStream
        }
    ]
}); 

Every so often, this log statement:

logger.debug('Processing message: ' + JSON.stringify(data));

throws the following error:

events.js:85
  throw er; // Unhandled 'error' event
        ^
Error: send EMSGSIZE
    at exports._errnoException (util.js:746:11)
    at SendWrap.afterSend [as oncomplete] (dgram.js:348:23)

Because my stream is configured to use UDP, I read up on the UDP(7) spec, which indicates that an EMSGSIZE error gets thrown when the size of a packet being transmitted exceeds the MTU (Maximum Transmission Unit) set for the network interface being used.

My first thought was that the data that is being passed into the log statement was too large (and indeed, the error stops being thrown if I only log small messages), as it can sometimes be multiple KB in size. However, shouldn't the size of the log message have nothing to do with the packet size being transmitted by the node-bunyan-syslog stream that I've instantiated?

I looked through the node-bunyan-syslog source code to try to determine if it does its own message decomposition into packets, but I couldn't locate any such logic.

One solution would be to check the size of proposed log messages first and only log if under a certain size, but that seems unnecessarily restrictive. I would expect node-bunyan-syslog or its underlying socket library to handle fundamental protocol limits like MTU without propagating related errors up to the user.

Is my assessment of the cause accurate? Is an EMSGSIZE error in this context caused by exceeding the MTU? How does one successfully log large messages through Bunyan and node-bunyan-syslog?

Answer 1

This is a fundamental restriction in the syslog protocol itself. A single message must be able to fit into a single UDP packet. And yes, EMSGSIZE means you're exceeding the MTU (with data + protocol overhead). Bunyan can't really compensate for this as it must speak syslog to syslog servers.

One option is to switch to tcp if your remote syslog server can be configured to accept it.