syslog-ng value mapping for variable sub-string

Question

i'm running syslog-ng in container balabit/syslog-ng:3.35.1

i would like to do value mapping for value in vpnrd variable

Below is config for value mapping + example of CSV file

parser p_json {
    json-parser(prefix(".json."));
};

parser p_acd_vrf {
    add-contextual-data(
        selector("${.json.vpnrd}")
        database("vrf_map.csv")
        default-selector("UNKNOWN")
        prefix(".meta_vrf.")
    );
};

Content of CSV file:

111,vrf_id,vrf_1
222,vrf_id,vrf_2
zzz,vrf_id,vrf_example
UNKNOWN,vrf_id,[No VRF; Global Instance peer]
...

The problem is that vpnrd contains string in format xxx:yyy:zzz and I need to split it (delimiter :) and use only last section zzz because only that part is relevant for value mapping.

Is there the way to do it in syslog-ng?

I was looking at custom python parser where it would be easy to do this kind of operation but then I don't know how to connect result of python parser with value mapping parser.

thanks for any pointers

---

Update: i wrote python parser based on this blog but unfortunately mapping still does not work and I always get [No VRF; Global Instance peer] which is mapped as default value. I also can't see the logs from python script (where are they sent?) so it's hard to tshoot what is going on

parser p_py_vrf_id {
    python(
        class("GetVrfId")
    );
};

python {
from syslogng import Logger
logger = Logger()
class GetVrfId(object):
    def parse(self, log_message):
        """
        extract vrf_id from rd variable
        """
        logger.info(f'log: {log_message}')
        try:
            vpnrd = log_message['.json.rd'].split(':')[-1].strip()
            log_message['.json.vpnrd'] = vpnrd
            logger.info(f'vpnrd: {vpnrd}')
        except KeyError:
            log_message['.json.vpnrd'] = 'UNKNOWN'
            logger.error(f'key-error: rd not present')
        except Exception as e:
            logger.error(f'catch-all-error: {e}')
        # return True, other way message is dropped
        return True
};

Answer 1

You have multiple options here, one of them is writing your own parser in Python.

Parsers usually produce new name-value pairs. The connection between add-contextual-data() and your parser would be the key you specify in selector().

LogMessage fields are bytes objects in Python 3, so they have to be decoded into strings before transforming them (for example: log_message['.json.vpnrd'].decode("utf-8")).

syslogng.Logger logs into the internal() source.

---

xxx:yyy:zzz seems to be a fixed length, so you can just remove the unnecessary parts (no custom parser is required):

parser p_acd_vrf {
    add-contextual-data(
        selector("$(substr ${.json.vpnrd} 8)")
        database("vrf_map.csv")
        prefix(".meta_vrf.")
    );
};

Alternatively, $(explode) can be used to split the string into a list (+ $(list-nth) to select the last element).

---

Another option is to use the built-in csv-parser(), configuring : as the separator character.