Search code examples
elasticsearchindexingcasting

How to cast a field in Elasticsearch pipelines / Painless script

I have an application which is logging level as integers. I am using filebeat to send the logs to ES. I have set level as string in the ES index, which is working for most of the applications. But when filebeat is receiving an integer, the indexation is failing of course with:

"type":"illegal_argument_exception","reason":"field [level] of type [java.lang.Integer] cannot be cast to [java.lang.String]"

In my document: "level":30

I added a step Script in my ingestion pipeline. But I can't manage to make it work: either I get a compilation error or the script is somehow failing and nothing at all get indexed.

Some very basic script I tried:

if (doc['level'].value == 30) {
    doc['level'].value = 'info';
}

Any idea on how to handle this in ES pipelines?

Regards

Solution

  • The best way is to transform data before sending to ES.

    You can usse processsor in filebeat to filter you data. https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html