Log4J JsonTemplateLayout replacing message content

Question

I use for logging Log4J

my appender config

    <RollingFile name="FILE-LOGSTASH"
                 fileName="C:\Users\p.bohomaz\IdeaProjects\o2-lcm\log4j-logstash.log"
                 filePattern="C:\Users\p.bohomaz\IdeaProjects\o2-lcm\log4j-logstash-%d{yyyy-MM-dd-HH}.log.gz">
        <LogStashJsonLayout/>
        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
    </RollingFile>

my problem that sometimes in the log I receive plain phone number and I want to replace it to 3********9

I found that LogStashJsonLayout is deprecated and haven't this option and I need to use JsonTemplateLayout

so I changed my configuration to this one

    <RollingFile name="FILE-LOGSTASH"
                 fileName="C:\Users\p.bohomaz\IdeaProjects\o2-lcm\log4j-logstash.log"
                 filePattern="C:\Users\p.bohomaz\IdeaProjects\o2-lcm\log4j-logstash-%d{yyyy-MM-dd-HH}.log.gz">
        <JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>

        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
    </RollingFile>

and my json template file

 {
  "@version": "1",
  "@timestamp": {
    "$resolver": "timestamp",
    "pattern": {
      "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
      "timeZone": "UTC"
    }
  },
  "logger": {
    "$resolver": "logger",
    "field": "name"
  },
  "priority": {
    "$resolver": "level",
    "field": "name"
  },
  "thread": {
    "$resolver": "thread",
    "field": "name"
  },
  "message": {
    "$resolver": "message",
    "replace": {
      "regex": "(\\d{2})(\\d{7,11})(\\d{2})",
      "replacement": "$1********$3"
    }
  }
}

But I still see plain number in the logs I checked documentation and there I see that I need to use pattern.replace(regex,replacement) but it also doesn't work so could someone help with it. Is it possible to do it in template or do I need to create filter for it?

Answer 1

The message resolver of the JSON Template Layout doesn't have a replace configuration – see its documentation. You can use the pattern resolver to fallback to Pattern Layout and use its replace converter instead:

{
  "message": {
    "$resolver": "pattern",
    "pattern": "%replace{%msg}{(\\d{2})(\\d{7,11})(\\d{2})}{$1********$3}"
  }
}

Note: Above I only show how you can achieve what you want to do, this does not mean that what you are doing is the right way. I strongly advise you to reconsider your sensitive data masking scheme. Find-and-replace over arbitrary strings can easily yield false negatives – consider a phone number that doesn't match your regex.