Wildfly 20 is connected with a Logstash instance listening on tcp port 5300:
logstash.conf:
input {
tcp {
codec => json
port => "5300"
}
}
output {
stdout {}
}
Making use of its built-in json logging capabilities with socket connection, as outpointed in wildfly-logstash does not send logs to logstash, Wildfly is configured on the Wildfly CLI, entering the following sequence of statements (that end up in standalone.xml automatically):
/subsystem=logging/json-formatter=LOG-STASH:add(key-overrides={timestamp=@timestamp,message=@message,logger-name=@source,host-name=@source_host}, exception-output-type=formatted)
/socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=log-stash:add(host=localhost, port=8000)
/subsystem=logging/socket-handler=LOGSTASH-SOCKET:add(named-formatter=LOG-STASH, outbound-socket-binding-ref=log-stash, level=DEBUG)
/subsystem=logging/async-handler=LOGSTASH-ASYNC:add(queue-length=512, subhandlers=[LOGSTASH-SOCKET])
/subsystem=logging/root-logger=ROOT:add-handler(name=LOGSTASH-ASYNC)
It produces log statements on standard out of the logstash node, as e.g.:
{
"level" => "DEBUG",
"host" => "gateway",
"processId" => 14972,
"sequence" => 34696,
"@version" => "1",
"@source" => "com.myapplication.TaskService",
"@source_host" => "device-01",
"threadName" => "EJB default - 6",
"threadId" => 215,
"loggerClassName" => "org.slf4j.impl.Slf4jLogger",
"mdc" => {},
"ndc" => "",
"port" => 64210,
"processName" => "jboss-modules.jar",
"@timestamp" => 2021-03-31T14:10:19.869Z,
"@message" => "task execution successfull: MailDaemon"
}
That is only half way to the goal, required is another set of attribute names (of the individual json log message) to fit in our enterprise logstash instances. Especially, neither "host-name" nor "logger-name" are written, although configured; instead "@source_host" and @source are logged.
Further adaption of the log-formatter LOG-STASH partially succeeds.
1) /subsystem=logging/json-formatter=LOG-STASH:write-attribute(name="meta-data",value={service="myapplication-api", serviceversion="1.1.0", instanceId="myapplication-api-1.1.0"})
2) /subsystem=logging/json-formatter=LOG-STASH:write-attribute(name="key-overrides",value=[severity=level,timestamp=@timestamp,message=msg,logger-name=@source,host-name=@source_host])
Further simplifaction results in attribute stored, but not applied:
3) /subsystem=logging/json-formatter=LOG-STASH:write-attribute(name="key-overrides",value={"level"="severity"})
4) /subsystem=logging/json-formatter=LOG-STASH:read-attribute(name="key-overrides")
works and meta data are added. 2. and 3. bring no results. 4. prints out like
INFO [org.jboss.as.cli.CommandContext] {
"outcome" => "success",
"result" => {"level" => "severity"}
}
{
"outcome" => "success",
"result" => {"level" => "severity"}
}
With the above setup the following Wildfly CLI command sucessfully renames the wanted keys' default values:
/subsystem=logging/json-formatter=LOG-STASH:write-attribute(name="key-overrides",value={"level"="severity","sequence"="trace","thread-id"="pid","logger-class-name"="class","thread-name"="thread"})
These settings end up in standalone.xml and logging.properties in the same folder on disk.
During my work there was a discrepancy between configured keys in both files.
Be aware that camel case key names like threadId produce a configuration error. You have to use thread-id instead. I found this by inspection of the JBoss logging library, i.e. looking on the Java source code.
The produced logging output is e.g.
{
"pid" => 212,
"message" => "Synchronizaing finished in 0ms",
"@version" => "1",
"loggerName" => "com.myapp.Cache",
"@timestamp" => 2021-04-08T13:49:00.178Z,
"port" => 59182,
"processName" => "jboss-modules.jar",
"trace" => 4245,
"host" => "gateway",
"severity" => "DEBUG",
"processId" => 10536,
"mdc" => {},
"hostName" => "host-alpha",
"timestamp" => "2021-04-08T15:49:00.176+02:00",
"class" => "org.slf4j.impl.Slf4jLogger",
"ndc" => "",
"thread" => "EJB default - 7"
}
What would be nice still, is to have fields mdc and ndc deprived of the output.