I am new to GCP AppEngine and I chose the Flexible environment for several reasons. However, I am shocked to find out that the flexible environment's non-"compatible" runtimes appear to not allow me to map my app's logging events to the appropriate log levels in cloud logging. Am I reading this correctly? https://cloud.google.com/appengine/docs/flexible/java/writing-application-logs#writing_application_logs_1
And this page was really unhelpful. https://cloud.google.com/java/getting-started/logging-application-events
This is after several hours of reading GAE logging woes and trying to determine which applied to the Standard environment vs. Flexible. Best I can tell, event level mapping is possible in the standard environment.
However, for more fine-grained control of the log level display in the Cloud Platform Console, the logging framework must use a java.util.logging adapter. https://cloud.google.com/appengine/docs/java/how-requests-are-handled#Java_Logging
OK. That's a vague reference, but I think I saw something more clear somewhere else.
Regardless, shouldn't this be easier in the "flexible" environment? Who doesn't want to easily filter events by Logging levels?
Update: I clarified the question to indicate that I am asking about the non-compatible runtimes in the GAE flexible environment.
Here is how I got cloud logging to work using SLF4J. This works on a non-compatible Java GAE Flex environment.
logback.xml
<configuration debug="true">
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>/var/log/app_engine/custom_logs/app.log.json</file>
<append>true</append>
<!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout
class="putyourpackagenamehere.GCPCloudLoggingJSONLayout">
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg</pattern>
</layout>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="FILE" />
</root>
</configuration>
Here is the PatternLayout class I used to produce the JSON on a single line in the log file.
import static ch.qos.logback.classic.Level.DEBUG_INT;
import static ch.qos.logback.classic.Level.ERROR_INT;
import static ch.qos.logback.classic.Level.INFO_INT;
import static ch.qos.logback.classic.Level.TRACE_INT;
import static ch.qos.logback.classic.Level.WARN_INT;
import java.util.Map;
import org.json.JSONObject;
import com.homedepot.ta.wh.common.logging.GCPCloudLoggingJSONLayout.GCPCloudLoggingEvent.GCPCloudLoggingTimestamp;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.spi.ILoggingEvent;
/**
* Format a LoggingEvent as a single line JSON object
*
* <br>https://cloud.google.com/appengine/docs/flexible/java/writing-application-logs
*
* <br>From https://cloud.google.com/appengine/articles/logging
* <quote>
* Applications using the flexible environment should write custom log files to the VM's log directory at
* /var/log/app_engine/custom_logs. These files are automatically collected and made available in the Logs Viewer.
* Custom log files must have the suffix .log or .log.json. If the suffix is .log.json, the logs must be in JSON
* format with one JSON object per line. If the suffix is .log, log entries are treated as plain text.
* </quote>
*
* Nathan: I can't find a reference to this format on the google pages but I do remember getting the format from some
* GO code that a googler on the community slack channel referred me to.
*/
public class GCPCloudLoggingJSONLayout extends PatternLayout {
@Override
public String doLayout(ILoggingEvent event) {
String formattedMessage = super.doLayout(event);
return doLayout_internal(formattedMessage, event);
}
/* for testing without having to deal wth the complexity of super.doLayout()
* Uses formattedMessage instead of event.getMessage() */
String doLayout_internal(String formattedMessage, ILoggingEvent event) {
GCPCloudLoggingEvent gcpLogEvent = new GCPCloudLoggingEvent(formattedMessage
, convertTimestampToGCPLogTimestamp(event.getTimeStamp())
, mapLevelToGCPLevel(event.getLevel())
, null);
JSONObject jsonObj = new JSONObject(gcpLogEvent);
/* Add a newline so that each JSON log entry is on its own line.
* Note that it is also important that the JSON log entry does not span multiple lines.
*/
return jsonObj.toString() + "\n";
}
static GCPCloudLoggingTimestamp convertTimestampToGCPLogTimestamp(long millisSinceEpoch) {
int nanos = ((int) (millisSinceEpoch % 1000)) * 1_000_000; // strip out just the milliseconds and convert to nanoseconds
long seconds = millisSinceEpoch / 1000L; // remove the milliseconds
return new GCPCloudLoggingTimestamp(seconds, nanos);
}
static String mapLevelToGCPLevel(Level level) {
switch (level.toInt()) {
case TRACE_INT:
return "TRACE";
case DEBUG_INT:
return "DEBUG";
case INFO_INT:
return "INFO";
case WARN_INT:
return "WARN";
case ERROR_INT:
return "ERROR";
default:
return null; /* This should map to no level in GCP Cloud Logging */
}
}
/* Must be public for JSON marshalling logic */
public static class GCPCloudLoggingEvent {
private String message;
private GCPCloudLoggingTimestamp timestamp;
private String traceId;
private String severity;
public GCPCloudLoggingEvent(String message, GCPCloudLoggingTimestamp timestamp, String severity,
String traceId) {
super();
this.message = message;
this.timestamp = timestamp;
this.traceId = traceId;
this.severity = severity;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
public GCPCloudLoggingTimestamp getTimestamp() {
return timestamp;
}
public void setTimestamp(GCPCloudLoggingTimestamp timestamp) {
this.timestamp = timestamp;
}
public String getTraceId() {
return traceId;
}
public void setTraceId(String traceId) {
this.traceId = traceId;
}
public String getSeverity() {
return severity;
}
public void setSeverity(String severity) {
this.severity = severity;
}
/* Must be public for JSON marshalling logic */
public static class GCPCloudLoggingTimestamp {
private long seconds;
private int nanos;
public GCPCloudLoggingTimestamp(long seconds, int nanos) {
super();
this.seconds = seconds;
this.nanos = nanos;
}
public long getSeconds() {
return seconds;
}
public void setSeconds(long seconds) {
this.seconds = seconds;
}
public int getNanos() {
return nanos;
}
public void setNanos(int nanos) {
this.nanos = nanos;
}
}
}
@Override
public Map<String, String> getDefaultConverterMap() {
return PatternLayout.defaultConverterMap;
}
}