I've had a lambda running for a few years under Java8, and I just updated it to Java 11. It immediately broke, giving me errors like:
Caused by: java.lang.ExceptionInInitializerError
at com.mycompany.rest.providers.JsonProvider.writeTo(JsonProvider.java:80)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
at org.glassfish.jersey.client.ClientRequest.doWriteEntity(ClientRequest.java:461)
at org.glassfish.jersey.client.ClientRequest.writeEntity(ClientRequest.java:443)
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:367)
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:265)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:297)
... 15 more
Caused by: java.lang.UnsupportedOperationException: No class provided, and an appropriate one cannot be found.
at org.apache.logging.log4j.LogManager.callerClass(LogManager.java:571)
at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:596)
at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:583)
at com.mycompany.rest.util.NonClosingOutputStream.<clinit>(NonClosingOutputStream.java:11)
... 25 more
The class in question isn't particularly exciting, and has a straightforward static initialization that is common in my classes:
public class NonClosingOutputStream extends ProxyOutputStream {
private static final Logger log = LogManager.getLogger(); // Line 11
public NonClosingOutputStream(final OutputStream proxy) {
super(proxy);
}
...
I've seen problems like this before, when I switched my (non-Lambda) java servers from 8 to 11; I needed to flag my jar's manifest as Multi-Release: true, because the ApacheLog4j artifact that I depend on provides alternate implementations for the org.apache.logging.log4j.util.StackLocator class in Java 8- and 9+. However, I kind of expect the JVM to just pick up the appropriate version of the class. Is there some configuration that I have to set somewhere? Is it possible that switching my Lambda from Java 8 -> Java 11 confused something, somewhere?
jar/META-INF/versions:
versions/
├── 11
│ └── org
│ └── glassfish
│ └── jersey
│ └── internal
│ └── jsr166
│ ├── JerseyFlowSubscriber$1.class
│ ├── JerseyFlowSubscriber.class
│ ├── SubmissionPublisher$1.class
│ ├── SubmissionPublisher$2.class
│ ├── SubmissionPublisher$3.class
│ ├── SubmissionPublisher$4.class
│ ├── SubmissionPublisher$5.class
│ ├── SubmissionPublisher$6.class
│ ├── SubmissionPublisher.class
│ └── SubmissionPublisherFactory.class
└── 9
├── module-info.class
└── org
└── apache
└── logging
└── log4j
├── core
│ └── util
│ └── SystemClock.class
└── util
├── Base64Util.class
├── ProcessIdUtil.class
├── StackLocator.class
└── internal
└── DefaultObjectInputFilter.class
Edit: I am finding some references indicating that, when AWS Lambda extracts a JAR, they don't extract the META-INF directory, which contains the MANIFEST.MF file that tells the JVM that the JAR is a Muli-Release JAR. Do Lambdas support Multi-Release JARs at all?
As others have mentioned, AWS Lambda doesn't seem to support multi-release JARS. (See JEP 238.)
Log4J uses a org.apache.logging.log4j.util.Stackwalker to walk up the stack. The old version uses sun.reflect.Reflection.getCallerClass(int), but I infer that the new version that works with Java 9+ doesn't work with earlier versions, so they created a multi-release JAR. (See LOG4J2-2537: Log4j 2 Performance issue with Java 11 for more back-story.) Unfortunately even if you use <Multi-Release>true</Multi-Release> in your Maven Shade ManifestResourceTransformer configuration, AWS Lambda doesn't know how to deal with multi-release JARs.
Basically in the multi-release JAR there is a META-INF/versions/9/ directory, under which is kept the Java 9+ version of Stackwalker and related classes, which don't use sun.reflect.Reflection.getCallerClass. If you're targeting Java 9+ you can do what the JVM would do and use these classes rather than the ones not inside META-INF/. The Maven Shade Plugin doesn't seem to have a way to do this manually, but it appears you can brute-force the issue by (mis?)using the Maven Shade Plugin <relocations> facility to copy over the version under META-INF/versions/9/, overwriting the old ones that use sun.reflect.Reflection.getCallerClass. One other trick: specify a fuller path to the classes, so you won't copy over the metadata file under META-INF/versions/9/ itself.
Thus it appears (after limited testing) that adding the following to your Maven Shade Plugin <configuration> section will produce a JAR using the Java 9+ version of Stackwalker, which doesn't even look for sun.reflect.Reflection.getCallerClass, thus preventing the warning:
<relocations>
<relocation>
<pattern>META-INF/versions/9/org/apache/logging/log4j/</pattern>
<shadedPattern>org/apache/logging/log4j/</shadedPattern>
</relocation>
</relocations>
Using this technique, you can remove the <Multi-Release>true</Multi-Release> of your ManifestResourceTransformer, because there's no point in making a multi-release JAR anymore, as you're manually creating a single-release JAR with the Java 9+ content.
See aws-lambda-java-libs Issue #204: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance where I first reported this workaround.
Note that this answer is only a workaround if you insist in using the Maven Shade Plugin! A better approach would be to use the Maven Assembly Plugin as Maarten Brak suggested, or the the Spring Boot Maven Plugin after they address Issue #36101, so you wouldn't need a workaround such as this.