I have a spring-boot 2.2.13 app that is multitenant. I am creating multiple Spring contexts one for each tenant, plus the main context and one more. So, there should be n+2 contexts loaded where n the number of tenants. I have two modes to generate the tenant Contexts, serially (in the main thread of the app) or in parallel using an ExecutorService.
While developing, I am using spring-boot-maven-plugin:run and all tenant contexts' are loaded as expected either in serial or in parallel.
But, in production I am deploying and starting the app as a regular fat-jar app with java -jar and not all contexts are loaded. The erroneous contexts are the ones that we try to start from a worker thread. Any contexts loaded from the main thread are Ok. The error is a class path resource [org/springframework/context/annotation/AutoProxyRegistrar.class] cannot be opened because it does not exist exception.
2021-10-29 09:43:03.724 WARN 8260 --- [onPool-worker-3] s.c.a.AnnotationConfigApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.piag.products.Application]; nested exception is java.io.FileNotFoundException: class path resource [org/springframework/context/annotation/AutoProxyRegistrar.class] cannot be opened because it does not exist
2021-10-29 09:43:03.737 ERROR 8260 --- [onPool-worker-3] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.piag.products.Application]; nested exception is java.io.FileNotFoundException: class path resource [org/springframework/context/annotation/AutoProxyRegistrar.class] cannot be opened because it does not exist
at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:610) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:311) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:250) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:207) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:175) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:319) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:236) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:280) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:96) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:707) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:533) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.13.RELEASE.jar!/:2.2.13.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:405) ~[spring-boot-2.2.13.RELEASE.jar!/:2.2.13.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.13.RELEASE.jar!/:2.2.13.RELEASE]
at com.piag.products.IPRunner.startUpTenantContext(IPRunner.java:377) ~[app-1.0-SNAPSHOT.jar!/:1.0-SNAPSHOT]
at com.piag.products.IPRunner.startTenantCustomerExecutionContext(IPRunner.java:358) ~[app-1.0-SNAPSHOT.jar!/:1.0-SNAPSHOT]
at com.piag.products.IPRunner.lambda$doStartTenantCustomerExecutionContext$22(IPRunner.java:322) ~[app-1.0-SNAPSHOT.jar!/:1.0-SNAPSHOT]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[na:na]
at java.base/java.util.concurrent.ConcurrentHashMap$KeySpliterator.forEachRemaining(ConcurrentHashMap.java:3566) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:952) ~[na:na]
at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:926) ~[na:na]
at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327) ~[na:na]
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) ~[na:na]
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[na:na]
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[na:na]
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[na:na]
Caused by: java.io.FileNotFoundException: class path resource [org/springframework/context/annotation/AutoProxyRegistrar.class] cannot be opened because it does not exist
at org.springframework.core.io.ClassPathResource.getInputStream(ClassPathResource.java:180) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.core.type.classreading.SimpleMetadataReader.getClassReader(SimpleMetadataReader.java:55) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.core.type.classreading.SimpleMetadataReader.<init>(SimpleMetadataReader.java:49) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:103) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.core.type.classreading.CachingMetadataReaderFactory.getMetadataReader(CachingMetadataReaderFactory.java:123) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.core.type.classreading.SimpleMetadataReaderFactory.getMetadataReader(SimpleMetadataReaderFactory.java:81) ~[spring-core-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.asSourceClass(ConfigurationClassParser.java:696) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.asSourceClasses(ConfigurationClassParser.java:675) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:582) ~[spring-context-5.2.12.RELEASE.jar!/:5.2.12.RELEASE]
... 29 common frames omitted
The way I start the app is the same. You can see that the internalAutoProxyCreator hasn't been registered in the erroneous contexts.
With spring-boot:run all is good. 7 contexts/ 7 internalAutoProxyCreator registered.
$ mvn spring-boot:run -Dspring-boot.run.arguments=--logging.level.org.springframework=DEBUG |grep internalAutoProxyCreator
2021-10-28 21:32:05.612 DEBUG 18068 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:25.214 DEBUG 18068 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:25.365 DEBUG 18068 --- [onPool-worker-7] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:31.737 DEBUG 18068 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:31.745 DEBUG 18068 --- [onPool-worker-7] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:38.153 DEBUG 18068 --- [onPool-worker-7] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:32:44.212 DEBUG 18068 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
With java -jar only 3/7 internalAutoProxyCreator are registered.
$ java -jar target/app-1.0-SNAPSHOT.jar --logging.level.org.springframework=DEBUG |grep internalAutoProxyCreator
2021-10-28 21:39:23.434 DEBUG 1672 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:39:45.387 DEBUG 1672 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
2021-10-28 21:39:52.776 DEBUG 1672 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Creating shared instance of singleton bean 'org.springframework.aop.config.internalAutoProxyCreator'
Am I missing something? Is it a possible bug in spring-aop or a classloading issue?
PS: I'm in the process of upgrading to latest Spring Boot. Waiting for v. 2.6 where another issue in Spring-Data-Jdbc I had reported is resolved.
The issue was that I was using a ForkJoinPool which doesn't transfer Classloader from main thread (where Spring Boot app is loading) to its child threads. Switched to an ExecutorService and all is good.