I encountered the common "org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "EPSG" found for object of type "EngineeringCRS" while trying to run my packaged jar on an Apache Flink cluster.
I already had gt-epsg-hsql added as a dependency when I got the above error. I managed to fix that by following the instructions on the official Geotools building FAQ and rearranging the dependency order in my pom.xml file and using maven-shade-plugin.
But it seems now I encounter a new exception which is this one.
Caused by: java.lang.IllegalArgumentException: Argument "ellipsoid" should not be null.
at org.geotools.referencing.GeodeticCalculator.<init>(GeodeticCalculator.java:195) ~[?:?]
at org.geotools.referencing.GeodeticCalculator.<init>(GeodeticCalculator.java:188) ~[?:?]
Spent the whole day pulling my hair trying to debug this but had no success. It all works when running it on a local cluster via my IDE but packaging and deploying the jar to my cluster causes these errors.
I tried emulating the suggested POM that the original asker had listed in this question and modified my pom.xml file to the current one attached below but it still gives the same error. Incase anyone needs to see my full pom.xml for details, it is hosted here on Textbin.
<dependencies>
<!-- GeoTools Related -->
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-shapefile</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-epsg-hsql</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<plugins>
<!-- Java Compiler -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. -->
<!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.apache.flink:force-shading</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
<exclude>org.slf4j:*</exclude>
<exclude>log4j:*</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<!-- Do not copy the signatures in the META-INF folder.
Otherwise, this might cause SecurityExceptions when using the JAR. -->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>DataGen.StreamingJob</mainClass>
</transformer>
<!-- This bit merges the various GeoTools META-INF/services files -->
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
</plugin>
</plugins>
Any ideas on how to fix this error?
EDIT: Apache Flink version 1.13.0. Geotools 28-SNAPSHOT. Java version "1.8.0_333".
Adding the contents of META-INF/services:
├── com.fasterxml.jackson.core.JsonFactory
├── com.fasterxml.jackson.core.ObjectCodec
├── it.geosolutions.imageio.compression.CompressorSpi
├── it.geosolutions.imageio.compression.DecompressorSpi
├── java.sql.Driver
├── javax.annotation.processing.Processor
├── javax.cache.spi.CachingProvider
├── javax.imageio.spi.ImageInputStreamSpi
├── javax.imageio.spi.ImageOutputStreamSpi
├── javax.imageio.spi.ImageReaderSpi
├── javax.imageio.spi.ImageWriterSpi
├── javax.measure.spi.FormatService
├── javax.measure.spi.ServiceProvider
├── javax.measure.spi.SystemOfUnitsService
├── javax.measure.spi.UnitFormatService
├── javax.media.jai.OperationRegistrySpi
├── javax.media.jai.OperationsRegistrySpi
├── javax.xml.bind.JAXBContext
├── org.apache.flink.client.deployment.ClusterClientFactory
├── org.apache.flink.core.execution.PipelineExecutorFactory
├── org.apache.flink.runtime.security.contexts.SecurityContextFactory
├── org.apache.flink.runtime.security.modules.SecurityModuleFactory
├── org.apache.flink.runtime.state.changelog.StateChangelogWriterFactory
├── org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonFactory
├── org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.ObjectCodec
├── org.apache.flink.table.factories.Factory
├── org.apache.flink.table.factories.TableFactory
├── org.apache.kafka.common.config.provider.ConfigProvider
├── org.apache.logging.log4j.core.util.ContextDataProvider
├── org.apache.logging.log4j.message.ThreadDumpMessage$ThreadInfoFactory
├── org.apache.logging.log4j.spi.Provider
├── org.apache.logging.log4j.util.PropertySource
├── org.geotools.coverage.grid.GridCoverageFactory
├── org.geotools.coverage.grid.io.footprint.FootprintLoaderSpi
├── org.geotools.data.DataStoreFactorySpi
├── org.geotools.data.FileDataStoreFactorySpi
├── org.geotools.feature.AttributeTypeFactory
├── org.geotools.filter.FunctionExpression
├── org.geotools.filter.FunctionFactory
├── org.geotools.filter.expression.PropertyAccessorFactory
├── org.geotools.http.HTTPClientFactory
├── org.geotools.referencing.factory.gridshift.GridShiftLocator
├── org.geotools.referencing.operation.MathTransformProvider
├── org.geotools.renderer.crs.ProjectionHandlerFactory
├── org.geotools.renderer.style.ExternalGraphicFactory
├── org.geotools.renderer.style.MarkFactory
├── org.geotools.styling.StyleFactory
├── org.geotools.swing.tool.InfoToolHelper
├── org.geotools.util.ConverterFactory
├── org.jaitools.numeric.Processor
├── org.jboss.marshalling.ProviderDescriptor
├── org.opengis.coverage.processing.Operation
├── org.opengis.feature.FeatureFactory
├── org.opengis.feature.type.FeatureTypeFactory
├── org.opengis.filter.FilterFactory
├── org.opengis.filter.expression.Function
├── org.opengis.referencing.crs.CRSAuthorityFactory
├── org.opengis.referencing.crs.CRSFactory
├── org.opengis.referencing.cs.CSAuthorityFactory
├── org.opengis.referencing.cs.CSFactory
├── org.opengis.referencing.datum.DatumAuthorityFactory
├── org.opengis.referencing.datum.DatumFactory
├── org.opengis.referencing.operation.CoordinateOperationAuthorityFactory
├── org.opengis.referencing.operation.CoordinateOperationFactory
├── org.opengis.referencing.operation.MathTransformFactory
├── reactor.blockhound.integration.BlockHoundIntegration
└── tech.units.indriya.spi.NumberSystem
1 directory, 67 files
Fixed it, the problem has nothing to do with Geotools.
The problem was in these specific lines of the my code:
crs = Params.coordinateReferenceSystem;
GeodeticCalculator gc = new GeodeticCalculator(crs);
and the crs was being set to null because Params.coordinateReferenceSystem was defined as a static variable. Static variables cannot be serialized hence are not replicated across all nodes in a distributed system.
The result of which it works perfectly on the local IDE but fails on the cluster.