I have to convert grib2 files to geotiff files using geotools library.
I received a Maven project which does the conversion and isn't SpringBoot-based. I will put only crucial parts of its pom.xml and java code below. Ask for more please if needed:
pom.xml
...
<repositories>
<repository>
<id>osgeo</id>
<name>Open Source Geospatial Foundation Repository</name>
<url>https://repo.osgeo.org/repository/release/</url>
</repository>
</repositories>
...
<dependencies>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-grib</artifactId>
<version>31.3</version>
</dependency>
</dependencies>
...
java:
package com.example;
import java.io.File;
import org.geotools.api.parameter.GeneralParameterValue;
import org.geotools.coverage.grid.GridCoverage2D;
import org.geotools.coverage.grid.io.AbstractGridCoverage2DReader;
import org.geotools.coverage.grid.io.GridFormatFinder;
import org.geotools.gce.geotiff.GeoTiffWriter;
public class Grib2ToGeoTiff {
public static void main(String[] args) {
try {
File gribFile = new File("./grib2/slav.2024061700.054.grib2");
File outputDir = new File("./output");
convertGrib2ToGeoTiff(gribFile, outputDir);
} catch (Exception e) {
e.printStackTrace();
}
}
public static void convertGrib2ToGeoTiff(File gribFile, File outputDir) {
try {
if (!gribFile.exists()) {
System.err.println("File does not exist: " + gribFile.getAbsolutePath());
return;
}
if (!outputDir.exists()) {
outputDir.mkdirs();
}
AbstractGridCoverage2DReader reader = GridFormatFinder.findFormat(gribFile).getReader(gribFile);
if (reader == null) {
System.err.println("Unable to find a reader for the file: " + gribFile.getAbsolutePath());
return;
}
String[] coverageNames = reader.getGridCoverageNames();
for (String name : coverageNames) {
System.out.println("Available coverage: " + name);
GridCoverage2D coverage = (GridCoverage2D) reader.read(name, null);
if (coverage == null) {
System.err.println("Failed to read coverage: " + name);
continue;
}
File tiffFile = new File(outputDir, name + ".tiff");
GeoTiffWriter writer = new GeoTiffWriter(tiffFile);
writer.write(coverage, new GeneralParameterValue[0]);
writer.dispose();
System.out.println("Conversion completed successfully for coverage: " + name);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
After running the Grib2ToGeoTiff.main method the specified grib2 file will be converted and the corresponding tiff files land in output folder, so that its contents will look like below:
Low_cloud_cover_surface_layer.tiff
Pressure_msl.tiff
Relative_humidity_height_above_ground.tiff
Temperature_height_above_ground.tiff
Total_cloud_cover_surface_layer.tiff
Total_snow_precipitation_surface_3_Hour_Accumulation.tiff
Total_water_precipitation_surface_3_Hour_Accumulation.tiff
u-component_of_wind_height_above_ground.tiff
v-component_of_wind_height_above_ground.tiff
Wind_direction_from_which_blowing_height_above_ground.tiff
Wind_speed_height_above_ground.tiff
As I have to use the same functionality in a SpringBoot based app I transfered the code mentioned above to it and executed the app using the same grib2-file as input. But I got the error:
org.geotools.api.data.DataSourceException: Unable to connect
Further interesting part of the error was:
Caused by: org.geotools.api.data.DataSourceException: java.io.IOException: java.io.IOException: Schema 'Low_cloud_cover_surface_layer' does not exist.
The full stack is below:
org.geotools.api.data.DataSourceException: Unable to connect
at org.geotools.coverage.io.netcdf.NetCDFReader.<init>(NetCDFReader.java:158) ~[gt-netcdf-31.3.jar:na]
at org.geotools.coverage.io.netcdf.NetCDFFormat.getReader(NetCDFFormat.java:94) ~[gt-netcdf-31.3.jar:na]
at org.geotools.coverage.io.netcdf.NetCDFFormat.getReader(NetCDFFormat.java:87) ~[gt-netcdf-31.3.jar:na]
at com.myapp.grib2.service.Grib2ToGeoTiffService.convert(Grib2ToGeoTiffService.java:58) ~[classes/:na]
at com.myapp.grib2.Grib2ToGeoTiffServiceITCase.upload(Grib2ToGeoTiffServiceITCase.java:26) ~[test-classes/:na]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) ~[junit-platform-commons-1.10.2.jar:1.10.2]
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) ~[junit-jupiter-engine-5.10.2.jar:5.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[na:na]
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) ~[junit-platform-engine-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63) ~[junit-platform-launcher-1.10.2.jar:1.10.2]
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57) ~[junit5-rt.jar:na]
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) ~[junit-rt.jar:na]
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) ~[idea_rt.jar:na]
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) ~[junit-rt.jar:na]
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232) ~[junit-rt.jar:na]
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55) ~[junit-rt.jar:na]
Caused by: org.geotools.api.data.DataSourceException: java.io.IOException: java.io.IOException: Schema 'Low_cloud_cover_surface_layer' does not exist.
at org.geotools.coverage.io.netcdf.NetCDFAccess.<init>(NetCDFAccess.java:124) ~[gt-netcdf-31.3.jar:na]
at org.geotools.coverage.io.netcdf.NetCDFDriver.connect(NetCDFDriver.java:93) ~[gt-netcdf-31.3.jar:na]
at org.geotools.coverage.io.impl.DefaultFileDriver.process(DefaultFileDriver.java:231) ~[gt-coverage-api-31.3.jar:na]
at org.geotools.coverage.io.netcdf.NetCDFReader.<init>(NetCDFReader.java:156) ~[gt-netcdf-31.3.jar:na]
... 72 common frames omitted
Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Schema 'Low_cloud_cover_surface_layer' does not exist.
at org.geotools.imageio.netcdf.NetCDFImageReader.setInput(NetCDFImageReader.java:307) ~[gt-netcdf-31.3.jar:na]
at java.desktop/javax.imageio.ImageReader.setInput(ImageReader.java:380) ~[na:na]
at org.geotools.coverage.io.netcdf.NetCDFAccess.<init>(NetCDFAccess.java:113) ~[gt-netcdf-31.3.jar:na]
... 75 common frames omitted
Caused by: java.io.IOException: java.io.IOException: Schema 'Low_cloud_cover_surface_layer' does not exist.
at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:414) ~[gt-netcdf-31.3.jar:na]
at org.geotools.imageio.netcdf.NetCDFImageReader.init(NetCDFImageReader.java:541) ~[gt-netcdf-31.3.jar:na]
at org.geotools.imageio.netcdf.NetCDFImageReader.setInput(NetCDFImageReader.java:305) ~[gt-netcdf-31.3.jar:na]
... 77 common frames omitted
Caused by: java.io.IOException: Schema 'Low_cloud_cover_surface_layer' does not exist.
at org.geotools.data.store.ContentDataStore.ensureEntry(ContentDataStore.java:545) ~[gt-main-31.3.jar:na]
at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:336) ~[gt-main-31.3.jar:na]
at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:308) ~[gt-main-31.3.jar:na]
at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:97) ~[gt-main-31.3.jar:na]
at org.geotools.coverage.io.catalog.CoverageSlicesCatalog.addGranules(CoverageSlicesCatalog.java:378) ~[gt-coverage-api-31.3.jar:na]
at org.geotools.imageio.netcdf.NetCDFImageReader.initIndex(NetCDFImageReader.java:391) ~[gt-netcdf-31.3.jar:na]
... 79 common frames omitted
Why doesn't the code work in the SpringBoot app? I noticed that geotools makes use of hsql-db and creates the files:
slav2024061700054grib2.mv.db
slav2024061700054grib2.trace.db
during the conversion. Can this be an obstacle? However I don't use spring-boot-starter-data-jpa in the app
I also ran mvn dependency:tree on both apps and extracted geotools related parts from its output:
non-springboot based app:
https://denis-aristov.ucoz.com/9Ekzvg8oiN/DenisAristov-demo-geotools-deps.log
springboot based app:
https://denis-aristov.ucoz.com/9Ekzvg8oiN/DenisAristov-springboot-geotools-deps.log
Using:
SpringBoot: 3.2.4
Java 21
My assumption about the DB problem turned out to be correct. I compared the h2 dependencies in the two apps and noticed that the h2 version in the SpringBoot based app was almost doubled. So I set a lower version in the pom.xml. At the end the following changes in the pom.xml led to a successful grib2 conversion.
...
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.4.0-b180830.0359</version>
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
<version>2.4.0-b180830.0438</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-grib</artifactId>
<version>31.3</version>
<exclusions>
<exclusion>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.1.119</version>
</dependency>
...