Search code examples
javahadoopparquet-mr

NoClassDefFoundError: org/apache/parquet/conf/ParquetConfiguration


I'm trying to read parquet file from file system in java:

public static void main(String[] args) throws IOException {
    String path = "a.parquet";
    var apr=new AvroParquetReader<GenericRecord>(new Path(path));
    try {
        apr.read();
    } catch (java.lang.Exception e) {
        throw new RuntimeException(e);
    }
}

These are my dependencies:

 <dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-common</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-column</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-encoding</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-hadoop</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-avro</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-format</artifactId>
        <version>2.4.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-format-structures</artifactId>
        <version>1.14.0</version>
    </dependency>
    <dependency>
        <groupId>com.twitter</groupId>
        <artifactId>parquet-jackson</artifactId>
        <version>1.6.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <version>1.2.1</version>
    </dependency>
</dependencies>

When I try to launch it, but I get this exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/parquet/conf/ParquetConfiguration
    at org.apache.parquet.avro.AvroParquetReader.<init>(AvroParquetReader.java:114)
    at Main.main(Main.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.parquet.conf.ParquetConfiguration
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
    ... 2 more

Now do I find which dependency am I missing? Tried to search on mavencentral, but it doesn't help.


Solution

  • Here are the right dependencies:

        <dependency>
            <groupId>com.exasol</groupId>
            <artifactId>parquet-io-java</artifactId>
            <version>2.0.8</version>
        </dependency>