Search code examples
mavenapache-flinkavroflink-streaming

Unable to get org.apache.flink.formats package in my Maven build


I have been trying to build my apache flink project through maven, but for some reason I com across a compilation error. The one of note is "package org.apache.flink.formats does not exist". I am confused because, I thought I put in the appropriate dependencies and plug-ins in my pom.xml file. I have posted parts of my pom.xml below. Am I just missing something else?

        <dependencies>
    <!-- Apache Flink dependencies -->
    <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-core</artifactId>
        <version>1.18.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-parquet</artifactId>
        <version>1.18.1</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-avro</artifactId>
        <version>1.13.1</version>
        <optional>true</optional>
        <exclusions>
            <exclusion>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-client</artifactId>
            </exclusion>
            <exclusion>
                <groupId>it.unimi.dsi</groupId>
                <artifactId>fastutil</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.11.3</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-files</artifactId>
        <version>1.18.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-avro</artifactId>
        <version>1.18.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-formats</artifactId>
        <version>1.18.1</version>
        <type>pom</type>
    </dependency>

I looked through the maven repository and flink documentation for specific dependencies and instructions on how they should be formatted within my pom.xml.


Solution

  • There is no such thing as a flink-formats artifact in Flink. There are only specific format implementations, such as flink-avro.