Search code examples
eclipsemavenapache-sparkapache-spark-sqlgeospark

Maven package error using geospark library


Currently, I am working on geospatial analytics use case and I am using spark 2.4.0 along with geospark library.When I am trying to create the application jar file using eclipse it is giving me the below error.Could you please help me to resolve the below maven dependency error?

Maven File:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.12</artifactId>
    <version>2.4.0</version>
</dependency> 

<!-- https://mvnrepository.com/artifact/org.scala-lang/scala-library -->
<dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-library</artifactId>
    <version>2.3.1</version>
</dependency>


<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.12</artifactId>
    <version>2.4.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.datasyslab/geospark -->
<dependency>
    <groupId>org.datasyslab</groupId>
    <artifactId>geospark</artifactId>
    <version>1.2.0</version>
    <scope>provided</scope>
</dependency>

<!-- https://mvnrepository.com/artifact/org.datasyslab/geospark-sql -->
<dependency>
    <groupId>org.datasyslab</groupId>
    <artifactId>geospark-sql_2.3</artifactId>
    <version>1.2.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.datasyslab/geospark-viz -->
<dependency>
    <groupId>org.datasyslab</groupId>
    <artifactId>geospark-viz_2.3</artifactId>
    <version>1.2.0</version>
</dependency> 


<!-- https://mvnrepository.com/artifact/org.apache.kudu/kudu-client -->
<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-client</artifactId>
    <version>1.6.0</version>
    <scope>test</scope>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.kudu/kudu-spark2 -->
<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-spark2_2.11</artifactId>
    <version>1.6.0</version>
    <scope>test</scope>
</dependency> 

Error: geospark-sql_2.3-1.2.0.jar of GFence build path is cross-compiled with an incompatible version of Scala (2.3.0). In case this report is mistaken, this check can be disabled in the compiler preference page. Description Resource Path Location Type geospark-viz_2.3-1.2.0.jar of GFence build path is cross-compiled with an incompatible version of Scala (2.3.0). In case this report is mistaken, this check can be disabled in the compiler preference page. kudu-spark2_2.11-1.6.0.jar of GFence build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page.

GeoSpark version = 1.2.0

Apache Spark version = 2.4.0

JRE version = 1.8.0

API type = Scala

Thanks, Sumit


Solution

  • Go to Windows -> Preferences

    Expand Scala preferences & select Compiler.

    In Compiler preferences tab go to Build manager & uncheck withVersionClasspathValidator check box.

    Screenshot of the same::

    enter image description here