Search code examples
apache-sparkignite

What kind of SQL does IgniteRDD.sql support

It looks that IgniteRDD.sql only supports ANSI SQL, not SparkSQL or HiveQL?

When I am using IgniteRDD.sql(sqlText) which throws exception for bad sql, it is something that traces to org.h2.jdbc.JdbcSQLException which means that h2 parse is paring the sql?

Is my understanding correct?

Exception in thread "main" javax.cache.CacheException: Failed to parse query: select __VAL from Integer
    at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:1137)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:732)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:730)
    at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1666)
    at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:730)
    at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:700)
    at org.apache.ignite.spark.IgniteRDD.sql(IgniteRDD.scala:147)
    at com.xyz.ignite.spark.IgniteSparkTest$.main(IgniteSparkTest.scala:33)
    at com.xyz.ignite.spark.IgniteSparkTest.main(IgniteSparkTest.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:613)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: org.h2.jdbc.JdbcSQLException: Column "__VAL" not found; SQL statement:
select __VAL from Integer [42122-191]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
    at org.h2.message.DbException.get(DbException.java:179)
    at org.h2.message.DbException.get(DbException.java:155)
    at org.h2.expression.ExpressionColumn.optimize(ExpressionColumn.java:147)
    at org.h2.command.dml.Select.prepare(Select.java:852)
    at org.h2.command.Parser.prepareCommand(Parser.java:257)
Solution

  • IgniteRDD.sql and objectSql are not working on the top of Spark SQL - they just call Ignite's SQL engine.

    Here is documentations with possibile instructions