Search code examples
apache-sparkhiveapache-spark-sql

Issues with executing multiple queries using Spark and HiveSQL

Within spark-shell where if I attempt something like below:

var sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) 

var query = "select * from myDB.table limit 10; drop table if exists"

var queryDF = sqlContext.sql(query)

I receive an error:

org.apache.spark.sql.AnlysisException: missing EOF at ';'....

I can execute simple simple single line queries just fine but multi-command queries always result in this error.

Has anyone ran into and solved this problem?

Solution

  • It is an expected behavior. SQLContext.sql doesn't support multiple queries so solution is pretty simple - pass only a single query at the time.