i am running below code in Zeppelin 0.7
%spark
//val sc: SparkContext // An existing SparkContext.
sc
import sqlContext.implicits._
import org.apache.spark.sql._
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val people = sqlContext.jsonFile("/Users/asdf/Desktop/people.json")
people.printSchema()
people.show()
people.select("name").show()
people.toDF().registerAsTable("people")
Its working till people.select("name").show()
but throws error at last line, below is the error:
+-------+
| name|
+-------+
|Michael|
| Andy|
| Justin|
+-------+
<console>:230: error: value registerAsTable is not a member of org.apache.spark.sql.DataFrame
people.toDF().registerAsTable("people")
asper my knowledge i imported all required and converted it to df before registering it as table. what am i missing here?
Below code worked for me, the issue was with Zeppelin that we should not create a SQLContext Zeppelin provides its own.
%spark
import sqlContext.implicits._
import org.apache.spark.sql._
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;
val people = sqlContext.jsonFile("/Users/asdf/Desktop/people.json")
people.printSchema()
people.show()
people.select("name").show()
people.toDF().registerTempTable("people")
Then above created Temp table can be used to Query for interactive charts in %sql
notebook of Zeppelin