Spark 2.1.0 Thrift server not show global temp tables

Question

I'm running a Spark 2.1.0 Thrift server with MySQL metastore.

In my long running application, I'm saving a dataframe with createGlobalTempView or createOrReplaceTempView function, but I can't see them with beeline

When I use write().saveAsTable(), it is OK and I can see the table with beeline.

My thrift server and application are 2 applications on the same Spark cluster and I set the

spark.sql.hive.thriftServer.singleSession=true

Answer 1

For standard temporary view is an expected behavior. Temporary tables are visible only in the scope of a specific SparkSession.

spark.sql.hive.thriftServer.singleSession indicates that all connections to a Thrift server should use a single session, not that there is a single global session for all applications.

To expose temporary tables with Thrift serve you have start it with the same context in which tables have been registered.

For global temporary views (2.1+) you should use the qualified name (including database name spark.sql.globalTempDatabase) to access the view.