Search code examples
c#hivemobius

JVM method execution failed: Nonstatic method load failed for class 14 when called with no parameters


I am trying to run Mobius HiveDataFrame Example but not able to run.

        var jsonFilePath = "file:///C:/Mobius/build/runtime/data/people.json";
        const string dbName = "SampleHiveDataBaseForMobius";
        const string tableName = "people";

        var builder = SparkSession.Builder().EnableHiveSupport();
        // The following setting is required to use Spark 2.0 in Windows
        // It may be provided in command line when running Mobius app
        //builder = builder.Config("spark.sql.warehouse.dir", "<hdfs or local path>");
        var session = builder.GetOrCreate();
        var peopleDataFrame = session.Read().Json(jsonFilePath);
        session.Sql(string.Format("CREATE DATABASE IF NOT EXISTS {0}", dbName)); // create database if not exists
        session.Sql(string.Format("USE {0}", dbName));
        //hiveContext.Sql(string.Format("DROP TABLE {0}", tableName)); // drop table if exists

        peopleDataFrame.Write().Mode(SaveMode.Overwrite).SaveAsTable(tableName); // create table
        var tablesDataFrame = session.Table(tableName); // get all tables in database
        logger.LogInfo(string.Format("table count in database {0}: {1}", dbName, tablesDataFrame.Count()));
        tablesDataFrame.Show();

        session.Sql(string.Format("SELECT * FROM {0}", tableName)).Show(); // select from table

I am getting error @

var peopleDataFrame = session.Read().Json(jsonFilePath);

Source: Microsoft.Spark.CSharp.Adapter

Message: JVM method execution failed: Nonstatic method load failed for class 14 when called with no parameters

And here is the stack trace

Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] parameters)
Microsoft.Spark.CSharp.Interop.Ipc.JvmBridge.CallNonStaticJavaMethod(JvmObjectReference objectId, String methodName)
Microsoft.Spark.CSharp.Proxy.Ipc.DataFrameReaderIpcProxy.Load()
Microsoft.Spark.CSharp.Sql.DataFrameReader.Load()
Microsoft.Spark.CSharp.Sql.DataFrameReader.Load(String path)
Microsoft.Spark.CSharp.Sql.DataFrameReader.Json(String path)
DemoSparkHiveDataFrame.Program.Main(String[] args) in e:\Work\Feb\VS 2012\DemoSparkHiveDataFrame\DemoSparkHiveDataFrame\Program.cs:line 26
System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
System.Threading.ThreadHelper.ThreadStart_Context(Object state)
System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
System.Threading.ThreadHelper.ThreadStart()

Solution

  • I got a solution i just need to set

    builder.config("spark.sql.warehouse.dir","file:///C:/spark-warehouse") 
    

    for set spark.sql.warehouse.dir path and its works fine for me.