JanusGraph with SparkComputer throws an exception: GetNewApplicationRequestProto cannot be cast to com.google.protobuf.Message

Question

when i use JanusGraph2.0 and spark on yarn to computer e.g.:

gremlin> graph = GraphFactory.open('conf/hadoop-graph/hadoop-load.properties')
==>hadoopgraph[gryoinputformat->nulloutputformat]
gremlin> blvp = BulkLoaderVertexProgram.build().writeGraph('conf/janusgraph-cassandra.properties').create(graph)
==>BulkLoaderVertexProgram[bulkLoader=IncrementalBulkLoader,vertexIdProperty=bulkLoader.vertex.id,userSuppliedIds=false,keepOriginalIds=true,batchSize=0]
gremlin> graph.compute(SparkGraphComputer).program(blvp).submit().get()

it throws an exception:

08:57:03 ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$GetNewApplicationRequestProto cannot be cast to com.google.protobuf.Message
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:226)
	at com.sun.proxy.$Proxy23.getNewApplication(Unknown Source)

Can anyone help me to solve this problem? thx

Answer 1

This question was cross-posted in janusgraph-users and answered there.

Check the version compatibility matrix. JanusGraph 0.2.0 is not compatible with Spark 2.11. http://docs.janusgraph.org/latest/version-compat.html

JanusGraph 0.2.0 is compiled/tested with Spark 1.6.1. Make sure to align the versions.