Search code examples
apache-sparkvora

Vora could not connect to Consul Agent

I've installed Vora 1.2 on my SLES 11 SP3 based HDP2.3.

When I try to do command-line based validation of Vora, as per section 2.7 (page 34) of the new Installation&Admin manual, I get a new error now:

scala> vc.sql(testsql)
   
com.sap.spark.vora.discovery.DiscoveryException: Could not connect to Consul Agent on localhost:8500 : null
    at com.sap.spark.vora.discovery.ConsulDiscoveryClient$ConsulDiscoveryClient.(ConsulDiscoveryClient.scala:38)
    at com.sap.spark.vora.discovery.ConsulDiscoveryClient$.getClient(ConsulDiscoveryClient.scala:21)
    at com.sap.spark.vora.discovery.DiscoveryClientFactory$.getClient(DiscoveryClientFactory.scala:9)
    at com.sap.spark.vora.config.VoraConfiguration$.apply(VoraConfiguration.scala:24)
    at com.sap.spark.vora.DefaultSource.buildConfiguration(DefaultSource.scala:403)
    at com.sap.spark.vora.DefaultSource.createRelation(DefaultSource.scala:149)
    at org.apache.spark.sql.execution.datasources.CreateTableUsingTemporaryAwareCommand.resolveDataSource(CreateTableUsingTemporaryAwareCommand.scala:73)
    at org.apache.spark.sql.execution.datasources.CreateTableUsingTemporaryAwareCommand.run(CreateTableUsingTemporaryAwareCommand.scala:31)
    at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57)
    at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57)
    at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:69)
    at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:140)
    at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:138)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
    at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:138)
    at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:933)
    at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:933)
    at org.apache.spark.sql.DataFrame.(DataFrame.scala:144)
    at org.apache.spark.sql.DataFrame.(DataFrame.scala:129)
    at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51)
    at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:725)

I'm getting the same error in Zeppelin. Is that Consul thing missing on my server?

Solution

  • As pointed out in the comments section of the Vora Troubleshooting Wiki:

    The Vora Discovery Service uses Consul (from HashiCorp) to register services. Each server needs to have either a Consul server or Consul agent (mutually exclusive as both server and agent listen on port 8500; at least 3 Consul servers needed - non-server hosts should have a client). Your error message indicates that the host does not have a Consul server or agent running.

    Vora1.2 has a different architecture compared to Vora1.1 with many new services. Please see the What's New in Vora 1.2 and the Vora Installation and Administration Guide. The Vora Discovery Service (=Consul) is one of the new components that need to be installed.

    At least 3 machines are required as we require to have at least 3 Vora Discovery Servers. Each server also acts as a client, so additional hosts with agents are optional (but if additional hosts exists they need to have a Discovery agent running).