Search code examples
hadoopelasticsearchapache-sparkelasticsearch-hadoop

Do I have to install Hadoop in order to use Elasticsearch ES-Hadoop connector


I have Elasticsaerch cluster and want to use Apache Spark to run computations on it. The ES-Hadoop connector contains a Spark library that can run computations on Elasticsearch. Do I have to install Hadoop first or can I just put elasticsearch-spark_2.10-2.2.0-rc1.jar in Spark classpath as documentation mentions in order to make it work?


Solution

  • If you want to use ElasticSearch with Spark then I would suggest you to use Elasticsearch's Spark Native Support.

    It is more easy to use and you can directly laod/save data to/from Elastic search as Spark RDD. For more detail refer : Elasticsearch Apache Spark support