elastic4s with ssl configured elastic Search not working

Question

Below is the script that I am using to access index data from elasticsearch in spark scala code. It is working perfectly.

import com.sksamuel.elastic4s.http.{ElasticClient, ElasticProperties}
import com.sksamuel.elastic4s.http.ElasticDsl._
val esUrl = "http://" + <ip> + ":" + <port>
val client = ElasticClient(ElasticProperties(esUrl))
val response = client.execute(
        deleteIn(Index/Type).by(matchQuery(indexField, values))
      )

Now I have configured ES as SSL and trying to access same data but it is not working since I need to pass SSL details here. So I have passed below url but it is giving invalid url error.

https://user:pass@<ip>:<port>?ssl=true"

or

https://user:pass@<ip>:<port>"

Could you please help me to change this script so that it can also work with SSL configured ES?

I have also tried to use HttpClient instead of ElasticClient but it is not working and giving weird error like below:

scala> import com.sksamuel.elastic4s.http.HttpClient
import com.sksamuel.elastic4s.http.HttpClient

scala> val esUrl = "elasticsearch://user:pass@<ip>:<port>?ssl=true"
esUrl: String = elasticsearch://user:pass@<ip>:<port>?ssl=true

scala> val client = HttpClient(ElasticsearchClientUri(esUrl))
    <console>:43: error: not found: value HttpClient

I want to use same ElasticClient for it. Please let me know if I have missed anything.

Please note: I am using below version:

Spark - 2.4.3

ElasticSearch - 6.2.4

elastic4s - 6.3.3

scala - 2.11

Answer 1

I have done few research for elastic4s and conclude that it support SSL but does not support Authentication (using Search Guard). So I have written my own Scala Method to perform all required Operation over Elastic Search. It is working perfectly.

My GitHub link is below:

https://medium.com/@n.suthar.online/scalaes-apache-spark-and-elasticsearch-connector-d6bc927f8c51