I am trying to create a custom track from an existing Elasticsearch cluster using the instructions in the Rally documentation page.
I am running Rally as a Docker Image. I am running elasticsearch:7.12.0 as well as a docker image in the same host. The docker container is running and host can be accessed by using basic auth.
$ sudo docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a60bca4121c6 docker.elastic.co/elasticsearch/elasticsearch:7.12.0 "/bin/tini -- /usr/l…" About an hour ago Up About an hour 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp es01
2241695bdfa7 docker.elastic.co/elasticsearch/elasticsearch:7.12.0 "/bin/tini -- /usr/l…" About an hour ago Up About an hour 9200/tcp, 9300/tcp es02
$ curl -XGET http://localhost:9200 -u elastic:esbackup-password
{
"name" : "es01",
"cluster_name" : "es-docker-cluster",
"cluster_uuid" : "04EdIVB_QC2il_3iJCarkw",
"version" : {
"number" : "7.12.0",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
"build_date" : "2021-03-18T06:17:15.410153305Z",
"build_snapshot" : false,
"lucene_version" : "8.8.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
When I try to execute create-track command using --client-options, I get the ConnectionError mentioned below.
$ sudo docker run elastic/rally create-track --track=mytrack --target-hosts=localhost:9200 --client-options="timeout:60,use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'esbackup-password'" --indices="my-index-local-*" --output-path=~/tracks
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
[ERROR] Cannot create-track. ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fdc8ca93220>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fdc8ca93220>: Failed to establish a new connection: [Errno 111] Connection refused).
Getting further help:
*********************
* Check the log files in /rally/.rally/logs for errors.
* Read the documentation at https://esrally.readthedocs.io/en/2.2.0/.
* Ask a question on the forum at https://discuss.elastic.co/tags/c/elastic-stack/elasticsearch/rally.
* Raise an issue at https://github.com/elastic/rally/issues and include the log files in /rally/.rally/logs.
-------------------------------
[INFO] FAILURE (took 1 seconds)
-------------------------------
From the error it is clear that Rally running in docker container is not able to create connection with Elasticsearch running in separate container. I am passing the credentials in --client-options but seems those are not getting picked up.
Not sure, what am I doing wrong here! Any pointers would be appreciated!
Thanks to Elastic Team for answering this question on their official discuss channel:
The setup required a docker network to be configured and shared between the Elasticsearch and Rally containers in order for them to communicate.
We don't state here how your Elasticsearch container is run, but once we have it executing with a network we can run rally with the following (provided you replace the network and container name placeholders):
$ docker run --network <es-network-name> elastic/rally create-track --track=mytrack --target-hosts=<es-container-name>:9200 --client-options="timeout:60,use_ssl:false,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'esbackup-password'" --indices="my-index-local-*" --output-path=~/tracks
Reference - https://discuss.elastic.co/t/connectionerror-while-creating-track-in-rally-with-docker-image/272448?u=akm