I am having issues getting the Python Elasticsearch library working with the Compose for Elasticsearch offering from Bluemix. When I use the following code against an Elasticsearch cluster I created using IBM containers it connects just fine.
(venv) ❯ python
>>> import requests
>>> import elasticsearch
>>> from elasticsearch import Elasticsearch, RequestsHttpConnection
>>> es = Elasticsearch(
... host='clustertestv5.mybluemix.net',
... port=80,
... http_auth=('es_admin', 'REDACTED'),
... timeout=60)
>>> es.info()
{u'cluster_name': u'elasticsearch', u'tagline': u'You Know, for Search', u'version': {u'lucene_version': u'5.5.0', u'build_hash': u'218bdf10790eef486ff2c41a3df5cfa32dadcfde', u'number': u'2.3.3', u'build_timestamp': u'2016-05-17T15:40:04Z', u'build_snapshot': False}, u'name': u'Forgotten One'}
When I try to use the same code but authenticate against the Elasticsearch cluster provided by Bluemix it fails with a ConnectionError response.
(venv) ❯ python
>>> import requests
>>> import elasticsearch
>>> from elasticsearch import Elasticsearch, RequestsHttpConnection
>>> es1 = Elasticsearch(
... host='bluemix-sandbox-dal-9-portal.3.dblayer.com',
... port=15206,
... http_auth=('admin', 'REDACTED'),
... timeout=60)
>>> es1.info()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 220, in info
return self.transport.perform_request('GET', '/', params=params)
File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 105, in perform_request
raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine("''",)))
>>> quit()
A straight curl against the Compose cluster shows that the username and password are correct so I know that is not causing the Connection issue.
(venv) ❯ curl -k 'https://bluemix-sandbox-dal-9-portal.3.dblayer.com:15206/'
<html><body><h1>401 Unauthorized</h1>
You need a valid user and password to access this content.
</body></html>
(venv) ❯ curl -k -u admin:REDACTED 'https://bluemix-sandbox-dal-9-portal.3.dblayer.com:15206/'
{
"name" : "elastic_search56_bluemix_sandbox_dal_9_data_2_dblayer_com",
"cluster_name" : "bmix_dal_yp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"version" : {
"number" : "2.4.0",
"build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
"build_timestamp" : "2016-08-29T09:14:17Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}
For reference here is the Elasticsearch python library - https://elasticsearch-py.readthedocs.io/en/master/
in your python code that creates the ES connection you're missing an option.
Add: use_ssl=True
It is going to complain at that point about missing certificate validation, but will proceed. To round it out you should use the certificate provided in the VCAP connection info, along with the verify_ssl=True option. That option is False by default. Note: default behavior is equivalent to your use of the -k option with curl.