Im trying to setup elasticsearch-curator (version 5.6.0) to delete indices in elasticsearch (version 7.3.1).
Theirs versions should be compatible (https://www.elastic.co/guide/en/elasticsearch/client/curator/current/version-compatibility.html).
Elasticseach is protected by SSL using self-signed certificate, so I need to turn off certificate verification.
This is my curator.yml conf:
client:
hosts:
- 127.0.0.1
port: 9201
url_prefix:
use_ssl: True
certificate: /opt/elastic-stack/curator/security/ca.crt
client_cert:
client_key:
ssl_no_validate: True
http_auth: curator:************
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/elastic-stack/curator/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']
When I run
curator --config /opt/elastic-stack/curator/curator.yml /opt/elastic-stack/curator/actions.yml
Even though ssl_no_validate is set to True, I am getting:
/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
return yaml.load(read_file(path))
/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py:175: UserWarning: Connecting to 127.0.0.1 using SSL with verify_certs=False is insecure.
% host
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 383, in ssl_wrap_socket
return context.wrap_socket(sock)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 217, in perform_request
method, url, body, retries=Retry(False), headers=request_headers, **kw
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 376, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/connection.py", line 394, in connect
ssl_context=context,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 383, in ssl_wrap_socket
return context.wrap_socket(sock)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 899, in get_client
check_version(client)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 685, in check_version
version_number = get_version(client)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 658, in get_version
version = client.info()['version']['number']
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 245, in info
return self.transport.perform_request("GET", "/", params=params)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/transport.py", line 353, in perform_request
timeout=timeout,
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 226, in perform_request
raise SSLError("N/A", str(e), e)
elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python36/python/opt/rh/rh-python36/root/usr/bin//curator", line 11, in <module>
sys.exit(cli())
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/cli.py", line 213, in cli
run(config, action_file, dry_run)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/cli.py", line 160, in run
client = get_client(**client_args)
File "/app/python36/python/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/curator/utils.py", line 906, in get_client
'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch. Error: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777))
I got successful response, when I run
curl -k -u curator:******** https://127.0.0.1:9201
Also kibana and logstash communicates with elasticsearch properly.
Does anyone knows, what might be the problem or how to get more info?
Edit 1:
unfortunately I dont have permissions to use yum, when I installed elasticsearch-curator-5.8.1-1.x86_64.rpm to my home dir using
cd {{ python_installation_dest }} && rpm2cpio ../elasticsearch-curator-5.8.1-1.x86_64.rpm| cpio -idmB
and then run curator, I got:
Fatal Python error: initfsencoding: Unable to get the locale encoding Traceback (most recent call last): File "/opt/python/3.7.4/lib/python3.7/encodings/__init__.py", line 31, in <module> zipimport.ZipImportError: can't decompress data; zlib not available
I dont use pythin 3.7.4, how can I change path to python?
The error appears to be tied to Python issues within RedHat and associated variants. As the official RPM version of Curator 5.8.x now bundles both its own version of Python 3.7.4 and its own up-to-date OpenSSL shared library (1.1.1c in Curator 5.8.1), you will get better results using the official RPM build.