I am attempting to establish a connection between Apache Superset and Dremio, both of which are deployed within a Kubernetes cluster. While the setup closely resembles the issue described in this post, there are some nuances related to Kubernetes which I think might be causing the problem.
Setup:
Apache Superset: Deployed on Kubernetes using the official helm chart (3.0.0).
Dremio: Also deployed on Kubernetes using its respective helm chart (24.1.0).
Python: v3.9.18
sqlachemy-dremio: v3.0.3
Connection String Used:
dremio+flight://dremioUser:dremioPwd@<dremio-service>.<namespace>.svc.cluster.local:32010/dremio
Error Encountered: Upon pressing the "test connection" button in Superset, I'm met with the following error:
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request() File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
return f(self, *args, **kwargs) File "/app/superset/views/base_api.py", line 127, in wraps
raise ex File "/app/superset/views/base_api.py", line 121, in wraps
duration, response = time_function(f, self, *args, **kwargs) File "/app/superset/utils/core.py", line 1518, in time_function
response = func(*args, **kwargs) File "/app/superset/utils/log.py", line 255, in wrapper
value = f(*args, **kwargs) File "/app/superset/views/base_api.py", line 93, in wraps
return f(self, *args, **kwargs) File "/app/superset/databases/api.py", line 905, in test_connection
TestConnectionDatabaseCommand(item).run() File "/app/superset/databases/commands/test_connection.py", line 192, in run
raise SupersetErrorsException(errors) from ex superset.exceptions.SupersetErrorsException: [SupersetError(message='(builtins.NoneType) None\n[SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:10.1.2.3:32010: Ssl handshake failed: SSL_ERROR_SSL: error:0A00010B:SSL routines::wrong version number]\n(Background on this error at: https://sqlalche.me/e/14/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Dremio', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})] 2023-10-13 12:09:50,495:WARNING:superset.views.base:SupersetErrorsException Traceback (most recent call last): File "/app/superset/databases/commands/test_connection.py", line 163, in run
raise DBAPIError(ex_str or None, None, None) sqlalchemy.exc.DBAPIError: (builtins.NoneType) None [SQL: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:10.1.2.3:32010: Ssl handshake failed: SSL_ERROR_SSL: error:0A00010B:SSL routines::wrong version number] (Background on this error at: https://sqlalche.me/e/14/dbapi)
Steps Taken:
However, connecting via the Superset UI still throws the error.
I've gone through the suggestions from the previous post, but haven't had any luck. Given that both services are on Kubernetes, I'm wondering if there are any specific configurations I might be overlooking.
If anyone has encountered similar issues in a Kubernetes environment or has insights into potential fixes, your input would be greatly appreciated.
Try exposing the dremio application to a public service e.g. loadbalancer and get a public IP address. Then you can use that IP to for your conenction uri like:
dremio+flight://dremioUser:dremioPwd@<dremio-public-ip>:32010/dremio?%3Coption%3E=%3Cvalue%3E%E2%80%9D&UseEncryption=false