Search code examples
apache-kafkastreamingapache-nifidruiddata-stream

Streaming with Druid, Kafka, Superset


I am testing data streaming by using Kafka, Druid and SuperSet.

I am at the point where I have some data in Druid (see 1. picutre).

After that I am able to generate Druid datasource in Superset by option "Refresh Druid metadata" (see 2.pic) Problem is when I wanna querie a data, I am getting this error message:

URLError: <urlopen error [Errno -2] Name or service not known>

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/superset/viz.py", line 329, in get_df_payload
    df = self.get_df(query_obj)
  File "/usr/lib/python2.7/site-packages/superset/viz.py", line 142, in get_df
    self.results = self.datasource.query(query_obj)
  File "/usr/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 1238, in query
    client=client, query_obj=query_obj, phase=2)
  File "/usr/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 959, in get_query_str
    return self.run_query(client=client, phase=phase, **query_obj)
  File "/usr/lib/python2.7/site-packages/superset/connectors/druid/models.py", line 1126, in run_query
    client.timeseries(**qry)
  File "/usr/lib/python2.7/site-packages/pydruid/client.py", line 167, in timeseries
    return self._post(query)
  File "/usr/lib/python2.7/site-packages/pydruid/client.py", line 484, in _post
    res = urllib.request.urlopen(req)
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno -2] Name or service not known>

And also see 3. pic

Any idea what can be a problem?

I am feeding Kafka thru the NiFi and then I have kafka source hooked to the druid target in SAM.

Thank you!

  1. picutre 1. picutre

  2. picutre enter image description here

  3. picutre enter image description here

  4. no data in superset enter image description here


Solution

  • Problem solved, problem was that broker host was not defined in cluster configuration in super set UI. I set it up to value: localhost and now it's up and running.