Search code examples
pythonclouderaimpala

Checking state of an operation in Impala

I am writing an Impala client using Golang. Mostly using this to execute SELECT queries and fetch results. I looked at the Impala-Python driver for reference. http://blog.cloudera.com/blog/2014/04/a-new-python-client-for-impala/

After sending an EXECUTE thrift message with the query to be executed, the client gets an operation handle for this execute operation. Then, it queries the status of this operation by sending a GETOPERATIONSTATE thrift message to impala server until the response is SUCCESS/FINISHED.

Is there a way to know the operation state in any other way than polling the server with multiple status requests? Can impala server send a message once that operation is successful?

I checked the packet capture when running impala shell. That too, sends an EXECUTE message followed by a GETOPERATIONSTATE message before looking at results.

Solution

  • Impala supports the HiveServer2 (Hive) and Beeswax (Hue) APIs, both of which will require polling to determine the query state.