Search code examples
hbaseapache-flinkflink-streaming

HBaseWriteStreamExample, but no HBaseReadStreamExample in Flink docs


Why is there no HBaseReadStreamExample in the Flink-HBase connector?

There seems to be a Read, Write, and WriteStream example, but no ReadStream.

Is stream reading from HBase into Flink a bad practice?


Solution

  • This is a fairly common limitation when it comes to connecting databases to stream processors. On the one hand, it is straightforward to implement writing a stream to a database. This might simply involve a series of INSERTs, or UPSERTS, for example. On the other hand, implementing a general purpose stream reader for a database involves ingesting the database's change data capture stream, which is much more complex to implement.

    Note that the HBase connector will support being used as a lookup source in Flink 1.11.

    Also note that better integration between HBase and Flink is planned, for example, see FLIP-117.