I've an use case where devices on the vehicle have to send flat binary files to a cloud server, process them as and when they come in and store the data into Hbase. I'm wondering what data ingestion frameworks support uploading flat binary file from remote device and what data processing framework needs to be used to parse those file and store data into HBase. Please suggest a design that would be appropriate for my application.
Take a look at the https://www.confluent.io/product/connectors/ Confluent platform. You can distribute all of your data via kafka, process it if you need with kafka streams and store the results to the HBase with kafka connectors.