Search code examples
hbasegeomesa

GeoMesa on HBase faster import


I am using GeoMesa v1.4.0 on HBase v1.2.6. Currently I am importing and indexing my data with the help of the HBaseQuickStart class. Unfortunately, the runtime for large files is very long. Does someone know of a bulkLoad with GeoMesa to speed up my process? Thanks in advance!


Solution

  • Out of the box, GeoMesa provides ingest through the binary distribution command line tools. You have to define a 'converter' file that has the mappings from your input files to your simple feature type. Once you have that, you can run locally with multiple threads, or remotely using map/reduce. The documentation for the Accumulo command is here, the HBase command will work the same but have slightly different parameters. The documentation on installing the HBase tools is here.

    If you want to keep your Java ingestion code, then you'll need to do something custom. There is an example of writing a map-reduce ingestion here.