Search code examples
hbasemapr

MapR initial get calls taking time

I need to optimize time taken for get operation for mapr distribution ina multithreaded distributed environment -

1) Data Service is running on multiple JVM's. 2) I am using HTablePool (have to stick with client version where there is no support for ConnectionFactory) and while profiling a performance found that using connection.getTable() introduces HOTSPOT.

Problem - Initial request take ~1 sec (which I believed is because client is not having cached resources to locate regions etc.) it took that extra time. Normally response time is few ms.

I need to clear my understanding on - 1) Is it that every time we get a new pooled HTableInterface there will be delay to create client side cache to locate region server. 2) If client is idle (pool is created yet no activity), I observed that again executing get call takes ~1 sec. Is it possible that I keep the already cached region locations available all the time.

Are there any client side setting that I should explore to always keep the response time less than 200ms.

Thanks

Solution

  • To get over connections getting stale added below property in core-site.xml

    fs.mapr.rpc.timeout => 400

    Instead of using connection.getTable every time, created HTable once and used with multiple threads.

    HTable for mapr implementation is thread safe.