Search code examples
hbaseycsb

Hbase [READ-FAILED] during YCSB Benchmarking


I am using CDH and through that insatlled HBase and downloaded YCSB Package.Now while running test on Hbase using following command :

ycsb run hbase10 -p columnfamily=cf -P /usr/share/ycsb/workloads/workloada -s -threads 100 -p  recordcount=5000 -p operationcount=5000

Its giving output as follows where near about 50% reads are failing :

[OVERALL], RunTime(ms), 7059.0
[OVERALL], Throughput(ops/sec), 708.3156254426973
[READ-FAILED], Operations, 938.0
[READ-FAILED], AverageLatency(us), 6417.266524520256
[READ-FAILED], MinLatency(us), 897.0
[READ-FAILED], MaxLatency(us), 157567.0
[READ-FAILED], 95thPercentileLatency(us), 18223.0
[READ-FAILED], 99thPercentileLatency(us), 45887.0
[CLEANUP], Operations, 200.0
[CLEANUP], AverageLatency(us), 1489648.93
[CLEANUP], MinLatency(us), 3.0
[CLEANUP], MaxLatency(us), 5054463.0
[CLEANUP], 95thPercentileLatency(us), 4960255.0
[CLEANUP], 99thPercentileLatency(us), 5046271.0
[READ], Operations, 1559.0
[READ], AverageLatency(us), 7588.287363694676
[READ], MinLatency(us), 970.0
[READ], MaxLatency(us), 178047.0
[READ], 95thPercentileLatency(us), 21903.0
[READ], 99thPercentileLatency(us), 59583.0
[READ], Return=OK, 1559
[READ], Return=NOT_FOUND, 938
[UPDATE], Operations, 2503.0
[UPDATE], AverageLatency(us), 13026.518977227328
[UPDATE], MinLatency(us), 2686.0
[UPDATE], MaxLatency(us), 200575.0
[UPDATE], 95thPercentileLatency(us), 33599.0
[UPDATE], 99thPercentileLatency(us), 61599.0
[UPDATE], Return=OK, 2503

I tried changing split_no during database creation,Also read replication factor also enabled and made 3 during table creation.

I am searching why this is happening, but no luck, Can anybody please help me to make this work ??


Solution

  • I'm getting the same behavior with my YCSB runs. I still didn't solve this problem but maybe this might be helpful for your case:

    https://github.com/brianfrankcooper/YCSB/issues/639

    The guy solved this by lowering the field_length value in the workload file from 255 to 250.