Search code examples
hbase

How to read log in hbase pe?


2018-02-24 16:13:22,575 INFO  [TestClient-0] hbase.PerformanceEvaluation: Timed test starting in thread TestClient-0
2018-02-24 16:13:22,577 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest latency log (microseconds), on 1 measures
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Min      = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Avg      = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest StdDev   = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 50th     = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 75th     = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 95th     = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99th     = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.9th   = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.99th  = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.999th = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Max      = 2054.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest valueSize after 0 measures
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Min      = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Avg      = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest StdDev   = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 50th     = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 75th     = 0.0
2018-02-24 16:13:22,578 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 95th     = 0.0
2018-02-24 16:13:22,579 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99th     = 0.0
2018-02-24 16:13:22,579 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.9th   = 0.0
2018-02-24 16:13:22,579 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.99th  = 0.0
2018-02-24 16:13:22,579 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest 99.999th = 0.0
2018-02-24 16:13:22,579 INFO  [TestClient-0] hbase.PerformanceEvaluation: SequentialWriteTest Max      = 0.0
2018-02-24 16:13:22,661 INFO  [TestClient-0] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0xb61c0e3bad8010a
2018-02-24 16:13:22,669 INFO  [TestClient-0] zookeeper.ZooKeeper: Session: 0xb61c0e3bad8010a closed
2018-02-24 16:13:22,669 INFO  [TestClient-0-EventThread] zookeeper.ClientCnxn: EventThread shut down
2018-02-24 16:13:22,682 INFO  [TestClient-0] hbase.PerformanceEvaluation: Finished class org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest in 106ms at offset 0 for 1 rows (0.01 MB/s)
2018-02-24 16:13:22,683 INFO  [TestClient-0] hbase.PerformanceEvaluation: Finished TestClient-0 in 106ms over 1 rows
2018-02-24 16:13:22,683 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Summary of timings (ms): [106]
2018-02-24 16:13:22,683 INFO  [main] hbase.PerformanceEvaluation: [SequentialWriteTest] Min: 106ms  Max: 106ms  Avg: 106ms
2018-02-24 16:13:22,683 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0xd61c0e3bd4a0120
2018-02-24 16:13:22,692 INFO  [main] zookeeper.ZooKeeper: Session: 0xd61c0e3bd4a0120 closed
2018-02-24 16:13:22,693 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
  • ./hbase pe --nomapred --rows=1 sequentialWrite 1

  • I inserted 1 row with 1 client

  • How to understand above result in hbase pe

  • what is 95%,99%,99.999%....

  • what is Summary of timings (ms): Min: 106ms Max: 106ms Avg: 106ms

  • what is relation between them

Anyone please help to solve this......


Solution

  • As I understand from here, with sequentialWrite one runs sequential write test. Here 99.99th and others are basically performance metrics for write. for example 99.99th is 99.99th percentile of all the write time taken, which you can understand here. As you have only inserted one row, all 95%,99%,99.999% numbers are same, but they will be different if you test by inserting many rows.

    Same is case with Min, Max and Avg, as with only one test case, they are same. if there are 10 rows inserted, Avg will be average of time taken of all the rows.