About table design in mysql or choose Hbase

Question

Our product application scenarios is like this:

1. store informations.
2. the count use number is about 80 million.
3. user have many properties, about 1000 thousand. The properties may be many different types: Date, Boolean, Long, Int .... or Null.

What we do on these informations is like this:

1. get one user's information（serveral propertys）
2. select users from computing properties.For example: find users whose propertyA>10 && propertyB=true &&...

Now, what we use is Hbase like a table with wide columns, but I think may be it is not suitable. When we get one or several user's information, it often happens time out, so we put a cache in the middle, but we can't put all the information in the cache, and then time out is still there! So, I think maybe hbase is not the best chance, because our datas is not too big to use hbase? If we choose mysql?

How to solve our problems?

Answer 1

There are few things which you need to analyze before coming to any conclusion, some of them are:

1. What the size of your data? no in terms of number of records but in terms of size of datafile?
2. What kind of HBase installation you are using? Is it standalone or clustered?
3. Is calling get/scan from HBase is getting timed-out?

IMO, if HBase will not give you any good performance results then it will not deteriorate the performance. But yes, the real use of any NoSQL db will be in clustered environment where data is multiplying.

If, you want to go with mysql or any other RDBMS then you have to think about db cluster, table sharding, load balancing and implementing caching at application level.

Overall, I think you should look at the installation of your HBase and try to tune it for your application.