I was trying to migrate a data from SQL db to Hadoop. I have successfully done this by configuring Hive, HBase & Hadoop.
My problem is that I was using Birt & Tableau with my SQL db and was able to load 10 million data in 5-10 mins, but my newly configured Hadoop, Hive & HBase System takes around 50 mins to fetch 10 million entries. How can I improve this performance?
As Hadoop is specially developed for processing tons of data, why I am not able to do so?
Is there any special configuration for performance?
After lot of research and for the answer of this question I went through HDP as well. Then I come across a scenrio that we cannot compare the performance of SQL Db with Hadoop as both are used for different purposes.
Also Hadoop will show its performance only after the data crosses a limit of Several TB's i.e. the case in which SQL Database fails. So it will be better if one should check first whether for an Application. If there is a requirement of Performance, choosing Hadoop is not a good option; go for the SQL Databases. But if the Application is such that it will have huge amount of Data & one has to do an analysis of such huge data where SQL DB fails; in such case Hadoop is prevalent.