Search code examples
amazon-web-serviceshivebigdatahadoop2hue

Possible ways to scale Hue + HiveServer2 setup


We have a Hue + HS2 (Hive Server 2) setup on AWS which has been working fine up until about 80 users, but now we are seeing a lot of slowdown on HS2 side. We are looking into doing a horizontal scaling by adding one more Hue + HS2. But, I feel there maybe some scope for vertical scaling.

Does anyone know some good parameters to tinker with and what each one with fetch us in terms of performance?


Solution

  • Hue HA is typically done with HTTPD, Apache or NGINX: http://gethue.com/automatic-high-availability-with-hue-and-cloudera-manager/

    For HS2, we recommend HAProxy: http://gethue.com/how-to-optimally-configure-your-analytic-database-for-high-availability-with-hue-and-other-sql-clients/