Search code examples
hadoop-yarnsnowflake-cloud-data-platform

What is the Amazon Snowflake execution engine


When you are on Hadoop, you can have YARN manage the Hadoop jobs, resources, etc ...

What is the equivalent form for Amazon's Snowflake?

Hadoop (HDFS) is to YARN as Snowflake is to __________


Solution

  • Interesting question here. To level set, Snowflake is a Data Warehouse SaaS offering that is deployed on both AWS and Azure, I wouldn't categorize it as "Amazon's Snowflake".

    Snowflake does not have a separate tool managing resources for the data warehouse clusters. It is built into it's 3 tier architecture. This link will provide some details on that: https://www.snowflake.com/product/architecture/

    As Snowflake is a cloud native data warehouse the inherent elasticity of the cloud does give administrators the ability to dynamically scale compute resources both vertically and horizontally to address things such as queuing and parallelism.

    Hope this helps.