Search code examples
azureapache-sparku-sql

Creating Spark job in Data Lake instead of a U-SQL job

Is it possible to create Spark job in Data Lake instead of a U-SQL job ?

Solution

  • Here are the options for big data services in Azure:

    • Azure Data Lake Analytics currently has only U-SQL jobs and not Spark.
    • Azure HDInsight supports Spark jobs.
    • Azure Databricks supports Spark jobs.
    • **Cloudera on Azure ** supports Spark jobs.
    • **Hortonworks on Azure ** supports Spark jobs.