Search code examples
apache-sparkpysparkdatabricksspark-submit

Is that possible to run "spark-submit" in databricks without creating jobs ? if yes ! What is the possiblities,

I am trying to execute spark-submit in databricks workspace notebook without creating jobs, Help me!

Solution

  • No, that is not possible like one would do with /bin/spark-submit as it does not fit in with their notebook approach to making things easier for less techy persons.

    The closest you can get is as stated here: https://docs.databricks.com/dev-tools/api/latest/examples.html#create-a-spark-submit-job