Search code examples
pythonbashcluster-computingdatabricksazure-databricks

The init script works fine in databricks notebook but fails when attached to cluster

I want to attach the init script (which has all library dependencies) with an interactive cluster in databricks. The init script look like the below attachment and works fine in databricks notebook. enter image description here

I want this shell script to run at cluster start, but when I configure it as a init script for the cluster it returns an error. Here's how I have attached the init script to my cluster, enter image description here

The error look like this, enter image description here

I found a url discussing the same problem but solution provided here (changing extension of the file from .sh to .bash) doesn't work for me. Please help me fix this issue.

Solution

  • Please do remove extra (") to successfully install init script.

    As per my repro: I have used the same code as above.

    enter image description here

    Result: Cluster terminated Reason: Init Script Failure

    enter image description here

    After removing extra quote (").

    enter image description here

    Result: Init_Scripts successfully installed.

    enter image description here