Search code examples
azure-databricks

In Databricks, call functions from notebooks with dbutils.notebook.run

In a Databricks brick, I am running this:

functions_notebooks = ["../functions/other_functions",
                      "../functions/general_functions"]

for notebook in functions_notebooks:
    dbutils.notebook.run(notebook, 60)

variable1, variable2, variable3 = function1_other_functions()

However, it cannot find the function. The status of the run is Succeeded for both notebooks. If I call the notebooks by separate in different bricks, which is impractical, then the functions work without issues.

The error is:

NameError: name 'function1_other_functions' is not defined
Solution

  • When you run a notebook with dbutils.notebook.run, the notebook will run in a different session, and you cannot use its variables and functions in the notebook you called it from.

    To be able to use functions, you will have to use %run "../functions/other_functions" for each notebook in a different cell.

    You can also call one notebook (with %run) and inside that notebook have many cells to call many notebooks.