How can I use relative paths from an R Notebook in Databricks?

Question

When working in R I usually store my functions in a folder ./R/

To bring these functions to the workspace I resort to the here::here() function. For a small script my code would start like:

library(here)
source(here::here("R", "custom_function1.R"))
source(here::here("R", "custom_function2.R"))

In this way sharing the projects amongst colleagues is quite straightforward.

I'm using Azure-Databricks Notebooks for a project. When using here::here() the function points to the server.

Is there a way to define relative paths in the Azure-Databricks Notebooks to keep functions decoupled from the Notebook itself?

Answer 1

If I am reading your problem correctly you can use relative paths.

Caveat: I am not sure however if you are looking for an equivalent to here library. I have not seen an equivalent to that but relative paths do work.

See:

https://docs.azuredatabricks.net/user-guide/notebooks/notebook-use.html#link-to-other-notebooks

Run a notebook from another notebook

You can run a notebook from another notebook by using the %run <notebook> magic command. This is roughly equivalent to a :load command in a Scala REPL on your local machine or an import statement in Python. All variables defined in become available in your current notebook.

%run must be in a cell by itself, because it runs the entire notebook inline.

Note You cannot use %run to run a Python file and import the entities defined in that file into a notebook. To import from a Python file you must package the file into a Python library, create an Azure Databricks library from that Python library, and install the library into the cluster you use to run your notebook.

Example

Suppose you have notebookA and notebookB. notebookA contains a cell that has the following Python code:

x = 5

Even though you did not define x in notebookB, you can access x in notebookB after you run %run notebookA.

%run /Users/path/to/notebookA
print(x) # => 5

Relative Paths

To specify a relative path, preface it with ./ or ../. For example, if notebookA and notebookB are in the same directory, you can alternatively run them from a relative path.

%run ./notebookA
print(x) # => 5

thus

%run ../someDirectory/notebookA # up a directory and into another    
print(x) # => 5

For more complex interactions between notebooks, see Notebook Workflows.