Why magic command doesn't work when running pipeline?

Question

I am running a pipeline in Azure Synapse Analytics, in this pipeline I have a notebook that call another inside his script with command like :

%run /folder1/extensions

Pipeline execution raise an error with this magic command :

MagicUsageError: Cannot read notebook /folder1/extensions. The possible reason is that the notebook doesn't exist. More details please visit https://go.microsoft.com/fwlink/?linkid=2173018
--> PartnerException: Exception of type 'Microsoft.Analytics.SynapseNotebookService.Infrastructure.Exceptions.ClientExceptions.PartnerException' was thrown.

When I execute manually the notebook the magic command works. What is weird is that the command work in pipeline but with other notebook in the magic command in the same folder.

Where can I be wrong and what can I do ?

Answer 1

You need to pass in the absolute path as in this question below (the error message is the same):

Synapse : Execute the Magic Command (%run) Notebook from Pipeline

Then also dont forget to publish your changes.

An example absolute path is something like: /mnt/workspace/folder/MyNotebook

/mnt is the root of your workspace storage account.

/workspace is a common directory where you might store your notebooks.