Search code examples
azureazure-devopsazure-pipelinesazure-synapse

Azure Synapse - Import python scripts in notebooks

Am using Azure Synapse in combination with jupyter notebooks: enter image description here

Many of my jupyter notebooks import some custom python scripts like the util-import: enter image description here

However, there's no option hold *.py files in Azure Synapse. Always when i use the import functionalty, the *.py is transformed to a notebook (On laptop it was util.py, after Synapse import it's a notebook):

enter image description here

How can custom *.py files be used in Azure Synapse Notebooks, without transforming them from *.py to notebook?

Solution

  • There are several options for adding python to Synapse. You can manage them at the workspace, pool, or session level. The method I've had the most success with is loading from PyPi to a Spark Pool, but you can also upload Wheel or JAR files to your workspace and reference those in your notebooks. One caveat on using PyPi is that if the package has a dependency on C library packages, Synapse won't load it into your pool.