Search code examples
azureazure-machine-learning-servicemicrosoft-fabric

How can I access a OneLake in Fabric from Azure Machine Learning?


As Azure Machine Learning has more maturity compared to Synapse Data Science from Fabric that I recently been using, I wanted to know:

  • If there is a way to access OneLake data (Files or Tables) from an Azure ML instance, within the same subscription ?
  • If so, how to do it ? (I did not find any documentation or tutorial for this case. The most similarish thing I found is an explanation on how to make model endpoint from Azure ML available in Fabric)
  • What are the possible bottlenecks (data transfer if I need to use an potential intermediate storage for example)

Thanks in advance


Solution

  • To access OneLake data from an Azure ML instance within the same subscription, you can use Azure Data Lake Storage Gen2 (ADLS Gen2) as the intermediate storage.

    1. Set up ADLS Gen2 and ensure Azure ML has access.
    2. Use the Azure ML SDK for Python to access data from ADLS Gen2.
    3. Bottlenecks:Data transfer latency and costs.

    You can use the Datastore and Dataset classes in the Azure ML SDK to access data.

    I hope this helps :)