Search code examples
azure-data-factorylinked-service

ADF Pipeline - Shall we create separate Linked service for different tasks of same nature running in parallel


I've 4 data-bricks notebook activities to run in parallel to do the same task but for different parameters (variables). Do I need to create separate Linked service for each notebook connection when same Databricks workspace is being used but notebooks are different?

enter image description here

Here inside each If activity I have one databricks activity.

Q2. Will that affect the performance if I use the same linked service?

enter image description here

I've made 4 different linked service with different names but it seems to be a bad idea.


Solution

  • you should use the same linked service. Linked service is just a connection to your source and sink and the actual transformations are happening there irrespective of from where you call. So ideally there won't be any change in performance if you use same linked service or multiple linked service since the engine being triggered is the same.