I have an azure data factory pipeline that copies a file from sftp folder and drops to azure blob storage. From there I want to link it to a databricks notebook and do some processing.
Right now I triggered manually but I would like the ADF pipeline to trigger when the file is created in the sftp folder. searching online I see 4 years ago someone used a azure logic app to accomplish this.
Is there a newer way? That was 4 years ago. Is there some new built in functionality to have ADF trigger directly?
Unfortunately ADF doesnt have event trigger based on traditional SFTP aspect. You can use Logic app/Power automate and in turn trigger the ADF pipeline.
You can use the SFTP support in Azure blob storage feature : MSFT doc : https://learn.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support
And then the blob trigger of ADF would work