Search code examples
azure-data-factorykqlazure-log-analytics

Log the status of the ADF (Azure Data Factory) pipeline run


I have an ADF Pipeline with a trigger that runs the pipeline once every day. I would like to find out the status of the pipeline run (latest pipeline run) and log it somewhere (maybe log analytics). How do I do that?

enter image description here

UPDATE:

I followed the solution suggested below however I don't see any logs on running the query ADFPipelineRun in log analytics. What am I missing?

enter image description here


Solution

  • In case if you want to have a log pertaining to only the recent run, then you would have to write a custom logic within your pipeline (like a script activity that would write in a database the status of the pipeline at the end)

    sample reference which is my blog: https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory/

    In case if you are comfortable querying a list of logs and filtering out the latest one based on some filter logics, then you can use log analytics or storage blob by enabling diagnostic settings

    enter image description here