Trying to start several ADf triggers using asyncio with azure python SDK, I get RuntimeError: got Future <Future pending> attached to a different loop

Question

I'm trying to write a simple python script to start several triggers in my Azure Data Factory. However, when I tried using the aio variants of the azure SDK classes, I get the following error:

RuntimeError: Task <Task pending name='Task-4' coro=<stop_trigger() running at /home/atlevesque/Sources/CustomDevopsScripts/trigger_pythonTests.py:37>> got Future attached to a different loop

Here is what my script looks like:

import asyncio
from azure.identity.aio import DefaultAzureCredential
from azure.mgmt.datafactory.aio import DataFactoryManagementClient


rg_name = "MY_RG_NAME"
adf_name = "MY_ADF_NAME"
subscription_id = "MY_SUBSCRIPTION_ID"

credentials = DefaultAzureCredential()

adf_client = DataFactoryManagementClient(credentials, subscription_id)

async def start_trigger(adf_client: DataFactoryManagementClient, trigger_name: str, resource_group_name=rg_name, factory_name=adf_name):
    print(f"Starting trigger {trigger_name}")
    await adf_client.triggers.begin_start(resource_group_name=rg_name, factory_name=adf_name, trigger_name=trigger_name)
    print(f"Trigger {trigger_name} started")

async def main(adf_client: DataFactoryManagementClient, triggers_to_start: list):
    trigger_start_tasks = [
        asyncio.create_task(start_trigger(adf_client, trigger))
        for trigger in triggers_to_start
    ]

    for trigger_task in (trigger_start_tasks):
        await trigger_task

triggers_to_start = [
    "Trigger_A",
    "Trigger_B",
]

asyncio.run(
    main(adf_client, triggers_to_start)
)

My script actually works when I await each task before starting the next one (but then there is no point in using aio...)

Thanks in advance for your help!

Answer 1

You can try either of the following ways to resolve this RuntimeError: Task <Task pending name='Task-4' coro=<stop_trigger() running at /home/atlevesque/Sources/CustomDevopsScripts/trigger_pythonTests.py:37>> got Future attached to a different loop error:

1. Add asyncio.get_event_loop().run_until_complete(main())` or `app.run(main())

2. Add loop = asyncio.get_event_loop() app.run(debug=1,loop=loop)

References: https://github.com/pyrogram/pyrogram/issues/391 and RuntimeError: Task got Future <Future pending> attached to a different loop