I want to expose a front-end user interface to accept some input parameters from the users for a data pipeline in the backend and then display the output to the user again on the front-end user interface. I can have the data pipeline as an Airflow DAG which is run on-demand whenever the user submits the input params. Essentially, I want to have Airflow as the backend for a web-app. Can I have a continuously running DAG that serves web-page to the user and whenever they submit input params, the data pipeline in executed and the result is displayed?
I found a close example - https://towardsdatascience.com/10-minutes-to-building-a-machine-learning-pipeline-with-apache-airflow-53cd09268977 but I'm not sure if it serves a web-page continuously or only when the DAG is run and then stops when DAG completes.
The most common solution in projects that I have worked on, have involved surfacing Flask App or Flask App Builder objects through the plugins folder in your setup.
The second link should take you through a good example.
From here you can have you Flask app essentially call Airflow models or interact with the backend database by using the Session concept from airflow/utils/db.py.
Best of luck!