I have used print statement to print the taskgroup. Where can I possibly see the value getting printed? In Airflow we can only see specific task logs I assume. Can we see the print statement value which is not defined under any tasks?
import datetime
from airflow import DAG
from airflow.operators.empty import EmptyOperator
from airflow.operators.python_operator import PythonOperator
from airflow.operators.latest_only import LatestOnlyOperator
from airflow.decorators import dag, task, task_group
def my_function():
print("hello")
with DAG(
dag_id="my_dag_name",
start_date=datetime.datetime(2023, 7, 7),
schedule="@daily",
catchup=False
):
latest_only = LatestOnlyOperator(task_id="latest_only")
first = EmptyOperator(task_id="first")
last = EmptyOperator(task_id="last")
@task_group()
def group():
options = ["branch_a", "branch_b", "branch_c", "branch_d"]
for option in options:
t = EmptyOperator(task_id=option)
group1 = group()
group2 = group()
print(group1)
first >> [group1, group2] >> last
latest_only >> first
You can see those logs under $AIRFLOW_HOME/logs/scheduler/$date/$dag_name.py.log
$AIRFLOW_HOME - you airflow home directory
$date - for example 2023-08-12
$dag_name.py.log - path of the dag under dags_folder. for example if you create sub_folder by name "temp" and have there my_dag_name.py then it will be temp/my_dag_name.py.log