How >> Operator Defines Task Dependencies In Airflow?
I was going through Apache Airflow tutorial https://github.com/hgrif/airflow-tutorial and encountered this section for defining task dependencies. with DAG('airflow_tutorial_v01',
Solution 1:
Airflow represents workflows as directed acyclic graphs. A workflow is any number of tasks that have to be executed, either in parallel or sequentially. The ">>" is Airflow syntax for setting a task downstream of another.
Diving into the incubator-airflow project repo, models.py
in the airflow
directory defines the behavior of much of the high level abstractions of Airflow. You can dig into the other classes if you'd like there, but the one that answers your question is the BaseOperator class. All operators in Airflow inherit from the BaseOperator. The __rshift__
method of the BaseOperator class implements the Python right shift logical operator in the context of setting a task or a DAG downstream of another.
See implementation here.
Post a Comment for "How >> Operator Defines Task Dependencies In Airflow?"