最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

directed acyclic graphs - Can we trigger one dag2 from dag 1 and then again dag3 from dag2 in airflow? - Stack Overflow

programmeradmin1浏览0评论

Using the airflow version 2.10.3, I'm trying to create interdependencies between a chain of dags. Details is as the following:

from __future__ import annotations

import json
from datetime import datetime, timedelta
from pathlib import Path
from airflow.operators.dagrun_operator import TriggerDagRunOperator
from airflow.configuration import conf
from orchestrator.config.environment import Environment
from orchestrator.core.models import DAG
from orchestrator.providers.optimus.config import Job

SCHEDULE = "0 2 * * 5"
with dag(
 dag_id="dag1",
 schedule_interval=SCHEDULE
 # some other parameters) as directed_acyclic_graph:
 transformationjob = Job(
   # some transformation job goes here, with some paths and a json config file is referred
 ).to_task

And DAG2 as the following:

 # The import is the same as the above DAG
 with DAG(
 dag_id="dag2",
 schedule=None,
 # same as above DAG) as directed_acyclic_graph:
 
   anothertransform_job= Job(
     # same as DAG1 some transformation happpens).to_task

And in the same way, I have one more dag called DAG3 which has some tasks, how do I create the interdependency that DAG1 calls DAG2 and then DAG2 triggers DAG3?

与本文相关的文章

发布评论

评论列表(0)

  1. 暂无评论