Automated Testing and Deployment of DAGs
Presented by Austin Bennett at Airflow Summit 2024.
DAG integrity is critical. So are coding conventions, consistency in standards for the group. In this talk, we share the various lessons learned for testing/verifying our DAGs as part of our GitHub workflows [ for testing as part of the pull request process, and for automated deployment - eventually to production - once merged ]. We dig into how we have unlocked additional efficiencies, catch errors before they get deployed, and generally how we are better off for having both Airflow & plenty of checks in our CI, before we merge/deploy.
