My Journey from DevOps to MLOps: Part 2.2/3 How to Configure Run Databricks Job using Apache Airflow
#mlops #apacheairflow #dag
LetsDevOps: A complete journey to learn and implement MLOps.
Part 2.2/3: How to Configure, Run Databricks Job using Apache Airflow
✅ Install databricks connector
✅ Setup Connection from Airflow to databricks
✅ Create DAG with Databricks workflow
✅ Run airflow DAG to trigger Databricks workflow
🚀🤩Blog URL: https://www.letsdevops.net/post/letsd...
GitHub: https://github.com/sumitraj0103/Letsd...
Part 1/3: https://www.letsdevops.net/post/letsd...
Part 2/3: https://www.letsdevops.net/post/letsd...
Part 2.1/3: https://www.letsdevops.net/post/letsd...
▬▬▬▬▬▬ Content Timestamp ⏰ ▬▬▬▬▬▬
Introduction: 00:03
Context: 00:17
Architecture: 00:32
Prerequisite: 00:49
Create Databricks PAT token: 01:42
Install Databricks Connection: 03:11
Create Connection: 05:49
Create DAG: 06:46
Update Job ID in DAG: 07:50
Update Connection ID in DAG: 08:55
Enable DAG: 09:38
Trigger DAG: 09:53
DAG type: 11:07
