Databricks CI/CD: Azure DevOps Pipeline + DABs

Опубликовано: 16 Февраль 2025
на канале: Dustin Vannoy

9,818

140

Many organizations choose Azure DevOps for automated deployments on Azure. When deploying to Databricks you can take similar deploy pipeline code that you use for other projects but use it with Databricks Asset Bundles. This video shows most of the steps involved in setting this up by following along with a blog post that shares example code and steps.

All thoughts and opinions are my own *

Blog post on DABs with Azure DevOps:   / integrating-databricks-asset-bundles-into-...

Prior videos on DABs...
Intro:    • Databricks CI/CD: Intro to Databricks...
Advanced:    • Databricks Asset Bundles: Advanced Ex...

More from Dustin:
Website: https://dustinvannoy.com
LinkedIn:   / dustinvannoy
Github: https://github.com/datakickstart

CHAPTERS
0:00 Intro
1:24 Repo overview
1:58 Service connection + Service Principal
4:11 Variable Group
5:13 Pipeline YAML review and changes
10:34 Release branch setup
11:35 Fix parallelization error
12:55 Test pipeline run
13:49 Add SP Permissions
17:48 Explain validation job
20:11 Setup production release
25:08 Review pipeline success
26:05 Outro