Automate Your Delta Lake or Practical Insights on Building Distributed Data Mesh
Automate Your Delta Lake or Practical Insights on Building Distributed Data Mesh
Databricks Data + AI Summit 2022
We all live in the exciting times and the hype of Distributed Data Mesh (or just mess). This talk will cover a couple architectural and organizational approaches on achieving Distributed Data Mesh, which is essentially a combination of mindset, fully automated infrastructure, continuous integration for data pipelines, dedicated team collaborative environments, and security enforcement.
As a Data Leader, you’ll learn what kinds of things you’d need to pay attention to, when starting (or reviving) a modern Data Engineering and Data Science strategy and how Databricks Unity Catalog may help you automating that.
As DevOps, you’ll learn about the best practices and pitfalls of Continuous Deployment on Databricks With Terraform and Continuous Integration with Databricks Repos. You’ll be excited how you can automate Data Security with Unity Catalog and Terraform.
As a Data Scientist, you’ll learn how you can get relevant infrastructure into “production” relatively faster.
#AI #Data #Databricks #DeltaLake #Lakehouse #MLOps
