Data Engineer's Lunch: CI/CD on the Data Lakehouse with Project Nessie
An insightful discourse on implementing Continuous Integration and Continuous Delivery (CI/CD) in a Data Lakehouse environment using Project Nessie.
Continuous Integration and Continuous Delivery (CI/CD) is a software development practice that aims to improve the quality and speed of software delivery. In a data lakehouse environment, CI/CD can be used to automate ingesting, transforming, and loading data.
Project Nessie is an open-source project that provides a Git-like approach to version control for data lakehouse tables. Project Nessie can be used to implement CI/CD for data lakehouse environments by providing a way to track changes to data over time and to automate the process of deploying changes to production.
In this lunch, we will discuss the benefits of implementing CI/CD in a data lakehouse environment and how Project Nessie can achieve this. We will also discuss some of the challenges of implementing CI/CD in a data lakehouse environment and how to overcome them.
Key takeaways:
CI/CD can be used to improve the quality and speed of software delivery in a data lakehouse environment.
Project Nessie is an open-source project that can be used to implement CI/CD for data lakehouse tables.
There are a number of challenges to implementing CI/CD in a data lakehouse environment, but these challenges can be overcome.
Associated Github: Coming Soon!
Accompanying SlideShare: Coming Soon!
Sign Up For Our Newsletter: http://eepurl.com/grdMkn
Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers...
Cassandra.Link:
https://cassandra.link/
Follow Us and Reach Us At:
Anant:
https://www.anant.us/
Awesome Cassandra:
https://github.com/Anant/awesome-cass...
Email:
[email protected]
LinkedIn:
/ anant
Twitter:
/ anantcorp
Eventbrite:
https://www.eventbrite.com/o/anant-10...
Facebook:
/ anantcorp
Join The Anant Team:
https://www.careers.anant.us
#data #dataengineering #cicd #datalakehouse #projectnessie
