AWS Reinvent 2021 Recap + DataOps Lakehouse Automation with Amazon Redshift and Apache Airflow

Опубликовано: 10 Октябрь 2024
на канале: Generative AI on AWS

1,165

Meetup event link: https://www.meetup.com/data-science-o...

RSVP Webinar: https://www.eventbrite.com/e/webinark...

[00:00] Talk #0: Introductions and Meetup Announcements By Chris Fregly and Antje Barth

[03:43] Talk #1: AWS Lakehouse Automation: Loading and unloading data from Amazon Redshift using Apache Airflow

Speaker:
Gary Stafford, Sr. Solutions Architect at AWS.

Abstract:
According to AWS, “DevOps is the combination of [well-established] cultural philosophies, practices, and tools that increases an organization's ability to deliver applications and services at high velocity.” In my experience, organizations that have well-developed, mature DevOps practices, do not always apply the same approach and level of operational rigor to their data analytics and machine learning processes. This may be a result of organizational silos or a lack of awareness of evolving DataOps and MLOps patterns. These methodologies apply proven DevOps practices of data analytics and machine learning: SCM, CI/CD, testing, and build and deployment pipeline automation.

In this presentation and brief demonstration, based on the video demonstration, AWS Lakehouse Automation: Loading and unloading data from Amazon Redshift using Apache Airflow, we will learn how to use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to programmatically load, unload, test, catalog, query, and monitor data using DevOps-like DataOps practices.

[49:26] Talk #2: AWS re:Invent 2021 - re:Cap - New AI/ML launches

Speaker:
Chris Fregly, Principal Engineer at AWS.

Abstract:
Learn about the new AI/ML services launched at the AWS re:Invent conference in December 2021.

RSVP Webinar: https://www.eventbrite.com/e/webinark...

Zoom link: https://us02web.zoom.us/j/82308186562

Related Links
=============
O'Reilly Book: https://www.amazon.com/dp/1492079391/
Website: https://datascienceonaws.com
Meetup: https://meetup.datascienceonaws.com
GitHub Repo: https://github.com/data-science-on-aws/
YouTube: https://youtube.datascienceonaws.com
Slideshare: https://slideshare.datascienceonaws.com