Serverless ETL Pipeline Demo with Amazon AWS Glue Studio, Redshift and S3 | AWS Glue Studio Tutorial

Опубликовано: 20 Январь 2025
на канале: Tech Eclipser

2,142

End to End Serverless ETL Pipeline Demo using AWS Glue Studio, Redshift, and S3. We would be reading the data from the Redshift Database and would set up crawlers to get the data from the Redshift Database into Data Catalog tables. Once the Data is loaded we will set up ETL Pipeline using AWS Glue Studio to read the data from Redshift Database and apply the SQL Transformation on the same and persist the result into S3.

Chapters
00:00 Introduction
00:45 IAM Role Setup for Redshift, Glue, and S3 Permission
02:44 Redshift Join Query Preparation
04:32 Redshift JDBC Connection Setup
06:13 VPC Endpoint Setup for S3 Permission
07:38 Glue Crawler Setup for Redshift Tables
09:22 Glue Studio Job Setup using Redshift and S3
12:08 Glue Studio Job Setup SQL Transformation Setup
17:20 Associate the IAM Role with Redshift Cluster
19:16 VPC Endpoint Setup for Glue Permission
20:33 Successful Run and Data Verification
21:28 Thank You

References :
AWS Main Page: https://aws.amazon.com/
AWS Free Tier Details: https://aws.amazon.com/free/

Redshift Cluster Setup:
• Amazon Redshift Setup Demo and JDBC C...

AWS Glue Studio Serverless Pipeline:
• Serverless ETL Pipeline Demo with Ama...

AWS S3 Bucket Name Guidelines: https://docs.aws.amazon.com/AmazonS3/...

#ETL #Glue #Redshift #Serverless #JDBC #AWS #Glue-Studio #S3 #IAM #Amazon #DataWareHouse