Serverless ETL Pipeline Demo with Amazon AWS Glue Studio | AWS Glue Studio Tutorial

Опубликовано: 20 Февраль 2025
на канале: Tech Eclipser

878

End to End Serverless ETL Pipeline Demo using AWS Glue Studio. We would be reading the data from S3 Bucket and would set up crawlers to get the data from the S3 into Data Catalog tables. Once the Data is loaded we will set up ETL Pipeline using AWS Glue Studio that would cover basic Enrichments, Filters, Join Aggregation, and Loading Data back to S3 Bucket.

Chapters
00:00 Introduction
00:10 Introduction to AWS Account
01:45 AWS Glue ETL Usecase Intro
02:32 AWS IAM Rule Setup
04:47 ETL Data Setup
07:24 AWS S3 Data Setup
11:08 AWS Glue Database & Crawler Setup
17:44 AWS S3 Permission Issue Resolution
19:30 AWS Athena Introduction
20:50 AWS Glue Studio Job Setup
23:47 AWS Glue Transformation Demo
25:08 AWS Glue Join Demo
26:05 AWS Glue Custom Transformation Demo
27:02 AWS Glue Select from Collection Demo
27:30 AWS Glue Target Demo
28:20 AWS Glue Data Preview Intro
29:23 AWS Glue Job Execution
31:00 AWS Glue Demo Result

References :
AWS Main Page: https://aws.amazon.com/
AWS Free Tier Details: https://aws.amazon.com/free/

Sample Data Used:
https://open.toronto.ca/dataset/parki...
https://open.toronto.ca/dataset/parki...

AWS S3 Bucket Name Guidelines: https://docs.aws.amazon.com/AmazonS3/...

#ETL #Glue #Serverless #AWS #Glue-Studio #S3 #Athena #IAM