Twitter Data Pipeline using Airflow for Beginners | Data Engineering Project

Опубликовано: 20 Сентябрь 2022
на канале: Darshil Parmar

322,682

8.9k

➡️ Check Out My Data Engineering Bootcamp: https://bit.ly/3yXsrcy
USE CODE: COMBO50 for a 50% discount

This is End-To-End Data Engineering Project using Airflow and Python. In this project, we will extract data using Twitter API, use python to transform data, deploy the code on Airflow/EC2 and save the final result on Amazon S3

Code and commands used in the project - https://github.com/darshilparmar/twit...

Other resources:
Twitter API -   / twitter-api
Python (Installation & Basics) -    • Python Tutorial - Python for Beginner...
Setup AWS Account -    • AWS Account Setup - Step by Step proc...
Airflow Doc - https://airflow.apache.org/docs/

Join Data With Darshil Discord Server:   / discord

Timestamps
0:00 Project introduction
0:56 Section 1 (Prerequisite)
2:21 Section 2 (Understand basic concept)
7:55 Section 3 (Project execution)

👦🏻 My Linkedin -   / darshil-parmar
📷 Instagram -   / darshilparmarr
🎯Twitter -   / parmardarshil07

🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟

3 Books You Should Read
📈Principles: Life and Work: https://amzn.to/3HQJDyP
👀Deep Work: https://amzn.to/3IParkk
💼Rework: https://amzn.to/3HW981O

Tech I use every day
💻MacBook Pro M1: https://amzn.to/3CiFVwC
📺LG 22 Inch Monitor: https://amzn.to/3zk0Dts
🎥Sony ZV1: https://amzn.to/3hRpSMJ
🎙Maono AU-A04: https://amzn.to/3Bnu53n
⽴Tripod Stand: https://amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: https://amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: https://amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: https://amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: https://amzn.to/3IRpiL2

👉Data Engineering Complete Roadmap:    • Data Engineer Complete Road Map 2022
👉Data Engineering Project Series:    • Data Engineering Project Portfolio
👉Become Full-Time Freelancer:    • Best Freelancer Series 2024
👉Data With Darshil Podcast:    • Podcast Series - Data With Darshil

✨ Tags ✨
data engineering project, complete data engineering project for beginners, learn data engineering, data engineering roadmap

✨ Hashtags ✨
#airflow #dataengineering #project