Data Engineer's Lunch #76: Airflow and Google Dataproc
In Data Engineer's Lunch #76, Arpan Patel will cover how to connect Airflow and Dataproc with a demo using an Airflow DAG to create a Dataproc cluster, submit an Apache Spark job to Dataproc, and destroy the Dataproc cluster upon completion.
Accompanying GitHub Repo: https://github.com/Anant/example-airf...
Accompanying Blog: Coming Soon!
Accompanying SlideShare: https://www.slideshare.net/AnantCorp/...
Sign Up For Our Newsletter: http://eepurl.com/grdMkn
Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers...
Cassandra.Link:
https://cassandra.link/
Follow Us and Reach Us At:
Anant:
https://www.anant.us/
Awesome Cassandra:
https://github.com/Anant/awesome-cass...
Email:
[email protected]
LinkedIn:
/ anant
Twitter:
/ anantcorp
Eventbrite:
https://www.eventbrite.com/o/anant-10...
Facebook:
/ anantcorp
Join The Anant Team:
https://www.careers.anant.us
#airflow #dataproc #etl
