Spark Monitoring: Basics
In this video I cover how to use the default UI for monitoring Apache Spark. I use Azure Databricks to demonstrate, but most of the methods are the same in any Spark environment. I focus most on how I use it in my day to day work as a Data Engineer.
Topics included: Spark UI, query plan basic, Ganglia Metrics, and driver logs.
Monitoring Streams with Custom Streaming Query Listener: • Data Collab Lab: Monitoring Spark Str...
More from Dustin:
Website: dustinvannoy.com
Twitter: @dustinvannoy
Github: https://github.com/datakickstart
![Wet Hands - Minecraft (Piano Tutorial) // [Synthesia]](https://images.videosashka.com/watch/Z7ZkOTDyVeE)
