Optimizing Kafka Producers and Consumers: A Hands-On Guide
blog post: https://blog.rockthejvm.com/optimizin...
code: https://github.com/polyzos/kafka-stre...
data: https://www.kaggle.com/datasets/mkech...
parallel consumer library: https://github.com/confluentinc/paral...
Contents:
0:00 intro, setup
4:10 code walkthrough
10:14 optimizing Kafka producers
18:05 optimizing Kafka consumers
This video is a hands-on guide to maximizing the performance of your Kafka producers and consumers. We will talk about the artchitecture of a typical application with Kafka, how producers work and how to tune Kafka producers for either max throughput or min latency. Then, we will switch sides to consumers and will discuss how you can scale your Kafka consumers vs number of partitions, and how to leverage multithreading inside a single consumer for 1000x increase in consumption performance.
Blog and code by Giannis Polyzos: https://github.com/polyzos
Follow Rock the JVM on:
LinkedIn: / rockthejvm
Twitter: / rockthejvm
Blog: https://blog.rockthejvm.com
-------------------------------------------------------------------------
Home: https://rockthejvm.com
-------------------------------------------------------------------------
