Installing Apache Kafka in Cloudera Quickstart VM
Apache Kafka is a very commonly used messaging queue, used in most of the big data streaming pipelines. As the matter of fact Kafka is not pre-installed in Cloudera quickstart vm, the most commonly used Big data virtual machine.
In this video i have explained how to install apache kafka in cloudera quickstart vm in very simple steps.
Additional links:
Download cloudera quick start VM: https://www.cloudera.com/downloads/qu...
Commands used in the video:
cd /etc/yum.repos.d
wget http://archive.cloudera.com/kafka/red...
sudo yum clean all
sudo yum install kafka
sudo yum install kafka-server
sudo service kafka-server start
--------------------------
sudo jps
---------------------------
Start zookeeper (if not already running)
bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties
Create topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic gkcodelabs
List topics
bin/kafka-topics.sh --list --zookeeper localhost:2181
Start producer
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic gkcodelabs
Start comsumer
bin/kafka-console-consumer.sh --zookeeper localhost:2181 -topic gkcodelabs --from-beginning
