Installing Apache Kafka and Zookeeper on Windows

In this article, I’ll talk about how to Install, configure and start Apache Zookeeper and Apache Kafka Server on Windows OS.

Prerequisite

• JRE running on your machine and path must set to Environment Variable
• Any Zip tool like 7-zip, WinZip or WinRAR.
• Download and Extract Apache Zookeeper using 7-zip.
• Download and Extract Apache Kafka using 7-zip

ZooKeeper Installation Instructions:

• Go to the conf directory of your Zookeeper. For me its under D:\Softwares\apache-zookeeper-3.5.5
• Copy and rename zoo_sample.cfg to zoo.cfg file.
• Open and Edit dataDIr=/tmp/zookeeper to dataDir=D:\Softwares\apache-zookeeper-3.5.5

• Add entries in System Environment Variables
• ZOOKEEPER_HOME=D:\Softwares\apache-zookeeper-3.5.5
• Append D:\Softwares\apache-zookeeper-3.5.5\bin to PATH system variable.
• Open command prompt and type zkserver to start the Zookeeper application.
• You can easily edit the default port ( 2181) in zoo.cfg file.

Congratulations, ZooKeeper is up and running on port 2181.

Installing Apache Kafka Instructions

• Please make sure that the Zookeeper instance is up and running before starting any kafka server.
• Go to the Kafka Installation directory (D:\Softwares\apache-tomcat-8.5.37) and run below command.
• .\bin\windows\kafka-server-start.bat .\config\server.properties

Congratulations, Apache kafka is up and running on port 9092

  

Create Topic

To create a topic, we need to run the kafka-topics script

• Go to Apache kafka installation directory and navigate to bin\windows folder (D:\Softwares\kafka_2.11-2.2.0\bin\windows).
• Execute below command to create topic where <TOPIC_NAME> should be name of your topic
• kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic <TOPIC_NAME>

Create Producer

Now the topic has been created, Lets start a producer so that we can published some message against sales topic.

·       Go to Apache kafka installation directory and navigate to bin\windows folder (D:\Softwares\kafka_2.11-2.2.0\bin\windows).

·       Execute: kafka-console-producer.bat --broker-list localhost:9092 --topic sales

Create Consumer

Let’s create a consumer which should subscribe to sales topic to get all the messages.

·       Go to Apache kafka installation directory and navigate to bin\windows folder (D:\Softwares\kafka_2.11-2.2.0\bin\windows).

·       Execute: kafka-console-consumer.bat --bootstrap-server localhost:2181 --topic sales

By running all four components (zookeeper, kafka, producer, and consumer) in different terminals, we will be able to enter messages from the producer’s terminal and can see them appearing in the consumer’s terminal and If everything works fine, you will be able to push and see messages.

Bonus Installation:

There are multiple UI applications which can be use for monitoring apache kafka. It will display the information like total brokers, topics, partition and even lets you view messages as well.

One of the tools is Kafdrop which we will install right now and see all those details on UI.

Requirements:

• Java 8
• Kafka (0.8.1 or 0.8.2 is known to work)
• Zookeeper (3.4.5 or later)
• Download and Extract Kafdrop using 7-zip.

Installation

·       Go to the Kafdrop directory and run mvn clean package which will generate an executable JAR in target directory.

·       Go to target directory and run

o   java -jar ./target/kafdrop-<version>.jar --zookeeper.connect=localhost:2181

·       Open a browser and navigate to http://localhost:9000 to view the Kafdrop UI.