Friday, July 23, 2021

Kafka Cluster Monitoring with Prometheus and Grafana

Grafana is popular open-source solution for monitoring applications. It will provide graphical dashboards to build monitoring visualization.

Any graphical view required data or metrics, so metrics data will be provided by Prometheus.

Prometheus is another monitoring tools to pull data from different application with help of JMX Exporter Agent.

Grafana is ability to connect Prometheus to pull metrics data and it will be represented as graphical view. We can build nice dashboards to show metrics data in different graphical view in the Grafana web UI.




Setup a Zookeeper Cluster


Setup a Kafka Cluster


Following are steps to Demonstrate Kafka Cluster Monitoring.


Setup Prometheus for Kafka Cluster

Install Grafana

Configure Prometheus Data source in Grafana

Create Kafka Cluster Dashboard


Setup Prometheus for Kafka Cluster

Follow the below article to setup Prometheus for Kafka cluster.


Install Grafana

Follow the below article to install Grafana on windows.

Configure Prometheus Data source in Grafana

Now its time to configure Prometheus Data source in Grafana. Prometheus already have metrics data which is pulled from Kafka with help of JMX exporter Java agent. That is already covered in the previous step.


Access Grafana Web UI with following URL








Home page click on Setting and Under Configuration click on “Data sources



Click on Add Data source.



Select Prometheus Data source in the List.



Provide Prometheus URL where its running. Default port is 9090. Once provided required information click on Test &Save.  Grafana successfully connected to Prometheus data source.







Create Kafka Cluster Dashboard

Grafana home page click on Dashboard icon and Click on Manage


There is free available Grafana dashboards for Kafka. Creating Grafana dashboard is very easy and its just need to import Dashboard JSON file.


Go to following URL and get Grafana Kafka overview dashboard file to local.




Import “kafka-overview.json” file into Grafana dashboard.


Click on import button.



Click on Upload JSON file button and Select “kafka-overview.json” file from local drive.


Once selected file, click on import then Dashboard will be imported into Grafana.


Very important is Job Name in Prometheus yml (prometheus.yml) file and Grafana Dashboard JSON file (kafka-overview.json) Job name should be same.



Go to Dashboards in the Grafana home page and select Kafka Overview Dashboard.


We Can see Dashboard with many panels and all metrics will be represented as Graphs.


Dashboard Screen: 1



Dashboard Screen: 2



Dashboard Screen: 3



Dashboard Screen: 4




  1. Both text and pictures are great. I will visit often and subscribe.
    Please come to our amazing website and have fun. 오피사이트

  2. When you include your devotees in large numbers, you can begin to take a gander at adapting TikTok. Organizations have been known to pay $200 to $20,000 per marked video advanced by powerhouses, contingent upon the singular's degree of impact. buy tiktok accounts


Recent Posts

Recent Posts Widget

Popular Posts