Event Driven Microservice Spring Boot with Apache Kafka

By -
Apache kafka logo

Apache Kafka in simple terms is a collection of logs of events when some data in a database changes, divided into topics. This is why it is called event driven architecture, when some service saves data in a database, you use Kafka to produce an event so that other services (wherever it is located) can listen on that event and do some computation whatever they want based on the event that is firing.

Kafka is usually used for microservices architecture, so instead of reading data from a database, each microservices communicate with Kafka events, which we call produce and consume.

Produce is Kafka term to emit a message to a specific topic, meanwhile consume is to listen to an event that contains data that is produced by some other microservices.

Monolithic application

Microservice architecture application

What is Event-driven Microservices

Microservice design is a way to develop an application and split it into small independently deployable services, each microservice has specific business capabilities that are separated to other microservice. Microservices make the development of some specific logic separated and be independent from each other. In a monolithic application if one fails it can impact the whole application, but with microservice if one microservice failed, it doesn’t affect other services. So, it prevents large-scale failure.

In request driven architectural style, communication between different microservices is not efficient. The request-driven style if you want some data, it then makes a single request to some service, and the client would wait until the service responds with the result.  

Event driven instead of waiting for an infinite period of time waiting for service response, the event driven messaging system will inform the other service when the response is ready via subscription to some specific topic.  Event-driven microservices code architecture provides real-time communication between microservices, so each microservice consumes the data in the form of events whenever other services produce the data somewhere out there.

Event Driven Microservice with Apache Kafka

Follow these steps to install Kafka on your local computer, so you can get a better understanding of how Kafka works. Using a simple spring boot microservices ping and pong example.

You need to have:

- Docker

- Java

- Git, to clone the example source code

1. Create docker-compose.yml to run Kafka server from docker

2. To run your Kafka server

docker-compose up -d

3. If you want to stop your Kafka server run this command, but don’t do this for now

docker-compose rm -fsv

4. Clone this full code

This is the full event driven microservice example using spring boot codes with Apache Kafka as messaging broker. You can clone this full code and test it yourself, understand better what microservice event driven software design.

Git repo: https://github.com/mudiadamz/event-driven-microservice-example.

Follow the Readme instruction on that repository to run and debug the application.

If you want to know what Apache Kafka is, there's a good explanation of what Apache Kafka is, and how it works for beginners from Confluent YouTube channel from here https://www.youtube.com/watch?v=06iRM1Ghr1k.

And what is event driven architecture is here the video explanation to help you understand better. Here from Coding tech watch the video here https://youtu.be/sSm2dRarhPo.


I am an experiences web developer, have a good problem-solving and deep understanding of software development principles. With comprehensive skill set in Java, PHP, Python, C and JavaScript. Able to do database thing like MySQL and Oracle. Along with expertise in frameworks like Spring, React, and Firebase. Have a solid understanding of software development methodologies, like Agile and Scrum. I applies best practices to ensure the development process is efficient and of the highest quality output.

Popular posts from this blog

Spring Kafka - how to use ReplyingKafkaTemplate send and reply synchronously

By -
Image
Microservice Spring with Apache Kafka Kafka is an asynchronous messaging queuing system, but when you have a case in your project that you need to have the result immediately from another service you can do that quite easily. Kafka from the client side produces a message to the server side, then waiting for the server to return back the message exactly what the client wants. It’s identified in the kafka header request named KafkaHeaders.CORRELATION_ID and a topic where the server should send. If the server side produces the same correlation id to the same topic in the request, then the client side will catch that message as its reply. Although you can work with kafka from a different framework and language other than Java and Spring. Here’s in this example i just specifically giving the example of sending and replying kafka messages using Spring Boot. The Project Overview The client-project will send a string message to the server-project over kafka, then the server-project wil
Read more

ERROR 1348 Column Password Is Not Updatable When Updating MySQL Root Password

By -
Image
MySQL and MariaDB password As already mentioned in the title of this blog, this error message sometimes occurred when we are trying to update our root password database, either MySQL or MariaDB, The error says: ERROR 1348 (HY000): Column 'Password' is not updatable This is because if we use this SQL command to update our database root password, and it turn out to be Restricted by MySQL to use update on mysql database. UPDATE mysql.user SET Password=PASSWORD('1234') WHERE User='root'; The solutions for MySQL Error1348 So you can not update a user password using Update commands, to update the password, use ALTER commands instead. like the following. ALTER USER 'root'@'localhost' IDENTIFIED BY '1234'; And then you need to do the flush privileges to commit previous queries (that alter command above) into the system, simply do like this. flush privileges; So now you have your root with password 1234. Although it is recommended to
Read more

How To Create Spring Boot Project Using Netbeans

By -
Image
Spring Boot is program that you can use to develops Spring Framework easier, without the need of using so many hard to handle configuration. This tutorial, we are gonna create Spring Boot Application by using Netbeans IDE. You can of course create Spring Boot Appilcation on Netbeans, but it will be easier if just using plugin for Spring development in Netbeans, 1. Open Netbeans, click tools menu and click plugin. After plugin window opened, click tab Available Plugins and find the "Nb SpringBoot" plugin then install and activate the plugin. 2. After restarting the Netbeans IDE. Now you gonna be able to create a new project with Spring boot application. File > New Project > Under categories, choose Maven > then choose  Spring Boot Basic. Now you can start the development of spring boot application using your Netbeans IDE. Open pom.xml to add spring boot dependencies you want.
Read more

How To Connect SSH Using PEM Certificate On Windows

By -
Image
Usually you will need a pair of SSH username and password to connect to a remote server. But there's also another way to connect to the server via SSH but without password, just using a dot pem (example.pem) file. Here's how we usually do on Windows 1. Let’s SSH login using pem to confirm if it’s throwing error or the pem is fine, if error then we need to move to step 2 to fix it: ssh adam@10.0.0.123  -i server-cert.pem And here's the usual error that will throw if your pem file is not ready for SSH login. You need to follow the next step to make your pem certificate ready to login to your server via SSH. 2. Open Windows Explorer, then right-click on the pem file, then select "Properties". Navigate to the "Security" tab, and click "Advanced" 3. Disable inheritance 4. Delete all permissions, so it doesn’t have any permission 5. Now add you as the only only owner. By click button Add and then Select a principal, then Advanced. 6. On the showing p
Read more

Upload and Download Rest API using Spring Boot Reactive WebFlux

By -
Image
Reactive Spring Logo This is maybe a very simple task, but for those of you that may not be familiar with WebFlux found it difficult to create a Rest API that handles upload and download in Reactive spring application. Create the skeleton Spring project Create a new project, go to start.spring.io , on the dependencies section, add Spring Reactive Web. Generate and extract it into your desired project location. Instead of using spring-boot-starter-web, use spring-boot-starter-webflux instead in your pom.xml. Here’s the pom.xml for our Reactive Rest API project. Create our Rest controller Now that you have set to use WebFlux instead of the usual web, we have to change a little bit in the Controller class that handles the API endpoints. It doesn’t change much but you have to wrap the Response in Mono and the request parameter to FilePart. So our upload and download file controller will look like this. Create utility to handle file upload We need to structure our project better, so we cr
Read more