Spring Kafka - How to use ReplyingKafkaTemplate send and reply synchronously

-
Microservice Spring with Apache Kafka

Today, I want to share how to use Kafka. It's quite interesting once you get the hang of it, and I hope my explanation will make it a bit easier for you.

So, Kafka is primarily known as an asynchronous messaging queuing system. This means it allows different parts of your application to communicate with each other without waiting for immediate responses. However, there are times when you might need a more immediate reply from another service in your project. Luckily, Kafka can handle this too.

Here’s how it works: The client side sends a message to the server side via Kafka. This message includes a special identifier in the Kafka header request, known as KafkaHeaders.CORRELATION_ID, along with a topic where the server should respond. The server, after processing the request, sends a message back to the same topic with the same correlation ID. This way, the client knows that the message is a response to its initial request and can act accordingly.

Although Kafka can be integrated with various frameworks and languages, I’ll focus on how to send and receive Kafka messages using Spring Boot, which is a popular choice among developers. Spring Boot simplifies many aspects of working with Kafka and makes the process more straightforward.

The Project Overview

The client-project will send a string message to the server-project over kafka, then the server-project will reverse the string and return it back to the client.

First let’s start with our pom.xml, for both services we named spring-kafka-client and spring-kafka-server. The dependencies required are as follows.


And also the spring configuration files are the same for both services. Here’s the application.yml that you must create.


1. Client Service


Spring projects the client side, which produces a message and waits for the result.

This is an entry application containing the main method, we annotated them with @EnableKafka.

Main application for spring-kafka-client


Kafka configuration

- We registered bean ReplyingKafkaTemplate into our application, so this kafka template will be available in our application.
- We also register ConcurrentMessageListenerContainer, in this configuration we put REPLY_TOPICS and CONSUMER_GROUPS that are in our Yaml configuration.

Service that produce message


This service will be reusable throughout your client application, it will send synchronous requests and expect results from the producer. Basically we send messages into the SEND_TOPICS topic, with specific payload and headers, then it will be detected by the consumer somewhere on our server project.

2. Server Service


Spring project that will do the process of a given input. It can be some specific business logic, database query, etc.

Main application for spring-kafka-server.


Consumer and producer
- It listens to only a specific topic Which are into topic “${myproject.send-topics}” and by consumer “${myproject.consumer-group}”, those are variables that we set early in properties.
- It then @SendTo the reply topic, Kafka will automatically set the Header and Reply Topic that is sent by the client.
- Then inside the method it’s just a simple stringBuilder reverse string method. You can put your real business logic here.

Test the Application


Because it’s not a rest API web based spring application, we can just do an integration test of our send and reply kafka project, using Junit tests.

Write and run the test.


The test should pass if the server responds with a reversed string, from the message that client is sending. It’s just a simple logic for a test.

You can create a rest API if you want, all you have to do is just autowiring the KafkaService and use the kafkaRequestReply method.

We only do the test in our client project, because we just want to know the replying result. We don't necessarily have to write the test in the server project. But you can write a test for your server(producer) project yourself.

The full project can be clone from this Github repository Spring Kafka Request and wait Reply.

You can find more on the Spring kafka documentation here basically about anything you want to know about Kafka in Spring, here's the spring documentation https://docs.spring.io/spring-kafka/reference/html/#replying-template.


Popular posts from this blog

ERROR 1348 Column Password Is Not Updatable When Updating MySQL Root Password

-
Image
MySQL and MariaDB password As already mentioned in the title of this blog, this error message sometimes occurred when we are trying to update our root password database, either MySQL or MariaDB, The error says: ERROR 1348 (HY000): Column 'Password' is not updatable This is because if we use this SQL command to update our database root password, and it turn out to be Restricted by MySQL to use update on mysql database. UPDATE mysql.user SET Password=PASSWORD('1234') WHERE User='root'; The solutions for MySQL Error1348 So you can not update a user password using Update commands, to update the password, use ALTER commands instead. like the following. ALTER USER 'root'@'localhost' IDENTIFIED BY '1234'; And then you need to do the flush privileges to commit previous queries (that alter command above) into the system, simply do like this. flush privileges; So now you have your root with password 1234. Although it is recommended to
Read more

How To Create Spring Boot Project Using Netbeans

-
Image
Spring Boot is a handy tool that simplifies the development of Spring Framework applications by reducing the need for cumbersome configuration. In this tutorial, I'll guide you through creating a Spring Boot application using the Netbeans IDE. While you can certainly create a Spring Boot application in Netbeans manually, it's much easier with a plugin designed for Spring development. Here's a step-by-step guide to get you started: 1. Install the Spring Boot Plugin: Open Netbeans and navigate to the Tools menu. Click on "Plugins." Once the Plugins window opens, select the "Available Plugins" tab. Look for the "Nb SpringBoot" plugin, then install and activate it. 2. Create a New Spring Boot Project: After restarting Netbeans IDE, you can create a new project with Spring Boot. Go to File > New Project. Under Categories, select Maven, then choose "Spring Boot Basic." 3. Add Dependencies: With your new project created, open the po
Read more

How To Connect SSH Using PEM Certificate On Windows

-
Image
Usually you will need a pair of SSH username and password to connect to a remote server. But there's also another way to connect to the server via SSH but without password, just using a dot pem (example.pem) file. Here's how we usually do on Windows 1. Let’s SSH login using pem to confirm if it’s throwing error or the pem is fine, if error then we need to move to step 2 to fix it: ssh adam@10.0.0.123  -i server-cert.pem And here's the usual error that will throw if your pem file is not ready for SSH login. You need to follow the next step to make your pem certificate ready to login to your server via SSH. 2. Open Windows Explorer, then right-click on the pem file, then select "Properties". Navigate to the "Security" tab, and click "Advanced" 3. Disable inheritance 4. Delete all permissions, so it doesn’t have any permission 5. Now add you as the only only owner. By click button Add and then Select a principal, then Advanced. 6. On the showing p
Read more