Test Spring Boot apps using EmbeddedKafka and Awaitility

This is the second article in “Spring Cloud Stream and Kafka” series. You can check the previous article here. This post talks about different ways in which we can test Spring Boot applications, using EmbeddedKafka and Awaitility.

While testing any synchronous application, it is all about “call and wait”. we invoke a particular API or endpoint and wait for the response. The test blocks the main execution thread until the API returns the response. Once the processing completes, we get the response and can compare the result with the expected output.

Asynchronous applications are tested differently as compared to the synchronous or blocking applications i.e. we need not block the main execution thread. In simple words, it will not wait for the response from the API and we manually need to program the test to hold the execution at a certain point and wait for the results from all the non-blocking operations. At this stage, we can write the assertions.

It is hard to manage different threads and concurrency issues and write a concise, readable unit test.

There are a few ways in which we can write tests for a Spring boot – Spring Cloud Stream based micro-services to connect with Kafka.

Let’s consider a simple use case for this purpose –

There is a producer bean which will send messages to Kafka topic.

A consumer bean which will listen to a Kafka topic and receives messages.

A Kafka broker with a topic created. For this test, we will use Embedded Kafka server, by spring-kafka-test.

public static EmbeddedKafkaRule embeddedKafka = new EmbeddedKafkaRule(1, true, TOPIC1);

EmbeddedKafkaRule – Spring-kafka-test provides an embedded Kafka broker. We can use a Junit “@ClassRule” annotation to create this Kafka broker. This rule starts the Kafka and Zookeeper servers on a random port before the tests execute and shuts them down after the tests complete. The embedded Kafka broker eliminates the need to have a real Kafka and zookeeper instance running while running the test.

Coming back to the tests, I have implemented this test in two ways, using Awaitility and using a countdown latch.

  • Test using Awaitility: This is a DSL library which provides very nice features to help writing nice JUnit tests for an asynchronous java application. You can check their official github page here. Below is an implementation of the test using Awaitility.
  • Test using CountDownLatch : As per Java documentation, CountDownLatch is an aid that allows one or more threads to wait until a set of operations being performed in other threads completes. To write this test using CountDownLatch, we initialize the latch first with a counter. The value of this counter depends on the number of tasks our test needs to wait for. Here, we initialize this counter with count 1. Once the producer has sent the message, the latch awaits for the count to reach 0. The consumer has the responsibility of decrementing the count. Hence, when the consumer is done with its part, the main thread resumes and performs the assertion.

Below is an implementation of the test using CountDownLatch.

You can find the complete source code here.


An expert software professional, Music junkie, Reader.. blogger ? can barely write!

Leave a Reply

Your email address will not be published. Required fields are marked *