Benchmark with JMH (Java Microbenchmark Harness)

  JMH stands for Java Micro-benchmark Harness and is a toolkit used for implementing benchmarks for applications running on JVM.

For those who are new to the term “benchmarking”, it is a process of  identifying indicators for any software application and using these indicators to assess and maintain quality. One of the key factors of ensuring a good quality of a software product is application performance. We can consider the execution time of any software program as one of the main performance indicators. Hence, it becomes important to measure and track this indicator effectively to certify the application’s quality.

Almost all software applications have small units of code which are performance-critical. For example, part of an application which involves a lot of computation and data processing is performance-critical, and measuring the execution time of these small units   is called micro-benchmarking.   These operations are executed several times in a certain environment with a standard CPU and memory configuration to produce consistent results. The execution time of each run  is  recorded and  consolidated to obtain a mean or average time, known as micro-benchmarks . 

Using JMH to create benchmarks is fairly simple. A lot of documentation and resources are available over the Internet. One can follow below steps to implement a simple benchmark project.

Developing Benchmarks Using JMH –

  • A benchmark project can be generated using a simple maven command:

mvn archetype:generate \
          -DinteractiveMode=false \
          -DarchetypeGroupId=org.openjdk.jmh \
          -DarchetypeArtifactId=jmh-java-benchmark-archetype \
          -DgroupId=org.sample \
          -DartifactId=test \
          -Dversion=1.0

And adding a couple of maven dependencies

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.21</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.21</version>
</dependency>
  • Create the benchmark Java class and test methods annotated with JMH annotations. Invoke the API/Method/Application for which you want to generate the benchmark results.

For example –

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
 
public class MySimpleBenchmark {
 
   private MathHelper mathHelper;
	
	@Benchmark
	@BenchmarkMode(org.openjdk.jmh.annotations.Mode.AverageTime)
	public void benchmarkAddMethod()
	{
		mathHelper.add(10, 15);
	}
 
  • Build this project using below maven command –
mvn clean install
  • Execute the generated benchmark.jar –
java -jar target/benchmarks.jar
  • Results are published in below format –

JMH Benchmark to Sort 25 integers

Benchmark                          Mode  Cnt         Score        Error  Units

 JMHSortBenchmark.arraysSort       thrpt   20  28779058.746   446412.014  ops/s 
 JMHSortBenchmark.collectionsSort  thrpt   20  26070145.869   206077.160  ops/s 

JMH Benchmark to Sort 1000 integers

Benchmark                          Mode  Cnt        Score       Error  Units

 JMHSortBenchmark.arraysSort       thrpt   20  3795959.757   43679.226  ops/s 
 JMHSortBenchmark.collectionsSort  thrpt   20   853014.250    6256.061  ops/s 

You can refer to a sample benchmark project here.

Here and here are some references on GitHub to the benchmarks developed by Spring.

Nakul Shukla

I am a programmer by profession. I have a penchant for learning and I believe in knowledge sharing.  I like to write content based on my professional experience, which may help aspiring techies. This blog is simply a step towards it.

Latest posts by Nakul Shukla (see all)

I am a programmer by profession. I have a penchant for learning and I believe in knowledge sharing.  I like to write content based on my professional experience, which may help aspiring techies. This blog is simply a step towards it.

4 thoughts on “Benchmark with JMH (Java Microbenchmark Harness)”

  1. I appreciate your efforts in taking time to share what you have learn from your experience.
    I understood the gist despite of java background cause you’ve written it concisely in simple words. and I have basic programming idea.

    Thank you.
    Keep writing and keep sharing.

Leave a Reply

Your email address will not be published. Required fields are marked *