Spring Boot JPA Performance Tuning: Speeding Up Hibernate Batch Processing Time

Hibernate is one of the most popular ORM (Object Relational Mapping) tools for Java due to its huge control and convenience. Some advantages of Hibernate in Java are –

• Lightweight and Open-source — Being lightweight and open-source makes it accessible and efficient.
• Increased Performance — Using cache memory helps in fast performance.
• Database Independence — Being database-independent gives it the ability to work with different databases.
• Auto DDL Operations — automatic table creation saves us from manually creating tables.
• Minimizes database access with smart fetching strategies.
• It provides simple querying of data.

For these along with a lot other significances, it’s very important to give Spring Boot JPA performance tuning a special care. Speeding up hibernate batch processing time is a crucial factor to consider in this process.

Today we will discuss such a similar hibernate performance issue that we faced during one of our projects and how we resolved it.

Speeding Up Hibernate Batch Processing Time — Code with Example

public boolean deleteCompany (long id) {
	List<Employee> employees = employeeService.getAllEmployees();  //returns 500+ employees.
    	for (Employee employee : employees) {
      		//lots of business logic.
      		//more than 5 database calls
      		//calculations
      		//logging
    }
}

We have got 500 employee information in a list. We had to check in several classes for the business logic. Business logic needed a lot of if-else and database call itself. After all the calculation output is saved into another class named “Present” and saved into the database.

This should not take a long time if the database is in the localhost right? Surprisingly, execution time for this code is more than an hour. To be exact, It took 1hour 34 minutes in our development machine to execute. Database on the remote server took more than 2 hours. Which is totally not normal.

We started to investigate the problem. We added several logs into the code for better understanding of the problem. We found that it is taking about 20/30 seconds at first for an employee. But, gradually the execution time was increasing. After 400 employees, it took about 3–4 minutes to create a Present object and save it to the database.

After many trial errors and google search we found that, That is how hibernate works. Each time you save an entity it is added to the first level cache, this cache is bound to the transaction. Now each time you call save it does a dirty check on all the entities it has in the first level cache. The first 100 probably aren’t noticeable but after that it adds up. What we need to do is after each X entities saved do a flush and clear the entitymanager/ session (depending on if you are using JPA or plain hibernate).

public boolean createPresent (long id) {
	List<Employee> employees = employeeService.getAllEmployees();  //returns 500+ employees.
    	for (Employee employee : employees) {
      		//lots of business logic.
      		//more than 5 database calls
      		//calculations
      		//logging
          entityManager.clear();
    }
}

This reduced the processing time to nearly 10 minutes!

That is impressive right?

But, still this is a lot of time to process. 10 minutes is a long time for this task. On localhost this should take a very less time.

We wanted to investigate and reduce the processing time more. We found that approximately 510 database save calls were made during the function. This is the reason for slowing down. We found a solution to save the Objects in the database in a batch. How to do that in spring boot? Let’s take a peek in the application.properties file. We have added these following lines into the file.

spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.order_inserts=true

From the above code you can already guess it right? What these lines are intended to do is to save all the objects in a batch of 50 objects. It will save 50 objects and save and commit into the database and then it will start to save 50 more objects until all the objects are persisted in the database.

Then, we saved all the objects in a list. And used the following line to save the objects.

dao.saveAll(newListOfObjects)

Voila! Execution time of the code was less than a minute. It felt like magic. Truly! Can you imagine? 1 hours 34 minutes to just a minute. It took mandatory 10 database calls and all other 500 insert queries were squeezed into 500/50 = 10 database calls. So, total 510 database calls converted and squeezed into 20 database calls. And that takes only 1 minute or less.

That’s all for today. Stay safe. Feel free to knock us in the inbox if you have any issues. Cheers!

The original post can be found here.