Hazelcast Paging Predicate with JSON Data: A Complete Guide

1. Introduction to Hazelcast and JSON Support

Hazelcast is one of the in-memory data grid and computing platform, which is designed to process the large volume of data into quickly and reliably. Hazelcast distribute the data into multiple clusters of nodes, which enables to access the data significantly faster and with low latency, compared to disk storage solutions.

In recent days, JSON (JavaScript Object Notation) is standard data format for web applications, APIs, and backend services. The complex and dynamic data can be easily represent by using JSON, it support various data types and structure, which is very ideal for modern applications. Hazelcast introduce the HazelcastJsonValue class for querying and storage of JSON data.

By using of HazelcastJsonValue users can store the JSON data directly into Hazelcast map, which allows to store and retrieve JSON Object into distributed environment. This is beneficial while working on unstructured and semi-structured data, because it does not require to follow the complex schema model and flexible to query the JSON data.

Whenever you process the large volume the data, it is important that data management platform must be efficient, especially storing, retrieving and displaying JSON data. In such use cases PagingPredicate with HazelcastJsonValue is important feature that allows chunking the data into pages, instead of retrieving all the data at one go. It is not just improve the application performance, also enhance the user experience, especially the scenario when user display the data set by paginated API.

In Hazelcast Paging Predicate with JSON Data article, we will explore about PagingPredicate with HazelcastJsonValue efficiently paginate store and retrieve the JSON data in Hazelcast. This technique is very useful for the applications that work with large data set, where distributed data stores and require the flexibility of JSON and the performance at in-memory grid.

2. Understanding Paging Predicate

In Hazelcast, Predicate is a condition which is used to query the set. By using Hazelcast specific Predicates filter criteria, we can access and query the data from Hazelcast map and collections. For example, if you need to retrieve the based on specific attribute or field, and then you use the Hazelcast predicate, to retrieve only specific data which is required.

Hazelcast PagingPredicate is a special type of Predicate which allows us to retrieve the data into paginated format. Normally, if you query into the Hazelcast Map then it returns all the data at once, which is not helpful for large data sets and load on the memory. But in case of large data sets, we can use the Hazelcast PagingPredicate to retrieve the data one by one into small paginated chunks.

Whenever we need to display large data set into user interface, like search result, we mostly use the pagination. In that scenario, PagingPredicate provided the feature where results can be divided into the pages, which improve both the application performance and user experience.

In PagingPredicate, to retrieve the data set into paginated manner, we can define the page size and sorting criteria. It follows the PageResultSet concept which represents an specific page. When you need to retrieve the next page, you can increment the page index and fetch the next page data.

In Short, PagingPredicate is a important tool which allows us to process or retrieve the large data set into paginated manner. This technique especially helpful into distributed systems to achieve the high-performance in large data scenario.

Also explore about:

Java Object – Hazelcast Paging Predicates with Example: How to Work in Data Driven Application

3. How Paging Predicate Works with HazelcastJsonValue

By using HazelcastJsonValue, we can store large, dynamic, and unstructured data into Hazelcast Map. Hazelcast PagingPredicate with HazelcastJsonValue feature is very useful when you want to query and paginate the large data

We can specify two parameter in Paging Predicate:

• Page Size: It defines the number of records in a page, for example, 10 or 20 records per page.
• Start Point: The point where we start reading the data of next page.
• Sorting/Ordering Criteria: It allows the user to sort the JSON into ascending or descending order by specific identity criteria.

When we use PagingPredicate with HazelcastJsonValue then each query returns the paginated result, the result stored in the PageResultSet. If you want to traverse into another page, then just increment the page number and fetch the next page JSON record. In this mechanism, and display the large data set in a paginated manner.

4. Setting Up Hazelcast JSON Paging Predicate

1. Add Maven Dependency

To setup the Hazelcast Paging Predicate with JSON Data project add the Hazelcast dependencies into pom.xml

<dependency>
   <groupId>com.hazelcast</groupId>
   <artifactId>hazelcast</artifactId>
   <version>5.5.0</version> <!-- use the latest version -->
</dependency>

2. Sample POJO classes

In the example, we have created Student, Department, and Course classes, which we will be converting to JSON by using Gson utilities

@AllArgsConstructor
@Getter
@ToString
public class Course {
    private int courseId;
    private String courseName;
    
    //create setter and getter or use lombok lib
}

@Getter
@AllArgsConstructor
@ToString
public class Department {
    private int id;
    private String departmentName;

    //create setter and getter or use lombok lib
}

@Getter
@AllArgsConstructor
@ToString
public class Student {
    private int studentId;
    private String studentName;
    private int departmentId;
    private List<Course> courses;

    //create setter and getter or use lombok lib
}

3. Initialize the Hazelcast Instance and Maps

Create a Hazelcast Instance standalone embedded or enterprise as you need, in the demo I am using embedded instance.

// Initialize Hazelcast instance
 HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();

Create a sample data map studentMap, courseMap and departmentMap.

// Define maps
studentMap = hazelcastInstance.getMap("studentMap");
departmentMap = hazelcastInstance.getMap("departmentMap");
courseMap = hazelcastInstance.getMap("courseMap");

//create index for a column
studentMap.addIndex(IndexType.SORTED, "departmentId");

//create sample data for demo
createSampleData();

4. Create Sample Data

Create sample data and add into the maps

private static void createSampleData() {
    Course course1 = new Course(501, "Physics");
    Course course2 = new Course(502, "Chemistry");
    Course course3 = new Course(501, "Maths");
    courseMap.put(501, new HazelcastJsonValue(gson.toJson(course1)));
    courseMap.put(502, new HazelcastJsonValue(gson.toJson(course2)));
    courseMap.put(503, new HazelcastJsonValue(gson.toJson(course3)));

    Student student1 = new Student(101, "Sachin", 1001, List.of(course1, course3));
    Student student2 = new Student(102, "Rahul", 1002, List.of(course2));
    Student student3 = new Student(103, "Rachel", 1001, List.of(course3));
    Student student4 = new Student(104, "Villa", 1001, List.of(course1));
    Student student5 = new Student(105, "Nev am", 1002, List.of(course2));
    studentMap.put(101, new HazelcastJsonValue(gson.toJson(student1)));
    studentMap.put(103, new HazelcastJsonValue(gson.toJson(student3)));
    studentMap.put(102, new HazelcastJsonValue(gson.toJson(student2)));
    studentMap.put(105, new HazelcastJsonValue(gson.toJson(student5)));
    studentMap.put(104, new HazelcastJsonValue(gson.toJson(student4)));

    Department department1 = new Department(1001, "Physics Department");
    Department department2 = new Department(1002, "Chemistry Department");
    departmentMap.put(1001, new HazelcastJsonValue(gson.toJson(department1)));
    departmentMap.put(1002, new HazelcastJsonValue(gson.toJson(department2)));
}

In the demo, we have created studentMap, departmentMap and courseMap with the value object HazelcastJsonValue where Student has departmentId and List<Course>. Here we have used gson.toJson() to convert the data Object to JSON String

5. Implementing Paging Predicate with JSON Data

To implement Hazelcast Paging Predicate with JSON Data, create a Predicate object and apply it to the PagingPredicate.

//create sample predicates
PagingPredicate<Integer, HazelcastJsonValue> pagingPredicate = Predicates.pagingPredicate(3);
Collection<HazelcastJsonValue> searchedResult = studentMap.values(pagingPredicate);
System.out.println("searchedResult : "+searchedResult);

• PagingPredicate instance initialize with page size 3
• Apply the predicate into studentMap and fetch the value
• It will fetch the top 3 record from studentMap.

//traverse to the next page
pagingPredicate.nextPage();
searchedResult = studentMap.values(pagingPredicate);
System.out.println("Next page - searchedResult : "+searchedResult);

Here we have demonstrated the traversing to the next page and print the result.

//Search array attribute
Predicate<Object, Object> predicates = Predicates.equal("courses[any].courseId", "501");
PagingPredicate<Integer, HazelcastJsonValue> arraySearch = Predicates.pagingPredicate(predicates,3);
searchedResult = studentMap.values(arraySearch);
System.out.println("Array Predicate - searchedResult : "+searchedResult);

In this code, we have demonstrated to create the predicate with JsonArray and filter the data based on value inside the Array.

• Create the predicate courses[any].courseId where course is 501
• Apply the predicate into PagingPredicate with the page size 3
• Use the PagingPredicate to fetch the search result from studentMap.

6. Benefits of PagingPredicate in Hazelcast:

1. Improved Resource Management: It improve application performance and resource management
2. Better Network Efficiency: PagingPredicate can be used to fetch the smaller chunks or pages of data
3. Scalability: Parallel thread can utilize the nodes, as it is highly scalable.
4. User Experience: Application and data fetch would be easy to understand, enhancing the user experience. Data would be fetched smoothly with page by page.

7. Conclusion

In the UI where large data must be fetched, PagingPredicate allows users to set the page size and fetch the date into paginated manner. PagingPredicate also provide a feature where you can traverse into the pages and get the data.

The code used on the demo, complete code is available over the GitHub.