Overview
Hazelcast, a leading in-memory data grid, offers various serialization mechanisms to boost performance and efficiency in distributed systems. Among these, Hazelcast Compact Serialization is a standout feature designed to minimize memory footprint and reduce network bandwidth consumption. As systems scale, efficient serialization becomes critical, and Hazelcast’s compact format offers a modern and optimized solution.
In this article, we explore Hazelcast’s compact serialization, detailing how it works, its advantages, and best practices for implementation.
What is Hazelcast Compact Serialization?
Serialization in Hazelcast is the process of converting an object into a byte stream for transmission or storage. Hazelcast Compact Serialization is a novel mechanism that emphasizes compactness and performance, making it more efficient than traditional Java serialization or even Hazelcast’s default serialization methods like IdentifiedDataSerializable and Portable.
Compact serialization differs in its ability to:
- Reduce Payload Size: By minimizing the amount of metadata stored along with data.
- Enhance Performance: Lower overhead during serialization and deserialization operations.
- Support Schema Evolution: Manage backward and forward compatibility when object structures change over time.
Also know about Serialization and Deserialization process
Understanding Hazelcast Serialization and Deserialization: Why It Matters
Key Features of Hazelcast Compact Serialization
1. Schema-Free Serialization
Unlike Portable Serialization, which requires a predefined schema, compact serialization eliminates this requirement, allowing objects to be serialized without registering their schema upfront. This makes the process simpler and more dynamic, especially in systems where frequent changes to object structures are expected.
2. Efficient Handling of Primitive Types
Compact serialization natively supports primitive types like int, long, boolean, and float, ensuring that these types are serialized with minimal overhead. This leads to faster encoding and decoding processes, making it ideal for high-performance applications.
3. Backward and Forward Compatibility
With Compact Serialization, Hazelcast introduces schema evolution. This feature allows a system to handle changes in object structures seamlessly. If new fields are added or some are removed from a serialized object, the system can still read and process older versions, ensuring robustness during system upgrades.
4. Space Optimization
Compact serialization dramatically reduces the size of the serialized object. This space-saving approach is particularly beneficial for large-scale, distributed systems where bandwidth and storage efficiency are paramount.
Benefits of Using Hazelcast Compact Serialization
1. Improved Performance in Distributed Systems
Serialization is a crucial factor in determining the performance of distributed systems. Traditional serialization mechanisms in Java are notorious for their inefficiency and overhead. Compact serialization, however, provides a leaner and faster approach, translating directly to reduced latency in data transfer between nodes in a Hazelcast cluster.
2. Lower Network Bandwidth
Hazelcast Compact serialization significantly cuts down on the amount of data transferred between Hazelcast nodes or between clients and servers. By eliminating unnecessary metadata and optimizing the encoding of objects, compact serialization reduces the network bandwidth required for communication, making the system more scalable.
3. Simplified Object Evolution
In many distributed systems, objects evolve over time as new features are added or old ones are removed. Compact serialization’s support for schema evolution makes it easier to manage these changes without breaking compatibility or requiring heavy rewrites of serialization logic. This ensures smoother deployments and fewer runtime errors when upgrading or modifying the application.
4. Compatibility with Multiple Platforms
Hazelcast compact serialization is designed with cross-platform compatibility in mind. It allows Java objects to be serialized and then deserialized by clients written in other languages such as C++ or .NET, making it ideal for polyglot environments.
How to Implement Hazelcast Compact Serialization
Implementing compact serialization in Hazelcast is a straightforward process. Below are the steps to enable it and leverage its benefits in your application:
1. Create Compact Serializer
To enable Hazelcast compact serialization, we need to define a compact serializer for the class. Here is an example for a class Student:
package com.javatecharc.demo.model;
import lombok.Getter;
import lombok.Setter;
@Getter
@Setter
public class Student {
private int id;
private String name;
private String department;
public Student(int id, String name, String department) {
this.id = id;
this.name = name;
this.department = department;
}
//Generate Getter and Setter or use Lombok @Getter, @Setter
}Next, we define a compact serializer StudentCompactSerializer for this class:
package com.javatecharc.demo.compact.serializer;
import com.hazelcast.nio.serialization.compact.CompactReader;
import com.hazelcast.nio.serialization.compact.CompactSerializer;
import com.hazelcast.nio.serialization.compact.CompactWriter;
import com.javatecharc.demo.model.Student;
public class StudentCompactSerializer implements CompactSerializer<Student> {
@Override
public Student read(CompactReader compactReader) {
int id = (int) compactReader.readInt64("id");
String name = compactReader.readString("name");
String department = compactReader.readString("department");
return new Student(id, name, department);
}
@Override
public void write(CompactWriter writer, Student student) {
writer.writeInt64("id", student.getId());
writer.writeString("name", student.getName());
writer.writeString("department", student.getDepartment());
}
@Override
public String getTypeName() {
return "Student";
}
@Override
public Class<Student> getCompactClass() {
return Student.class;
}
}2. Register the Serializer with Hazelcast
After defining the serializer, it needs to be registered with the Hazelcast configuration by Config programmatic or hazelcast.xml config. Here’s how to configure it:
Programmatic Configuration:
Register the class into Config object by using addClass(Student.class) or addSerializer(new StudentCompactSerializer())
Config config = new Config();
config.getSerializationConfig()
.getCompactSerializationConfig()
.addSerializer(new StudentCompactSerializer());
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
Declarative Configuration:
Declare the StudentCompactSerializer class in to hazelcast.xml configuration.
<hazelcast>
<serialization>
<compact-serialization>
<classes>
<class>
com.javatecharc.demo.compact.serializer.StudentCompactSerializer
</class>
</classes>
</compact-serialization>
</serialization>
</hazelcast>3. Using Compact Serialization in Maps
Now, when you store objects in Hazelcast’s distributed Map, the compact serialization will be applied automatically:
package com.javatecharc.demo.compact.serializer;
import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.map.IMap;
import com.javatecharc.demo.model.Student;
public class HazelcastCompactSerializerExample {
public static void main(String[] args) {
Config config = new Config();
config.getSerializationConfig().getCompactSerializationConfig()
.addSerializer(new StudentCompactSerializer());
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
IMap<Integer, Student> studentMap = hazelcastInstance.getMap("students");
studentMap.put(101, new Student(101, "Sachin", "Java Tech ARC"));
Student student = studentMap.get(101);
System.out.println(student.getName());
hazelcastInstance.shutdown();
}
}In this example, when an Student object is stored in the map, it is serialized using the compact format, reducing the memory footprint and enhancing performance.
Best Practices for Compact Serialization
1. Use Compact Serialization for Large Datasets
If your Hazelcast cluster handles large datasets or frequently communicates between nodes, compact serialization can drastically reduce the overhead of transferring data. This can improve the system’s responsiveness, especially under high-load conditions.
2. Test Schema Evolution Carefully
While compact serialization supports schema evolution, it is important to test thoroughly when adding or removing fields from serialized objects. In particular, ensure that changes are backward-compatible and that the application can gracefully handle older object versions.
3. Optimize Primitive Data Handling
Where possible, prefer using primitive data types in your serialized objects. Compact serialization handles primitive types more efficiently, which leads to faster serialization/deserialization and smaller serialized objects.
Conclusion
Hazelcast’s Compact Serialization is a powerful tool for optimizing data transfer and storage in distributed systems. By offering a schema-free, compact, and efficient approach to serialization, it greatly enhances the performance of Hazelcast applications. Its support for schema evolution and compatibility with multiple platforms further solidifies its position as an essential feature for modern distributed environments.
The sample code available over the github.
Happy Coding! 😊
