Hazelcast ReplicatedMap如何实现非阻塞式get操作？

Problem Background #

We're using Hazelcast 4.2.1's ReplicatedMap with matching client/server versions. The map is tiny (fewer than 10 entries, each value <100 bytes), our client only performs read operations, and server-side updates are extremely infrequent. We expected ReplicatedMap.get() to be non-blocking, but after 6 hours of performance testing, Vert.x's blocked thread checker threw the following warning. Since the issue only occurs after long runtime, it's hard to reproduce.

Our core concern isn't just fixing the Vert.x thread block (we could move calls to a Worker Verticle), but eliminating query latency entirely—our business requires message processing to complete within 50ms, so even offloading to worker threads won't meet this requirement.

Blocked Thread Warning #

[vertx-blocked-thread-checker] WARN io.vertx.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 12777 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
	at jdk.internal.misc.Unsafe.park(Native Method) ~[?:?]
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:323) ~[?:?]
	at com.hazelcast.spi.impl.AbstractInvocationFuture.manageParking(AbstractInvocationFuture.java:693) ~[hazelcast-4.2.1.jar!/:4.2.1]
	at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:615) ~[hazelcast-4.2.1.jar!/:4.2.1]
	at com.hazelcast.client.impl.spi.ClientProxy.invokeOnPartition(ClientProxy.java:188) ~[hazelcast-4.2.1.jar!/:4.2.1]
	at com.hazelcast.client.impl.spi.ClientProxy.invoke(ClientProxy.java:182) ~[hazelcast-4.2.1.jar!/:4.2.1]
	at com.hazelcast.client.impl.proxy.ClientReplicatedMapProxy.get(ClientReplicatedMapProxy.java:214) ~[hazelcast-4.2.1.jar!/:4.2.1]
	at my.package.StateGetter.getState(StateGetter.java:44) ~[classes!/:1.5.189]

Root Cause #

Even though ReplicatedMap replicates data to all nodes/clients, the client-side proxy can still block on get() in edge cases:

• When the local client's copy of the entry has expired (based on TTL)
• When the client needs to sync the latest value from the server (e.g., after a server-side update that hasn't propagated yet)
• Under rare network/thread contention scenarios with the Hazelcast cluster

The synchronous get() method waits on a Future to complete, which blocks the Vert.x Event Loop.

Solutions to Achieve Non-Blocking, Low-Latency Reads #

1. Use the Async API Instead of Synchronous get()

Hazelcast provides an asynchronous getAsync() method that returns a CompletableFuture, which plays nicely with Vert.x's non-blocking model. This avoids blocking the Event Loop entirely, even if a remote sync is needed.

Example code integrating with Vert.x:

// Get the CompletableFuture from ReplicatedMap
CompletableFuture<Object> hazelcastFuture = replicatedMap.getAsync("your-key");

// Convert to Vert.x Future for idiomatic handling
io.vertx.core.Future.fromCompletionStage(hazelcastFuture)
    .onSuccess(value -> {
        // Process the value without blocking the event loop
        handleBusinessLogic(value);
    })
    .onFailure(error -> {
        // Handle any sync/network errors
        log.error("Failed to get value from ReplicatedMap", error);
    });

This approach requires minimal code changes and keeps your Event Loop free, making it easier to stay under the 50ms latency threshold.

2. Maintain a Local ConcurrentHashMap with EntryListeners

If you need absolute zero-latency reads (no chance of remote calls), you can mirror the ReplicatedMap in a local ConcurrentHashMap using Hazelcast's EntryListener. Since your map is tiny and updates are rare, this adds negligible overhead.

Implementation steps:

1. Initialize the local cache with all entries from the ReplicatedMap on client startup
2. Register listeners to sync local cache with server-side updates
3. Read directly from the local cache in your business logic

Example code:

// Local cache to mirror ReplicatedMap
ConcurrentHashMap<String, Object> localCache = new ConcurrentHashMap<>();
ReplicatedMap<String, Object> replicatedMap = hazelcastClient.getReplicatedMap("your-map-name");

// Initialize local cache with existing entries
replicatedMap.entrySet().forEach(entry -> localCache.put(entry.getKey(), entry.getValue()));

// Register listeners to sync updates
replicatedMap.addEntryListener(new EntryListener<String, Object>() {
    @Override
    public void entryAdded(EntryEvent<String, Object> event) {
        localCache.put(event.getKey(), event.getValue());
    }

    @Override
    public void entryUpdated(EntryEvent<String, Object> event) {
        localCache.put(event.getKey(), event.getValue());
    }

    @Override
    public void entryRemoved(EntryEvent<String, Object> event) {
        localCache.remove(event.getKey());
    }

    @Override
    public void entryEvicted(EntryEvent<String, Object> event) {
        localCache.remove(event.getKey());
    }
}, true); // true = include old value in events (optional)

// Business read operation - completely non-blocking
Object value = localCache.get("your-key");

This eliminates any remote call overhead entirely, guaranteeing sub-millisecond reads.

3. Tune ReplicatedMap Configuration

Adjust Hazelcast client settings to reduce the chance of needing remote syncs:

• setAsyncFillup(true): Fills the local replica asynchronously on client startup, avoiding initial blocking
• setTtlSeconds(): Increase the TTL to reduce entry expiration (since updates are rare, a TTL of 1 day or longer makes sense)

Example configuration:

ReplicatedMapConfig mapConfig = new ReplicatedMapConfig();
mapConfig.setName("your-map-name");
mapConfig.setAsyncFillup(true);
mapConfig.setTtlSeconds(86400); // 1 day TTL

ClientConfig clientConfig = new ClientConfig();
clientConfig.addReplicatedMapConfig(mapConfig);

HazelcastInstance hazelcastClient = HazelcastClient.newHazelcastClient(clientConfig);

Recommendation #

Start with the asynchronous getAsync() method—it's the simplest fix and will likely resolve your latency issues. If you still see occasional delays that exceed 50ms, move to the local cache approach, which is the most reliable way to guarantee zero-latency reads for your use case.

内容的提问来源于stack exchange，提问作者Niels Harremoes