Kubernetes上Infinispan兼作Vert.x集群管理器与缓存管理器时缓存不共享问题

项目目标是借助Infinispan搭建Vert.x应用（verticles）集群，让verticles共享由Infinispan管理的单个复制缓存集合。

已确认Kubernetes上部署后，Infinispan集群管理功能正常，Kubernetes部署信息如下：

root@vertx-ollama-control-plane:/# kubectl get all
NAME                                       READY   STATUS    RESTARTS        AGE
pod/backend-deployment-9dd7d994c-d6w5m     1/1     Running   1 (7m34s ago)   8m13s
pod/backend-deployment-9dd7d994c-d7ttq     1/1     Running   0               8m13s
pod/backend-deployment-9dd7d994c-zgjhx     1/1     Running   1 (7m34s ago)   8m13s
pod/frontend-deployment-6557dd4466-bf2gj   1/1     Running   0               8m13s
pod/frontend-deployment-6557dd4466-f6fqz   1/1     Running   0               8m13s
pod/ollama-858d4f8c8d-fggwb                1/1     Running   0               8m13s

NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/clustered-app   ClusterIP      None            <none>        7800/TCP       8m13s
service/frontend        LoadBalancer   10.96.229.141   <pending>     80:32375/TCP   8m13s
service/kubernetes      ClusterIP      10.96.0.1       <none>        443/TCP        3h26m
service/ollama          ClusterIP      10.96.54.50     <none>        11434/TCP      8m13s

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/backend-deployment    3/3     3            3           8m13s
deployment.apps/frontend-deployment   2/2     2            2           8m13s
deployment.apps/ollama                1/1     1            1           8m13s

NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/backend-deployment-9dd7d994c     3         3         3       8m13s
replicaset.apps/frontend-deployment-6557dd4466   2         2         2       8m13s
replicaset.apps/ollama-858d4f8c8d                1         1         1       8m13s

通过无头服务暴露的verticles端点信息：

root@vertx-ollama-control-plane:/# kubectl get endpoints
NAME            ENDPOINTS                                                        AGE
clustered-app   10.244.0.80:7800,10.244.0.81:7800,10.244.0.82:7800 + 2 more...   9m16s
frontend        10.244.0.83:8080,10.244.0.84:8080                                9m16s
kubernetes      172.18.0.4:6443                                                  3h27m
ollama          10.244.0.79:11434                                                9m16s

当前缓存与集群配置代码 #

用于集群共享缓存的配置与代码如下：

// Configure default cache manager
DefaultCacheManager cacheManager = new DefaultCacheManager(
        new GlobalConfigurationBuilder()
                .transport()
                .defaultTransport()
                .build()
);
clusterManager = new InfinispanClusterManager(cacheManager);

// Configure the cache for embeddings
Configuration cacheConfig = new ConfigurationBuilder().clustering()
        .cacheMode(CacheMode.REPL_ASYNC)
        .encoding()
        .mediaType(MediaType.APPLICATION_OBJECT_TYPE)
        .build();

... 

if (cacheManager.cacheExists("embeddings")) {
    logger.info(String.format("Cache %s exists with the hashcode of %d on %s node.",
            "embeddings",
            cacheManager.getCache("embeddings").hashCode(),
            cacheManager.getNodeAddress())
    );
    collection = cacheManager.getCache("embeddings");
} else {
    logger.info(String.format("Cache %s does not exist, a new cache is created on %s node.",
            "embeddings",
            cacheManager.getNodeAddress()
    ));
    collection = cacheManager.createCache("embeddings", cacheConfig);
}

// ...

public static void main(String[] args) {
    Vertx.clusteredVertx(new VertxOptions().setClusterManager(clusterManager))
            .compose(v -> v.deployVerticle(new Main()))
            .onFailure(Throwable::printStackTrace);
}

部署Pod时已通过环境变量注入Jgroups Kubernetes配置文件：-Dvertx.jgroups.config=default-configs/default-jgroups-kubernetes.xml。

问题现象 #

多次向Infinispan缓存存储嵌入信息时，每个Pod都会创建自己的"embeddings"缓存，请求与日志示例如下：

请求1：

http POST :8080/embed prompt="Llamas are members of the camelid family meaning they're pretty closely related to vicuñas and camels"
HTTP/1.1 200 OK
content-length: 105

Embedding entry stored with key: 451439790
From: backend-deployment-9dd7d994c-zgjhx (Collection Size: 1)

对应Pod A日志：

Dec 14, 2024 4:10:26 AM cynicdog.io.api.OllamaAPI
INFO: Cache embeddings does not exist, a new cache is created on backend-deployment-9dd7d994c-d6w5m-37175 node

请求2：

http POST :8080/embed prompt="Llamas are members of the camelid family meaning they're pretty closely related to vicuñas and camels"
HTTP/1.1 200 OK
content-length: 105

Embedding entry stored with key: 451439790
From: backend-deployment-9dd7d994c-d6w5m (Collection Size: 1)

对应Pod B日志：

INFO: Model mxbai-embed-large:latest pulled.
Dec 14, 2024 4:10:26 AM cynicdog.io.api.OllamaAPI
INFO: Cache embeddings does not exist, a new cache is created on backend-deployment-9dd7d994c-d6w5m-37175 node.

请问遗漏了哪些配置或步骤？

内容的提问来源于stack exchange，提问作者cynicdog