AKS部署Apache Ignite集群时IP Finder获取注册地址失败

部署配置 #

按官方文档在AKS部署Apache Ignite集群，使用以下资源配置：

service.yaml #

apiVersion: v1
kind: Service
metadata:
  # The name must be equal to KubernetesConnectionConfiguration.serviceName
  name: ignite-service
  # The name must be equal to KubernetesConnectionConfiguration.namespace
  namespace: ignite
  labels:
    app: ignite
spec:
  type: LoadBalancer
  ports:
    - name: rest
      port: 8080
      targetPort: 8080
    - name: thinclients
      port: 10800
      targetPort: 10800
  # Optional - remove 'sessionAffinity' property if the cluster
  # and applications are deployed within Kubernetes
  #  sessionAffinity: ClientIP
  selector:
    # Must be equal to the label set for pods.
    app: ignite
status:
  loadBalancer: {}

创建服务账号 #

kubectl create sa ignite -n ignite

cluster-role.yaml #

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ignite
  namespace: ignite
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ignite
roleRef:
  kind: ClusterRole
  name: ignite
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: ignite
  namespace: ignite

deployment.yaml #

# An example of a Kubernetes configuration for pod deployment.
apiVersion: apps/v1
kind: Deployment
metadata:
  # Cluster name.
  name: ignite-cluster
  namespace: ignite
spec:
  # The initial number of pods to be started by Kubernetes.
  replicas: 2
  selector:
    matchLabels:
      app: ignite
  template:
    metadata:
      labels:
        app: ignite
    spec:
      serviceAccountName: ignite
      terminationGracePeriodSeconds: 100000
      containers:
        # Custom pod name.
      - name: ignite-node
        image: apacheignite/ignite:2.14.0
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes,ignite-rest-http
        - name: CONFIG_URI
          value: file:///ignite/config/node-configuration.xml
        ports:
        # Ports to open.
        - containerPort: 47100 # communication SPI port
        - containerPort: 47500 # discovery SPI port
        - containerPort: 49112 # dafault JMX port
        - containerPort: 10800 # thin clients/JDBC driver port
        - containerPort: 8080 # REST API
        volumeMounts:
        - mountPath: /ignite/config
          name: config-vol
      volumes:
      - name: config-vol
        configMap:
          name: ignite-config

node-configuration.xml #

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean class="org.apache.ignite.configuration.IgniteConfiguration">

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                        <property name="namespace" value="ignite"/>
                        <property name="serviceName" value="ignite-service"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

创建ConfigMap #

kubectl create configmap ignite-config -n ignite --from-file=node-configuration.xml

问题现象 #

Pod已正常运行，但日志中持续出现SSL握手失败错误，导致Ignite节点无法发现彼此：

[07:22:28,352][SEVERE][main][TcpDiscoverySpi] Failed to get registered addresses from IP finder (retrying every 2000ms; change 'reconnectDelay' to configure the frequency of retries) [maxTimeout=0]
class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite pods IP addresses.
        at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:81)
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:2057)
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1992)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1283)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1120)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:472)
        at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2212)
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:278)
        at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:1089)
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1766)
        at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1147)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1757)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1679)
        at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1121)
        at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1015)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:921)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:840)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:710)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:679)
        at org.apache.ignite.Ignition.start(Ignition.java:353)
        at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
Caused by: class org.apache.ignite.IgniteException: Failed to retrieve Ignite pods IP addresses.
        at org.apache.ignite.internal.kubernetes.connection.KubernetesServiceAddressResolver.getServiceAddresses(KubernetesServiceAddressResolver.java:123)
        at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:76)
        ... 20 more
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
        at sun.security.ssl.Alert.createSSLException(Alert.java:131)
        at sun.security.ssl.Alert.createSSLException(Alert.java:117)
        at sun.security.ssl.TransportContext.fatal(TransportContext.java:311)
        at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
        at sun.security.ssl.TransportContext.dispatch(TransportContext.java:185)
        at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152)
        at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397)
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1572)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1500)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
        at org.apache.ignite.internal.kubernetes.connection.KubernetesServiceAddressResolver.getServiceAddresses(KubernetesServiceAddressResolver.java:111)
        ... 21 more

解决方案建议 #

1. 指定JVM TLS协议版本 #

Ignite容器中的JDK可能默认禁用了AKS API Server使用的TLS版本，在deployment.yaml的容器环境变量中添加JVM参数，强制使用兼容的TLS协议：

env:
- name: OPTION_LIBS
  value: ignite-kubernetes,ignite-rest-http
- name: CONFIG_URI
  value: file:///ignite/config/node-configuration.xml
# 添加以下环境变量
- name: IGNITE_JVM_OPTS
  value: "-Dhttps.protocols=TLSv1.2,TLSv1.3"

2. 升级Ignite版本 #

当前使用的apacheignite/ignite:2.14.0版本较旧，可能存在与新版AKS的Kubernetes API兼容性问题，建议升级到2.15.x或更高稳定版本，修改deployment.yaml中的镜像：

image: apacheignite/ignite:2.15.0

3. 验证ServiceAccount权限与证书 #

虽然配置了全权限ClusterRole，但需确认ServiceAccount的token能正常访问K8s API：

• 进入Ignite容器，执行curl -v https://kubernetes.default.svc/api/v1/namespaces/ignite/services/ignite-service测试API连通性
• 如果证书验证失败，可将AKS的根CA证书挂载到容器的JRE信任存储，或使用自定义镜像预先导入CA证书

4. 调整Service类型（可选） #

节点发现无需使用LoadBalancer类型的Service，将service.yaml中的type改为ClusterIP，减少外部网络依赖：

spec:
  type: ClusterIP

内容的提问来源于stack exchange，提问作者Shivani K