Kubernetes多实例部署JBPM启动超时问题及YAML配置优化咨询

通过Kubernetes的Deployment部署JBPM，将replicas副本数设置为2时，第二个Pod状态显示为Running，但日志中出现启动错误，提示WildFly管理接口添加操作超时，最终服务启动失败。

错误日志 #

2024-12-19T17:26:59.925260684+08:00 [0m[31m17:26:59,924 ERROR [org.jboss.as.controller.client] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.controller.AbstractAddStepHandler$1@3fb027aa for operation add at address [("interface" => "management")] failed -- java.util.concurrent.TimeoutException
2024-12-19T17:26:59.925260684+08:00 [0m[31m17:27:04,926 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.controller.AbstractAddStepHandler$1@7c9ebd45 for operation add at address [
2024-12-19T17:27:04.926556870+08:00     ("core-service" => "management"),
2024-12-19T17:27:04.926556870+08:00     ("management-interface" => "http-interface")
2024-12-19T17:27:04.926556870+08:00 ] failed -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:525)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1559)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1513)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1496)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext$Step.access$400(AbstractOperationContext.java:1360)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:912)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:762)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:468)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1415)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:529)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:515)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:477)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.server@15.0.0.Final//org.jboss.as.server.ServerService.boot(ServerService.java:459)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.server@15.0.0.Final//org.jboss.as.server.ServerService.boot(ServerService.java:412)
2024-12-19T17:27:04.926556870+08:00     at org.jboss.as.controller@15.0.0.Final//org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:416)
2024-12-19T17:27:04.926556870+08:00     at java.base/java.lang.Thread.run(Thread.java:834)
2024-12-19T17:27:04.926556870+08:00 
2024-12-19T17:27:04.926556870+08:00 [0m[31m17:27:04,926 ERROR [org.jboss.as.controller.client] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.controller.AbstractAddStepHandler$1@7c9ebd45 for operation add at address [
2024-12-19T17:27:04.926662472+08:00     ("core-service" => "management"),
2024-12-19T17:27:04.926662472+08:00     ("management-interface" => "http-interface")
2024-12-19T17:27:04.926662472+08:00 ] failed -- java.util.concurrent.TimeoutException
2024-12-19T17:27:04.926662472+08:00 [0m[31m17:27:09,926 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0349: Timeout after [5] seconds waiting for service container stability while finalizing an operation. Process must be restarted. Step that first updated the service container was 'add' at address '[
2024-12-19T17:27:09.927512515+08:00     ("core-service" => "management"),
2024-12-19T17:27:09.927512515+08:00     ("management-interface" => "http-interface")
2024-12-19T17:27:09.927512515+08:00 ]'
2024-12-19T17:27:09.927512515+08:00 [0m[31m17:27:09,931 FATAL [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
2024-12-19T17:27:09.932232545+08:00 [0m

当前使用的Deployment和Service配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jbpm-oracle
  namespace: carrefour
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jbpm-oracle
  template:
    metadata:
      labels:
        app: jbpm-oracle
    spec:
      initContainers:
      - name: init-copy-bin
        image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.74.0.final:oracle-0821-1
        command: ["/bin/sh", "-c"]
        args:
          - >
            if [ ! -f /mnt/wildfly-bin/.initialized ]; then
              cp -r /opt/jboss/wildfly/bin/* /mnt/wildfly-bin/;
              chown -R jboss:jboss /mnt/wildfly-bin;
              touch /mnt/wildfly-bin/.initialized;
            fi
        volumeMounts:
        - name: jbpm-bin-data
          mountPath: /mnt/wildfly-bin
      - name: init-copy-standalone-conf
        image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.74.0.final:oracle-0821-1
        command: ["/bin/sh", "-c"]
        args:
          - >
            if [ ! -f /mnt/wildfly-standalone-conf/.initialized ]; then
              cp -r /opt/jboss/wildfly/standalone/configuration/* /mnt/wildfly-standalone-conf/;
              chown -R jboss:jboss /mnt/wildfly-standalone-conf/;
              touch /mnt/wildfly-standalone-conf/.initialized;
            fi
        volumeMounts:
        - name: jbpm-standalone-conf
          mountPath: /mnt/wildfly-standalone-conf
      - name: init-copy-standalone-data
        image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.74.0.final:oracle-0821-1
        command: ["/bin/sh", "-c"]
        args:
          - >
            if [ ! -f /mnt/wildfly-standalone-data/.initialized ]; then
              cp -r /opt/jboss/wildfly/standalone/data/* /mnt/wildfly-standalone-data/;
              chown -R jboss:jboss /mnt/wildfly-standalone-data/;
              touch /mnt/wildfly-standalone-data/.initialized;
            fi
        volumeMounts:
        - name: jbpm-standalone-data
          mountPath: /mnt/wildfly-standalone-data
      containers:
      - name: jbpm-oracle
        image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.74.0.final:oracle-0821-1
        # image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.61.0.final:oracle-0821-7
        env:
        - name: TZ
          value: Asia/Taipei
        - name: JBPM_DB_DRIVER
          value: "oracle"
        - name: JBPM_DB_HOST
          value: "192.168.51.208"
        - name: JBPM_DB_PORT
          value: "1522"
        - name: JBPM_DB_NAME
          value: "jbpm"
        - name: JBPM_DB_USER
          value: "jbpm"
        - name: JBPM_DB_PASSWORD
          value: "jbpm"
        ports:
        - containerPort: 8080
        - containerPort: 8001
        volumeMounts:
        - name: jbpm-bin-data
          mountPath: /opt/jboss/wildfly/bin
        - name: jbpm-standalone-conf
          mountPath: /opt/jboss/wildfly/standalone/configuration
        - name: jbpm-standalone-data
          mountPath: /opt/jboss/wildfly/standalone/data
      volumes:
      - name: jbpm-bin-data
        persistentVolumeClaim:
          claimName: jbpm-bin-pvc
      - name: jbpm-standalone-conf
        persistentVolumeClaim:
          claimName: jbpm-standalone-conf-pvc
      - name: jbpm-standalone-data
        persistentVolumeClaim:
          claimName: jbpm-standalone-data-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: jbpm-oracle
  namespace: carrefour
spec:
  ports:
    - name: business-central
      port: 8080
      targetPort: 8080
    - name: jbpm-internal-git
      port: 8001
      targetPort: 8001
  selector:
    app: jbpm-oracle
---
apiVersion: v1
kind: Service
metadata:
  name: jbpm-oracle-np
  namespace: carrefour
spec:
  type: NodePort
  ports:
    - name: business-central
      port: 8080
      targetPort: 8080
      nodePort: 30062  
    - name: jbpm-internal-git
      port: 8001
      targetPort: 8001
      nodePort: 30063  
    - name: business-central-ssl
      port: 8443
      targetPort: 8443
      nodePort: 30064  
    - name: wildfly-console
      port: 9990
      targetPort: 9990
      nodePort: 30065  
  selector:
    app: jbpm-oracle

问题根源是多个Pod共享读写模式的PersistentVolume（PV），导致WildFly启动时多实例同时操作同一配置/数据目录，引发文件锁冲突或资源竞争，最终导致管理接口初始化超时。需从以下几方面修改配置：

1. 调整存储卷配置，避免多Pod共享非集群目录 #

• 对于jbpm-bin-data、jbpm-standalone-conf这类无需跨Pod共享的目录，改用emptyDir，让每个Pod拥有独立的本地副本：

# 修改Deployment的volumes部分
volumes:
- name: jbpm-bin-data
  emptyDir: {}
- name: jbpm-standalone-conf
  emptyDir: {}
- name: jbpm-standalone-data
  persistentVolumeClaim:
    claimName: jbpm-standalone-data-pvc

• 若jbpm-standalone-data需要跨Pod共享，必须使用支持多节点读写的存储（如NFS），并将PVC的访问模式设置为ReadWriteMany（RWX），同时确保JBPM配置使用数据库存储状态而非本地文件。

2. 配置WildFly绑定到所有网络接口 #

默认WildFly可能仅绑定localhost，导致Pod内服务初始化时无法访问自身管理接口。在Deployment的container环境变量中添加：

env:
# 新增以下环境变量
- name: JBOSS_BIND_ADDRESS
  value: "0.0.0.0"
- name: JBOSS_BIND_ADDRESS_MANAGEMENT
  value: "0.0.0.0"

3. 延长WildFly启动超时时间 #

日志显示超时时间为5秒，可通过启动参数延长超时：

containers:
- name: jbpm-oracle
  image: reg.harbor.asi.com.tw/carrefour/jbpm-server-full-7.74.0.final:oracle-0821-1