Pega Helm部署失败：pega-web-tomcat容器启动报空指针异常

环境信息 #

• Kubernetes集群：microk8s v1.27.8（3节点，单节点32GB内存）
• 存储类：nfs
• 操作系统：Ubuntu 22.04.3 LTS（GNU/Linux 5.15.0-91-generic x86_64）
• 系统状态（2024年1月21日UTC 22:22:24）：
  • 系统负载：0.6806640625
  • 根目录使用率：97.87GB的26.5%
  • 内存使用率：18%
  • 交换分区使用率：0%
  • 进程数：223

部署配置（values.yaml） #

---
global:
  provider: "k8s"

  deployment:
    name: "dev1"

  actions:
    execute: "install"

  storageClassName: ""
  jdbc:
    url: "jdbc:postgresql://192.168.1.xxx:5432/Pega"
    driverClass: "org.postgresql.Driver"
        driverUri: "https://jdbc.postgresql.org/download/postgresql-42.5.4.jar"
    username: "pega"
    password: "rules"
    external_secret_name: ""
    connectionProperties: ""
    rulesSchema: "rules"
    dataSchema: "data"
    customerDataSchema: "customer"

  customArtifactory:
    authentication:
      basic:
        username: ""
        password: ""
      apiKey:
        headerName: ""
        value: ""
      external_secret_name: ""
    enableSSLVerification: true
    certificate:

  docker:
    registry:
      url: "https://hub.docker.com"
      username: "myrepo"
      password: "xxx"
    imagePullSecretNames: []
    pega:
      image: "myrepo/my-pega:23"

  utilityImages:
    busybox:
      image: busybox:1.31.0
      imagePullPolicy: IfNotPresent
    k8s_wait_for:
      image: Pegasystems/k8s-wait-for
      imagePullPolicy: "IfNotPresent"
      # waitTimeSeconds: 2
      # maxRetries: 1

  upgrade:
    kube-apiserver:
      serviceHost: "https://192.168.1.xxx"
      httpsServicePort: "16443"

  compressedConfigurations: false

  tier:
    - name: "k8s-dev"
      nodeType: "Stream,BackgroundProcessing,WebUser,Search,DataFlow"
      requestor:
        passivationTimeSec: 900
      service:
        httpEnabled: true
        port: 80
        targetPort: 8080
        tls:
          enabled: false
          external_secret_name: ""
          keystore:
          keystorepassword:
          port: 443
          targetPort: 8443
          cacertificate:
          certificateFile:
          certificateKeyFile:
          traefik:
            enabled: false
            serverName: ""
            insecureSkipVerify: false

      ingress:
        domain: "web.k8s-dev.org.local"
        tls:
          enabled: false
          certificate:
          key:
          cacertificate:

      replicas: 1
      javaOpts: ""
      pegaDiagnosticUser: ""
      pegaDiagnosticPassword: ""

      volumeClaimTemplate:
        resources:
          requests:
            storage: 5Gi

      deploymentStrategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate

      livenessProbe:
        port: 8081

      hpa:
        enabled: false

      pdb:
        enabled: false
        minAvailable: 1

cassandra:
  enabled: true
  persistence:
    enabled: true
  resources:
    requests:
      memory: "4Gi"
      cpu: 2
    limits:
      memory: "8Gi"
      cpu: 4

dds:
  port: "9042"
  username: "dnode_ext"
  password: "dnode_ext"
  clientEncryption: false
  trustStore: ""
  trustStorePassword: ""
  keyStore: ""
  keyStorePassword: ""
  external_secret_name: ""
  asyncProcessingEnabled: false
  keyspacesPrefix: ""
  extendedTokenAwarePolicy: false
  latencyAwarePolicy: false
  customRetryPolicy: false
  customRetryPolicyEnabled: false
  customRetryPolicyCount: 1
  speculativeExecutionPolicy: false
  speculativeExecutionPolicyEnabled: false
  speculativeExecutionPolicyDelay: 100
  speculativeExecutionPolicyMaxExecutions: 2
  jmxMetricsEnabled: true
  csvMetricsEnabled: false
  logMetricsEnabled: false

pegasearch:
  image: "myrepo/my-pega-search:23"
  memLimit: "3Gi"
  replicas: 1

installer:
  image: "myrepo/my-pega-installer:23"
  imagePullPolicy: Always
  adminPassword: "passwordX"
  upgrade:
    upgradeType: "in-place"
    targetRulesSchema: "rules"
    targetDataSchema: "data"
    pegaRESTUsername: ""
    pegaRESTPassword: ""

hazelcast:
  image: "YOUR_HAZELCAST_IMAGE:TAG"
  clusteringServiceImage: "YOUR_CLUSTERING_SERVICE_IMAGE:TAG"
  enabled: false
  clusteringServiceEnabled: false
  migration:
    initiateMigration: false
    migrationJobImage: "YOUR_MIGRATION_JOB_IMAGE:TAG"
    embeddedToCSMigration: false
  replicas: 3
  username: ""
  password: ""
  external_secret_name: ""

stream:
  enabled: "false"
  bootstrapServer: ""
  securityProtocol: PLAINTEXT
  trustStore: ""
  trustStorePassword: ""
  keyStore: ""
  keyStorePassword: ""
  jaasConfig: ""
  saslMechanism: PLAIN
  streamNamePattern: "pega-{stream.name}"
  replicationFactor: "1"
  external_secret_name: ""

问题描述 #

通过Helm部署Pega，采用外部Kafka、自建Cassandra作为DSS存储，启动接近完成时pega-web-tomcat容器无法正常启动，日志显示调用doPostInner方法时出现NullPointerException。

报错日志 #

21-Jan-2024 21:41:43.305 SEVERE [http-nio2-8080-exec-5] com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega. 方法调用（doPostInner）出错
    java.lang.NullPointerException
        at com.pega.pegarules.session.internal.PRSessionProviderImpl.borrowRequestor(PRSessionProviderImpl.java:740)
        at com.pega.pegarules.session.internal.PRSessionProviderImpl.borrowRequestor(PRSessionProviderImpl.java:755)
        at com.pega.pegarules.session.internal.PRSessionProviderImplForModules.borrowRequestor(PRSessionProviderImplForModules.java:60)
        at com.pega.platform.web.request.internal.ResponseProcessorImpl.getBrowserRequestor(ResponseProcessorImpl.java:222)
        at com.pega.platform.web.request.internal.ResponseProcessorImpl.getStream(ResponseProcessorImpl.java:184)
        at com.pega.platform.web.request.internal.ResponseProcessorImpl.getStatusPageStream(ResponseProcessorImpl.java:174)
        at com.pega.platform.web.request.internal.ResponseProcessorImpl.writeStatusOutput(ResponseProcessorImpl.java:191)
        at com.pega.pegarules.web.impl.WebStandardImpl.doPostInner(WebStandardImpl.java:484)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:388)
        at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:430)
        at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:225)
        at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:274)
        at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPost(WebStandardBoot.java:164)
        at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doGet(WebStandardBoot.java:121)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:529)
        at com.pega.pegarules.internal.web.servlet.WebStandardBoot.service(WebStandardBoot.java:191)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:670)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:765)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:928)
        at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRun(Nio2Endpoint.java:1732)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.net.AbstractEndpoint.processSocket(AbstractEndpoint.java:1295)
        at org.apache.tomcat.util.net.Nio2Endpoint.setSocketOptions(Nio2Endpoint.java:339)
        at org.apache.tomcat.util.net.Nio2Endpoint$Nio2Acceptor.completed(Nio2Endpoint.java:479)
        at org.apache.tomcat.util.net.Nio2Endpoint$Nio2Acceptor.completed(Nio2Endpoint.java:415)
        at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:127)
        at java.base/sun.nio.ch.Invoker$2.run(Invoker.java:219)
        at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:829)

排查与解决方向 #

• 检查Cassandra集群状态：执行kubectl exec -it <cassandra-pod-name> -- nodetool status，确认所有节点处于Up/Normal状态，Pega可正常连接。
• 验证JDBC连接：在web容器内执行psql -h 192.168.1.xxx -U pega -d Pega，确认PostgreSQL可访问、用户名密码正确、schema权限充足。
• 添加JVM参数：在tier.javaOpts中配置-Xms4g -Xmx8g，避免内存不足导致初始化失败。
• 检查installer作业状态：执行kubectl logs <installer-pod-name>，确认数据库初始化无报错。
• 调整探针配置：给livenessProbe添加initialDelaySeconds: 300（根据实际初始化时长调整），避免过早触发探测导致容器重启。
• 检查存储权限：执行kubectl exec -it <web-pod-name> -- ls -ld /path/to/mounted/volume，确认NFS卷挂载正常、容器有读写权限。

内容的提问来源于stack exchange，提问作者Thomas