Elasticsearch从7.17.7迁移至8.5.2后Kubernetes Pod CrashLoopBackOff
问题描述
在Kubernetes中使用以下清单已成功部署Elasticsearch 7.17.7版本,运行正常:
apiVersion: apps/v1 kind: StatefulSet metadata: name: elastic spec: serviceName: elastic replicas: 3 selector: matchLabels: app: elastic template: metadata: labels: app: elastic spec: containers: - name: elastic image: docker.elastic.co/elasticsearch/elasticsearch:7.17.7 # image: docker.elastic.co/elasticsearch/elasticsearch:8.5.2 resources: limits: cpu: 1000m memory: 1G requests: cpu: 100m memory: 1G ports: - containerPort: 9200 name: rest protocol: TCP - containerPort: 9300 name: inter-node protocol: TCP volumeMounts: - name: data mountPath: /usr/share/elasticsearch/data env: - name: cluster.name value: mynamespace - name: node.name valueFrom: fieldRef: fieldPath: metadata.name - name: discovery.seed_hosts value: "elastic-0.elasticsearch,elastic-1.elasticsearch,elastic-2.elasticsearch" - name: cluster.initial_master_nodes value: "elastic-0,elastic-1,elastic-2" - name: ES_JAVA_OPTS value: "-Xms512m -Xmx512m" initContainers: - name: increase-vm-max-map image: busybox command: ["sysctl", "-w", "vm.max_map_count=262144"] securityContext: privileged: true - name: increase-fd-ulimit image: busybox command: ["sh", "-c", "ulimit -n 65536"] securityContext: privileged: true volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: nfs-1 resources: requests: storage: 50Mi --- kind: Service apiVersion: v1 metadata: name: elastic labels: app: elastic spec: selector: app: elastic clusterIP: None ports: - port: 9200 name: rest - port: 9300 name: inter-node
仅将镜像版本替换为8.5.2后重新部署,所有Pod出现CrashLoopBackOff状态,退出码为78,日志报错如下:
2022-12-06 07:12:44,449 process reaper (pid 86) ERROR Recursive call to appender rolling {"@timestamp":"2022-12-06T07:12:44.250Z", "log.level":"ERROR", "message":"uncaught exception in thread [process reaper (pid 86)]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"process reaper (pid 86)","log.logger":"org.elasticsearch.bootstrap.ElasticsearchUncaughtExceptionHandler","elasticsearch.node.name":"elastic-0","error.type":"java.security.AccessControlException","error.message":"access denied (\"java.lang.RuntimePermission\" \"modifyThread\")","error.stack_trace":"java.security.AccessControlException: access denied (\"java.lang.RuntimePermission\" \"modifyThread\")\n\tat java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)\n\tat java.base/java.security.AccessController.checkPermission(AccessController.java:1068)\n\tat java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:411)\n\tat org.elasticsearch.securesm@8.5.2/org.elasticsearch.secure_sm.SecureSM.checkThreadAccess(SecureSM.java:166)\n\tat org.elasticsearch.securesm@8.5.2/org.elasticsearch.secure_sm.SecureSM.checkAccess(SecureSM.java:120)\n\tat java.base/java.lang.Thread.checkAccess(Thread.java:2360)\n\tat java.base/java.lang.Thread.setDaemon(Thread.java:2308)\n\tat java.base/java.lang.ProcessHandleImpl.lambda$static$0(ProcessHandleImpl.java:103)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:637)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:928)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1021)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1589)\n\tat java.base/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)\n"}
问题原因分析
- Elasticsearch 8.x版本默认启用了安全管理器(Security Manager),原有7.x的部署配置未提供足够的容器权限来满足安全管理器的运行要求。
- 报错中的
java.security.AccessControlException明确指出,安全管理器阻止了进程获取modifyThread权限的操作——这是因为Elasticsearch 8.x基于Java 17,Java 17对线程操作的权限控制更严格,同时容器的安全上下文配置不足以让Elasticsearch进程绕过这个限制。 - 此外,Elasticsearch 8.x默认开启了HTTPS、身份验证等安全特性,这些默认设置也会和旧版本的无安全配置产生冲突,但当前直接导致Pod启动失败的核心原因是安全管理器的权限不足问题。
内容的提问来源于stack exchange,提问作者PatPanda
相关产品推荐
相关产品推荐

