在K8s集群通过Helm部署Elasticsearch时遇GeoLite2数据库获取失败问题
问题描述
在AWS节点搭建的K8s集群上,通过Helm执行命令 helm install elasticsearch elastic/elasticsearch -f values-aws.yaml 部署elasticsearch-8.5.1集群后,出现如下错误日志:
{"@timestamp":"2023-05-16T01:22:09.414Z", "log.level":"ERROR", "message":"failed to retrieve database [GeoLite2-City.mmdb]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch-master-0][generic][T#2]","log.logger":"org.elasticsearch.ingest.geoip.DatabaseNodeService","elasticsearch.node.name":"elasticsearch-master-0","elasticsearch.cluster.name":"elasticsearch","error.type":"org.elasticsearch.cluster.block.ClusterBlockException","error.message":"blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];"}
使用的values-aws.yaml配置内容如下:
# Shrink default JVM heap. esJavaOpts: "-Xmx128m -Xms128m" # Allocate smaller chunks of memory per pod. resources: requests: cpu: "100m" memory: "512M" limits: cpu: "1000m" memory: "512M" # To allocate more space if above requests and limits aren't enough # resources: # requests: # cpu: "500m" # memory: "2Gi" # limits: # cpu: "1000m" # memory: "5G" # Request smaller persistent volumes. volumeClaimTemplate: accessModes: [ "ReadWriteOnce" ] storageClassName: "efs-sc" resources: requests: storage: 100M
排查步骤
- 检查集群健康状态:执行以下命令查看集群初始化进度
重点关注kubectl exec -it elasticsearch-master-0 -- curl -u <elastic-user>:<elastic-password> http://localhost:9200/_cluster/health?prettystatus字段(red/yellow/green)和unassigned_shards数值,red状态说明主分片未分配,集群无法正常提供服务。 - 验证节点资源占用:用命令查看Pod的CPU和内存使用率
若内存使用率接近上限,说明资源配置不足导致节点无法完成初始化。kubectl top pod elasticsearch-master-0 - 检查存储挂载情况:执行命令查看Pod的存储绑定详情
确认kubectl describe pod elasticsearch-master-0efs-sc存储类对应的PVC是否成功绑定,挂载路径是否正常,同时注意EFS的访问模式是否适配多节点集群。 - 定位错误根源:日志中的GeoIP数据库获取失败是集群未就绪的结果,而非原因,核心问题是集群处于未恢复/未初始化状态。
解决建议
- 调整资源配置(核心修复):Elasticsearch对内存要求较高,修改
values-aws.yaml中的JVM参数和资源限制,满足最低运行要求:esJavaOpts: "-Xmx2g -Xms2g" # JVM堆内存建议为物理内存的50%,且不超过32G resources: requests: cpu: "500m" memory: "4Gi" limits: cpu: "1000m" memory: "4Gi" - 修正存储访问模式:若部署多节点集群,EFS存储类需使用
ReadWriteMany模式,否则仅单个节点能挂载存储,导致集群无法形成quorum:volumeClaimTemplate: accessModes: [ "ReadWriteMany" ] storageClassName: "efs-sc" resources: requests: storage: 10Gi # 测试环境建议至少10G,避免空间不足 - 重新部署集群:先卸载现有实例,再重新安装
helm uninstall elasticsearch helm install elasticsearch elastic/elasticsearch -f values-aws.yaml - 等待集群就绪:执行命令监控StatefulSet的部署状态
待状态变为success后,再次执行集群健康检查确认状态为green。kubectl rollout status statefulset/elasticsearch-master - 可选:禁用GeoIP插件(若无需该功能)
plugins: install: [] # 不安装额外插件,包括GeoIP
内容的提问来源于stack exchange,提问作者Raj
相关产品推荐
相关产品推荐

