GitHub Actions中Terraform销毁EKS集群失败问题咨询
在GitHub Actions集成工作流中执行Terraform Destroy操作失败。
GitHub Actions工作流配置
name: 'integration' on: push: branches: - '**' - '!main' workflow_dispatch: permissions: id-token: write contents: read deployments: write jobs: integration: runs-on: ubuntu-latest concurrency: group: canary cancel-in-progress: false defaults: run: working-directory: examples/complete/ steps: - name: 'Checkout' uses: actions/checkout@v3.5.2 - name: 'Extract branch name' id: extract_branch shell: bash run: echo "##[set-output name=branch;]$(echo ${{ github.ref_name }})" - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v2 with: role-to-assume: arn:aws:iam::702906146300:role/terraform-aws-eks-primary aws-region: us-west-2 - name: 'Setup Terraform' uses: hashicorp/setup-terraform@v2.0.3 with: terraform_version: 1.3.7 - name: 'Terraform Init' id: init run: terraform init -force-copy -input=false - name: 'Terraform Validate' id: validate run: terraform validate -no-color - name: 'Terraform Plan' id: plan run: terraform plan -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false - name: 'Start deployment' uses: bobheadxi/deployments@v1.4.0 id: deployment with: step: start token: ${{ secrets.GITHUB_TOKEN }} env: canary - name: 'Terraform Apply' id: apply run: | terraform apply -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false -auto-approve terraform apply -no-color -input=false -auto-approve - name: 'Terraform Destroy' id: destroy if: always() run: terraform destroy -no-color -input=false -auto-approve - name: 'Finish deployment' uses: bobheadxi/deployments@v1.4.0 if: always() with: step: finish token: ${{ secrets.GITHUB_TOKEN }} status: ${{ job.status }} env: ${{ steps.deployment.outputs.env }} env_url: https://github.com/${{ github.repository }}/actions?query=workflow%3A${{ github.workflow }}+branch%3A${{ steps.extract_branch.outputs.branch }} deployment_id: ${{ steps.deployment.outputs.deployment_id }}
失败命令
terraform destroy -no-color -input=false -auto-approve
我们使用terraform-aws-eks模块部署AWS EKS集群,尝试过多个版本但问题依旧。推测问题不在Terraform本身,而是模块或销毁流程的依赖关系导致:EKS集群能被销毁,但模块管理的集群安全组因虚拟节点、VPC访问的依赖关系无法删除。
错误信息
Error: deleting Security Group (sg-0b11ee4a81d0092b2): DependencyViolation: resource sg-0b11ee4a81d0092b2 has a dependent object status code: 400, request id: 8a8dfd26-5198-4bbd-9f0b-84131c248434
main.tf配置
################################################ # KMS CLUSTER ENCRYPTION KEY # ################################################ resource "aws_kms_key" "this" { description = "EKS Cluster Encyrption Key" deletion_window_in_days = 7 enable_key_rotation = true } resource "aws_kms_alias" "this" { name = "alias/eks_cluster_encryption_key" target_key_id = aws_kms_key.this.key_id } ################################## # KUBERNETES CLUSTER # ################################## module "primary" { source = "terraform-aws-modules/eks/aws" version = "~> 19.4.3" cluster_name = var.cluster_name cluster_version = var.cluster_version cluster_endpoint_private_access = var.cluster_endpoint_private_access cluster_endpoint_public_access = var.cluster_endpoint_public_access create_cloudwatch_log_group = false create_kms_key = false cluster_encryption_config = { resources = ["secrets"] provider_key_arn = aws_kms_key.this.arn } create_cni_ipv6_iam_policy = var.create_cni_ipv6_iam_policy manage_aws_auth_configmap = true aws_auth_roles = var.aws_auth_roles vpc_id = var.vpc_id subnet_ids = var.subnet_ids eks_managed_node_group_defaults = { ami_type = var.ami_type disk_size = var.disk_size instance_types = var.instance_types iam_role_attach_cni_policy = var.iam_role_attach_cni_policy } eks_managed_node_groups = { primary = { min_size = 1 max_size = 5 desired_size = 1 capacity_type = "ON_DEMAND" } secondary = { min_size = 1 max_size = 5 desired_size = 1 capacity_type = "SPOT" } } cluster_addons = { coredns = { most_recent = true resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" timeouts = { create = "20m" delete = "20m" update = "20m" } } kube-proxy = { most_recent = true resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" timeouts = { create = "20m" delete = "20m" update = "20m" } } aws-ebs-csi-driver = { most_recent = true resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" timeouts = { create = "20m" delete = "20m" update = "20m" } } vpc-cni = { most_recent = true resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "PRESERVE" timeouts = { create = "20m" delete = "20m" update = "20m" } } } tags = { repo = "https://github.com/impinj-di/terraform-aws-eks-primary" team = "di" owner = "di_admins@impinj.com" } } #################################### # KUBERNETES RESOURCES # #################################### resource "kubernetes_namespace" "this" { depends_on = [module.primary] for_each = toset(local.eks_namespaces) metadata { name = each.key } }
未指定集群及节点间安全组,使用模块默认配置。
疑问
是否应在工作流中使用如下命令销毁基础设施?-force参数是否有效?
terraform destroy -force -no-color -input=false -auto-approve
首先明确:-force参数在Terraform 0.15及以后版本中已被移除,你当前使用的Terraform 1.3.7完全不支持该参数,添加它无法解决任何问题,不要在命令中使用。
针对安全组删除失败的问题,核心原因是AWS侧存在未被Terraform管理的依赖对象(如弹性网卡ENI、负载均衡关联),或Terraform销毁顺序不符合AWS实际依赖逻辑,以下是具体解决思路:
修正重复执行Terraform Apply的问题
工作流中连续执行两次terraform apply,第二次未传递变量可能导致资源状态不一致,增加销毁时的依赖问题。建议合并为一次执行,确保变量传递完整:terraform apply -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false -auto-approve调整销毁顺序,先清理EKS核心依赖
在完整销毁前,先手动删除节点组和集群附加组件,确保安全组的关联依赖被清除:# 先删除EKS托管节点组 terraform destroy -target=module.primary.eks_managed_node_groups -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false -auto-approve # 再删除集群附加组件 terraform destroy -target=module.primary.cluster_addons -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false -auto-approve # 最后执行完整销毁 terraform destroy -var="create_cni_ipv6_iam_policy=true" -var="iam_role_attach_cni_policy=true" -no-color -input=false -auto-approve注意必须传递与apply阶段一致的变量,避免状态不匹配。
检查并清理AWS侧残留资源
如果上述步骤仍失败,登录AWS控制台查看报错的安全组,检查是否有未被Terraform销毁的ENI、负载均衡等关联资源。手动清理这些依赖后,重新执行销毁命令。升级terraform-aws-eks模块版本
你当前使用的19.4.3版本可能存在销毁顺序的bug,尝试升级到较新的稳定版本(如>=19.15.x),新版本通常已修复这类依赖销毁问题。
内容的提问来源于stack exchange,提问作者R. Barrett

