如何通过Terraform自动为AWS EKS关联的EBS卷添加标签

我用Terraform部署了AWS EKS集群，集群运行正常，但执行kubectl apply部署Zookeeper后，Pod一直卡在Init状态。排查发现是关联的EBS卷缺少企业要求的必要标签，手动在AWS控制台添加标签后问题解决。我曾尝试在EBS资源中添加volume_tags字段但无效，请问如何通过Terraform脚本自动为这些动态创建的EBS卷添加所需标签？

附相关脚本及错误信息：

EKS.tf #

provider "aws" {
  region  = var.region
  profile = var.profile
  default_tags {
    tags = {
      Name          = "Example Eks cluster"
      Owner         = ""
      ChargeCode    = ""
      ProjectId     = ""
      ApplicationId = ""
      #Environment = ""
      PatchGroup = ""
      Eeol       = ""
      Oic        = ""
      GovId      = ""
      CommId     = ""
    }
  }
}

# Filter out local zones, which are not currently supported 
# with managed node groups
data "aws_availability_zones" "available" {
  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

locals {
  cluster_name              = "test-eks-cluster"
  cluster_enabled_log_types = []
}



module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "19.15.3"

  cluster_name    = local.cluster_name
  cluster_version = "1.27"

  cluster_endpoint_public_access  = true
  cluster_endpoint_private_access = true

  vpc_id     = ""
  subnet_ids = ["", ""]

  cluster_enabled_log_types = local.cluster_enabled_log_types

  create_cloudwatch_log_group = false

  tags = var.tags


  eks_managed_node_group_defaults = {
    #ami_type = "ami-06d7aa002b2e3009b"
    ami_type = "AL2_x86_64"
    tags     = var.tags


  }

  eks_managed_node_groups = {
    one = {
      name = "node-group-1"

      instance_types = ["t3.small"]

      min_size     = 1
      max_size     = 3
      desired_size = 2
      tags         = var.tags
    }
  }
  # access_entries = {

  #}
}

data "aws_iam_policy" "ebs_csi_policy" {
  arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

module "irsa-ebs-csi" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
  version = "4.7.0"

  create_role                   = true
  role_name                     = "AmazonEKSTFEBSCSIRole-${module.eks.cluster_name}"
  provider_url                  = module.eks.oidc_provider
  role_policy_arns              = [data.aws_iam_policy.ebs_csi_policy.arn]
  oidc_fully_qualified_subjects = ["system:serviceaccount:kube-system:ebs-csi-controller-sa"]
  tags                          = var.tags
}

resource "aws_eks_addon" "ebs-csi" {
  cluster_name             = module.eks.cluster_name
  addon_name               = "aws-ebs-csi-driver"
  addon_version            = "v1.30.0-eksbuild.1"
  service_account_role_arn = module.irsa-ebs-csi.iam_role_arn
  tags = {
    "eks_addon"   = ""
    "terraform"   = ""
    Name          = ""
    Owner         = ""
    ChargeCode    = ""
    ProjectId     = ""
    ApplicationId = ""
    #Environment = ""
    PatchGroup = ""
    Eeol       = ""
    Oic        = ""
    GovId      = ""
    CommId     = ""
  }
}

zk.yml #

apiVersion: platform.confluent.io/v1beta1
kind: Zookeeper
metadata:
  name: zookeeper
  namespace: confluent
spec:
  replicas: 3
  image:
    application: confluentinc/cp-zookeeper:7.5.0
    init: confluentinc/confluent-init-container:2.7.0
  dataVolumeCapacity: 10Gi
  logVolumeCapacity: 10Gi

错误示例 #

AttachVolume.Attach failed for volume "pvc-68f45cb0-d06e-4e19-b0ff-c88ecb53f4c8" : rpc error: code = Internal desc = Could not attach volume "vol-0ccc9938279c3a256" to node "i-061ea33a9f07efffb": could not attach volume "vol-0ccc9938279c3a256" to node "i-061ea33a9f07efffb": operation error EC2: AttachVolume, https response error StatusCode: 403, RequestID: a8d8596c-d9b6-4210-9919-bd5ce67f37f0, api error UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws-us-gov:sts::000451337248:assumed-role/AmazonEKSTFEBSCSIRole-test-eks-cluster/1716299655525948774 is not authorized to perform: ec2:AttachVolume on resource: arn:aws-us-gov:ec2:us-gov-east-1:000451337248:volume/vol-0ccc9938279c3a256 with an explicit deny in a service control policy

因为你的EBS卷是由AWS EBS CSI Driver动态创建的（通过Zookeeper的PVC触发），所以无法直接在Terraform中通过aws_ebs_volume资源添加标签。以下两种方法可以实现自动添加企业要求的标签：

方法1：配置EBS CSI Driver添加默认标签 #

通过修改Terraform中的aws_eks_addon资源，为EBS CSI Driver添加配置参数，让它为所有动态创建的EBS卷自动添加标签。

修改aws_eks_addon "ebs-csi"部分，添加configuration_values字段：

resource "aws_eks_addon" "ebs-csi" {
  cluster_name             = module.eks.cluster_name
  addon_name               = "aws-ebs-csi-driver"
  addon_version            = "v1.30.0-eksbuild.1"
  service_account_role_arn = module.irsa-ebs-csi.iam_role_arn
  
  # 添加CSI Driver的配置，设置默认卷标签
  configuration_values = jsonencode({
    controller = {
      volumeTags = {
        Name          = "Zookeeper-EBS-Volume"
        Owner         = "your-owner-value"
        ChargeCode    = "your-chargecode-value"
        ProjectId     = "your-projectid-value"
        ApplicationId = "your-appid-value"
        PatchGroup    = "your-patchgroup-value"
        Eeol          = "your-eeol-value"
        Oic           = "your-oic-value"
        GovId         = "your-govid-value"
        CommId        = "your-commid-value"
      }
    }
  })

  tags = {
    "eks_addon"   = ""
    "terraform"   = ""
    Name          = ""
    Owner         = ""
    ChargeCode    = ""
    ProjectId     = ""
    ApplicationId = ""
    #Environment = ""
    PatchGroup = ""
    Eeol       = ""
    Oic        = ""
    GovId      = ""
    CommId     = ""
  }
}

同时，确保IRSA角色（AmazonEKSTFEBSCSIRole-test-eks-cluster）拥有ec2:CreateTags权限，默认的AmazonEBSCSIDriverPolicy已经包含该权限，但可以检查确认。

方法2：创建带标签的StorageClass并让Zookeeper使用 #

通过Terraform创建自定义StorageClass，指定卷标签，然后修改Zookeeper的配置使用该StorageClass。

步骤1：在Terraform中添加StorageClass资源 #

在EKS.tf中添加以下内容：

resource "kubernetes_storage_class" "zookeeper_ebs" {
  metadata {
    name = "zookeeper-ebs-sc"
    annotations = {
      "storageclass.kubernetes.io/is-default-class" = "false"
    }
  }
  provisioner = "ebs.csi.aws.com"
  parameters = {
    type = "gp3"
    # 添加企业要求的标签
    "tag:Name"          = "Zookeeper-EBS-Volume"
    "tag:Owner"         = "your-owner-value"
    "tag:ChargeCode"    = "your-chargecode-value"
    "tag:ProjectId"     = "your-projectid-value"
    "tag:ApplicationId" = "your-appid-value"
    "tag:PatchGroup"    = "your-patchgroup-value"
    "tag:Eeol"          = "your-eeol-value"
    "tag:Oic"           = "your-oic-value"
    "tag:GovId"         = "your-govid-value"
    "tag:CommId"        = "your-commid-value"
  }
  reclaim_policy = "Retain"
  volume_binding_mode = "WaitForFirstConsumer"
}

步骤2：修改Zookeeper配置使用该StorageClass #

修改zk.yml，指定dataVolumeStorageClass和logVolumeStorageClass：

apiVersion: platform.confluent.io/v1beta1
kind: Zookeeper
metadata:
  name: zookeeper
  namespace: confluent
spec:
  replicas: 3
  image:
    application: confluentinc/cp-zookeeper:7.5.0
    init: confluentinc/confluent-init-container:2.7.0
  dataVolumeCapacity: 10Gi
  dataVolumeStorageClass: "zookeeper-ebs-sc" # 添加这行
  logVolumeCapacity: 10Gi
  logVolumeStorageClass: "zookeeper-ebs-sc" # 添加这行

注意事项 #

• 两种方法选其一即可，方法1是全局生效，所有CSI创建的卷都会带标签；方法2是针对Zookeeper的卷单独配置。
• 确保标签值和企业要求一致，替换示例中的your-xxx-value为实际值。
• 应用Terraform变更后，需要删除现有PVC和Pod，重新创建才能让新的标签生效（因为已有卷的标签不会自动更新）。

内容的提问来源于stack exchange，提问作者user23627827