Terraform部署私有子网内RDS与无公网访问EKS集群的配置难题及解决方案咨询

兄弟，我之前踩过一模一样的坑！把RDS放私有子网、EKS设成私有端点确实是最佳实践，但本地跑的Terraform根本碰不到这些私有资源，直接卡壳。下面给你两个问题的具体解决办法，全自动化，不用手动改任何东西：

一、解决私有子网RDS的数据库创建问题 #

你的核心问题是postgresql provider运行在本地，没有直接网络通路到私有子网的RDS。这里最安全且不需要额外公网资源的方案是用AWS SSM Session Manager做端口转发，配合Terraform的null_resource临时建立隧道，让provider能通过隧道访问RDS。

具体实现步骤 #

1. 先确保你的VPC已经配置了SSM的VPC端点（com.amazonaws.<region>.ssm、com.amazonaws.<region>.ec2messages、com.amazonaws.<region>.ssmmessages），这样不用公网也能使用SSM服务。
2. 添加一个仅通过SSM管理的Bastion实例（不需要公网IP），用来做隧道跳板：

# SSM实例角色，允许实例被AWS管理
resource "aws_iam_role" "ssm_bastion_role" {
  name = "ssm-bastion-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = { Service = "ec2.amazonaws.com" }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ssm_bastion_attach" {
  role       = aws_iam_role.ssm_bastion_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_instance_profile" "ssm_bastion_profile" {
  name = "ssm-bastion-profile"
  role = aws_iam_role.ssm_bastion_role.name
}

# Bastion实例（放公有子网，但关闭公网IP，仅通过SSM访问）
resource "aws_instance" "ssm_bastion" {
  ami           = "ami-0c55b159cbfafe1f0" # Amazon Linux 2 AMI（带SSM Agent）
  instance_type = "t4g.nano"
  subnet_id     = module.vpc.public_subnets[0]
  vpc_security_group_ids = [aws_security_group.bastion_sg.id]
  iam_instance_profile = aws_iam_instance_profile.ssm_bastion_profile.name
  associate_public_ip_address = false # 关闭公网IP，更安全

  tags = { Name = "ssm-bastion" }
}

# Bastion安全组：仅允许SSM服务访问
resource "aws_security_group" "bastion_sg" {
  name        = "bastion-ssm-sg"
  description = "Allow SSM access"
  vpc_id      = module.vpc.vpc_id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"] # 可限制为SSM服务IP段，更安全
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

3. 用null_resource启动SSM端口转发，让本地端口映射到RDS的5432端口：

resource "null_resource" "rds_port_forward" {
  triggers = {
    rds_address = module.rds.db_instance_address
    bastion_id  = aws_instance.ssm_bastion.id
  }

  provisioner "local-exec" {
    # 启动SSM隧道，后台运行
    command = <<EOF
aws ssm start-session \
  --target ${aws_instance.ssm_bastion.id} \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["${module.rds.db_instance_address}"], "portNumber":["5432"], "localPortNumber":["54320"]}' &
sleep 10 # 等待隧道建立完成
EOF

    # Terraform销毁时关闭SSM会话
    on_destroy = <<EOF
pkill -f "aws ssm start-session --target ${aws_instance.ssm_bastion.id}"
EOF
  }
}

4. 修改postgresql provider，使用本地转发的端口，并依赖隧道资源：

provider "postgresql" {
  alias = "instance_admin"

  host            = "localhost"
  port            = 54320
  database        = "postgres"
  username        = module.rds.db_instance_username
  password        = module.rds.db_instance_password
  sslmode         = "require"
  connect_timeout = 15
  superuser       = false

  depends_on = [null_resource.rds_port_forward]
}

# 关联别名provider到你的数据库和角色资源
resource "postgresql_database" "env" {
  provider = postgresql.instance_admin
  name     = "app_database"
}

resource "postgresql_role" "db_role" {
  provider = postgresql.instance_admin
  name     = "db_role"
  login    = true
  password = random_string.postgres_pass.result
}

替代方案：如果不想用postgresql provider，也可以直接在RDS的provisioner里用psql命令执行SQL，省掉provider配置：

resource "aws_db_instance" "app_db" {
  # 你的RDS配置...

  provisioner "local-exec" {
    command = <<EOF
aws ssm start-session \
  --target ${aws_instance.ssm_bastion.id} \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["${self.address}"], "portNumber":["5432"], "localPortNumber":["54320"]}' &
sleep 10
# 创建数据库和角色
psql -h localhost -p 54320 -U ${self.username} -d postgres -c "CREATE DATABASE app_database;"
psql -h localhost -p 54320 -U ${self.username} -d postgres -c "CREATE ROLE db_role WITH LOGIN PASSWORD '${random_string.postgres_pass.result}';"
# 关闭隧道
pkill -f "aws ssm start-session --target ${aws_instance.ssm_bastion.id}"
EOF
    environment = {
      PGPASSWORD = self.password
    }
  }
}

二、解决私有端点EKS的应用部署问题 #

私有端点的EKS，本地Terraform的kubernetes/helm provider无法直接连接集群，推荐用SSM代理+IAM认证的方案，完全不需要公网通路：

具体实现步骤 #

1. 生成带SSM代理配置的kubeconfig：

resource "null_resource" "generate_eks_kubeconfig" {
  triggers = {
    cluster_name = module.eks.cluster_name
    region       = var.region
  }

  provisioner "local-exec" {
    command = <<EOF
# 生成基础kubeconfig
aws eks update-kubeconfig --name ${module.eks.cluster_name} --region ${var.region} --kubeconfig ./eks-kubeconfig
# 添加SSM代理配置，让kubectl通过SSM访问集群
sed -i.bak 's/server: https:\/\/.*/&
    proxy-command: aws ssm start-session --target %n --document-name AWS-StartPortForwardingSessionToRemoteHost --parameters '\''{"host":["%h"], "portNumber":["%p"]}'\''/' ./eks-kubeconfig
EOF
  }
}

2. 配置kubernetes和helm provider使用这个kubeconfig：

provider "kubernetes" {
  config_path = "./eks-kubeconfig"
  depends_on  = [null_resource.generate_eks_kubeconfig]
}

provider "helm" {
  kubernetes {
    config_path = "./eks-kubeconfig"
  }
  depends_on = [null_resource.generate_eks_kubeconfig]
}

# 现在可以正常部署Helm Chart了
resource "helm_release" "my_app" {
  name       = "my-app"
  repository = "https://charts.example.com/"
  chart      = "my-app"
  version    = "1.0.0"

  values = [
    yamlencode({
      database = {
        host     = module.rds.db_instance_address
        port     = module.rds.db_instance_port
        username = postgresql_role.db_role.name
        password = random_string.postgres_pass.result
        dbname   = postgresql_database.env.name
      }
    })
  ]
}

额外优化：如果你的Terraform执行角色（本地AWS用户或CI/CD角色）有EKS访问权限，可以直接用IAM认证获取凭证，连kubeconfig都省了：

provider "kubernetes" {
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)

  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args = [
      "eks", "get-token",
      "--cluster-name", module.eks.cluster_name,
      "--region", var.region,
      "--role-arn", "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/eks-admin-role"
    ]
  }
}

这个方案需要你在EKS的aws-auth ConfigMap里添加该IAM角色的映射，赋予集群管理员权限。

关键注意事项 #

• RDS的安全组要允许Bastion实例的安全组访问5432端口，否则隧道通不了。
• 所有SSM操作都依赖VPC的SSM端点，记得提前配置，不然得给Bastion开公网IP，安全性下降。
• Terraform销毁时会自动关闭SSM会话，不会残留进程。

内容的提问来源于stack exchange，提问作者saraf.gahl