Terraform部署Beanstalk环境失败：镜像构建与RDS连接问题排查

报错信息 #

执行terraform apply时返回如下错误：

Error: creating Elastic Beanstalk Environment (app-dev): waiting for completion: 3 errors occurred:
│       * 2023-02-19 18:36:21.159 +0000 UTC (e-jnwgwcdy7q) : Instance deployment failed. For details, see 'eb-engine.log'.
│       * 2023-02-19 18:36:21.159 +0000 UTC (e-jnwgwcdy7q) : Instance deployment failed to build the Docker image. The deployment failed.
│       * 2023-02-19 18:36:24.538 +0000 UTC (e-jnwgwcdy7q) : [Instance: i-0feeb08c4da8ff1b5] Command failed on instance. Return code: 1 Output: Engine execution has encountered an error..

环境说明 #

自定义Docker镜像包含Django Rest API应用，需安装Python依赖并向独立部署的RDS实例执行数据库迁移，API通过环境变量获取数据库连接信息。应用版本通过S3存储的包含docker-compose.yaml的压缩包部署。

Terraform配置 #

Elastic Beanstalk相关配置 #

resource "aws_elastic_beanstalk_application" "beanstalk_app" {
  name        = var.project_ref
  description = "Beanstalk application for Django App"
}

resource "aws_elastic_beanstalk_environment" "beanstalk_env" {
  name                = var.project_ref
  application         = aws_elastic_beanstalk_application.beanstalk_app.name
  solution_stack_name = "64bit Amazon Linux 2 v3.5.4 running Docker"
  tier                = "WebServer"
  version_label       = aws_elastic_beanstalk_application_version.beanstalk_app_version.id
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "IamInstanceProfile"
    value = aws_iam_instance_profile.ec2_eb_profile.name
  }

  # Env vars
  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "DB_PORT"
    value     = var.project_port
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "DB_NAME"
    value     = var.project_name
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "DB_USER"
    value     = var.project_db_user
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "DB_PASSWORD"
    value     = var.project_db_pwd
  }

  setting {
    namespace = "aws:elasticbeanstalk:application:environment"
    name      = "DJANDO_ADMIN_PWD"
    value     = var.project_admin_pwd
  }

  # Logs
  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "StreamLogs"
    value     = "true"
  }

  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "RetentionInDays"
    value     = "1"
  }

  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "DeleteOnTerminate"
    value     = "false"
  }

  # Logs health
  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs:health"
    name      = "HealthStreamingEnabled"
    value     = "true"
  }

  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs:health"
    name      = "DeleteOnTerminate"
    value     = "true"
  }

  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs:health"
    name      = "RetentionInDays"
    value     = "1"
  }

  # Tags
  tags = {
    Name = var.project_ref
    Environment = var.project_env
  }
}
  
resource "aws_elastic_beanstalk_application_version" "beanstalk_app_version" {
  name        = var.project_ref
  description = "application version created by terraform"
  application = aws_elastic_beanstalk_application.beanstalk_app.id
  bucket      = aws_s3_bucket.s3_bucket_back.id
  key         = aws_s3_object.s3_bucket_back_zip.id
}

resource "aws_iam_instance_profile" "ec2_eb_profile" {
  name = "beanstalk-service-user"
  role = "${aws_iam_role.ec2_eb_role.name}"
  tags = {
    Name = var.project_ref
    Environment = var.project_env
  }
}

resource "aws_iam_role" "ec2_eb_role" {
  name = "beanstalk-service-role"
  assume_role_policy = data.aws_iam_policy_document.instance_assume_role_policy.json
}

RDS实例配置 #

resource "aws_db_instance" "db_instance_project" {
  allocated_storage    = 10
  db_name              = var.project_name
  engine               = "postgres"
  engine_version       = "13.7"
  instance_class       = "db.t3.micro"
  username             = var.project_db_user
  password             = var.project_db_pwd
  skip_final_snapshot  = true
  vpc_security_group_ids = [aws_default_security_group.security_group.id]
  publicly_accessible  = true
}

疑问与更新 #

1. 调试方法：eb-engine.log和CloudWatch日志为空，如何查看Docker构建或部署的详细日志？
2. RDS连接：已将RDS设为公开访问，但仍无法确认是否是连接问题导致部署失败。

2023年2月20日更新：尝试在Beanstalk环境中添加RDS关联配置，但无法设置DB_NAME，故改用Terraform独立创建RDS，但不知如何在Beanstalk中传递RDS主机地址：

setting {
    namespace = "aws:rds:dbinstance"
    name      = "HasCoupleDatabase"
    value     = false
}
setting {
    namespace = "aws:rds:dbinstance"
    name      = "DBEngine"
    value     = "postgres"
}
setting {
    namespace = "aws:rds:dbinstance"
    name      = "DBEngineVersion"
    value     = "13.7"
}
setting {
    namespace = "aws:rds:dbinstance"
    name      = "DBInstanceClass"
    value     = "db.t3.micro"
}
setting {
    namespace = "aws:rds:dbinstance"
    name      = "DBUser"
    value     = var.project_db_user
}
setting {
    namespace = "aws:rds:dbinstance"
    name      = "DBPassword"
    value     = var.project_db_pwd
}

排查思路与解决方案 #

一、获取详细调试日志 #

1. 直接登录EC2实例查看
   • 找到Beanstalk环境对应的EC2实例，通过SSH登录
   • 查看核心日志文件：
     • /var/log/eb-engine.log：Beanstalk引擎执行日志，包含Docker构建过程
     • /var/log/eb-hooks.log：部署钩子脚本执行日志
     • /var/log/docker：Docker服务日志
     • 用docker logs <container-id>查看容器日志（如果容器曾启动过）
2. 使用EB CLI拉取日志
   • 安装EB CLI后，执行eb logs --all，可拉取环境下所有实例的完整日志，包括本地未同步到CloudWatch的内容
3. 本地验证应用包
   • 下载S3中的压缩包，本地解压后测试docker-compose build和docker-compose up，确认镜像构建、容器启动是否正常，排除应用包本身的问题

二、解决RDS连接与部署问题 #

1. 补充DB_HOST环境变量
   在Beanstalk环境配置中添加RDS端点的环境变量，通过Terraform动态引用：

   setting {
     namespace = "aws:elasticbeanstalk:application:environment"
     name      = "DB_HOST"
     value     = aws_db_instance.db_instance_project.endpoint
   }
   

   这样Django应用就能通过DB_HOST环境变量获取RDS地址
2. 避免混用Beanstalk内置RDS与独立RDS
   移除aws:rds:dbinstance命名空间的所有配置，因为你已经用Terraform独立创建了RDS，Beanstalk的内置RDS配置会造成冲突
3. 验证RDS连接可用性
   • 在EC2实例上安装PostgreSQL客户端：sudo amazon-linux-extras install postgresql13
   • 用环境变量中的参数测试连接：psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME
   • 确认安全组配置：确保RDS的安全组允许Beanstalk EC2实例的IP或安全组ID访问5432端口；默认安全组默认允许同一VPC内的流量，若EC2和RDS在同一VPC，可直接放行
4. 调整数据库迁移时机
   不要在Docker镜像构建阶段执行数据库迁移（此时RDS可能未就绪，且构建阶段无网络访问权限），改为：
   • 在Docker容器的entrypoint脚本中，先等待RDS就绪（可使用wait-for-it工具或自定义脚本），再执行python manage.py migrate，最后启动服务
   • 或使用Beanstalk的部署钩子：在应用包中添加.platform/hooks/postdeploy/01_migrate.sh脚本，内容如下：

     #!/bin/bash
     source /opt/python/current/env
     cd /opt/python/current/app
     python manage.py migrate --noinput
     

     给脚本添加执行权限：chmod +x .platform/hooks/postdeploy/01_migrate.sh

内容的提问来源于stack exchange，提问作者Gati