AWS Glue Crawler未生成数据表求助（附Terraform配置）

问题背景 #

运行AWS Glue Crawler后未生成任何数据表，CloudWatch日志显示爬虫执行完成但无异常提示，现附上相关Terraform配置、启动命令及日志信息，请求协助排查原因。

Terraform配置 #

resource "aws_iam_role" "cu_test_glue_service_role" {
  name = "AWSGlueServiceRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Action = "sts:AssumeRole",
      Principal = {
        Service = "glue.amazonaws.com"
      },
      Effect = "Allow",
    }]
  })
}

resource "aws_iam_policy_attachment" "cu_test_glue_service_policy" {
  name       = "glue-service-role-policy"
  roles      = [aws_iam_role.cu_test_glue_service_role.name]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}
resource "aws_iam_policy" "cu_test_glue_s3_access_policy" {
  name        = "cu-test-glue-s3-access-policy"
  description = "Grants Glue service access to S3 buckets"

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Action = [
            "s3:*",
            "s3-object-lambda:*"
        ],
        Resource = [
          "arn:aws:s3:::cur-report-eu-west-1-xxxxxxxxxx",
          "arn:aws:s3:::cur-report-eu-west-1-xxxxxxxxxx/*"
        ]
      }
    ]
  })
}
resource "aws_iam_role_policy_attachment" "cu_test_glue_s3_access_policy_attachment" {
  policy_arn = aws_iam_policy.cu_test_glue_s3_access_policy.arn
  role       = aws_iam_role.cu_test_glue_service_role.name
}
resource "aws_glue_catalog_database" "cu_test_db" {
  name = "cu_test_data_db"
}

resource "aws_glue_crawler" "my_crawler" {
  name         = "my_cur_data_crawler_test"
  role         = aws_iam_role.cu_test_glue_service_role.arn
  database_name = aws_glue_catalog_database.cu_test_db.name

  s3_target {
    path = "s3://cur-report-eu-west-1-xxxxxxxxxx/test/report_test/cost_and_usage_data_status/"
  }

  schema_change_policy {
    delete_behavior = "LOG"
  }
}

爬虫启动命令 #

aws glue start-crawler --name "my_cur_data_crawler_test"

CloudWatch日志 #

BENCHMARK : Running Start Crawl for Crawler my_cur_data_crawler_test
BENCHMARK : Classification complete, writing results to database cu_test_data_db
INFO : Crawler configured with SchemaChangePolicy {"UpdateBehavior":"UPDATE_IN_DATABASE","DeleteBehavior":"LOG"}.
BENCHMARK : Finished writing to Catalog
BENCHMARK : Crawler has finished running and is in state READY

排查方向及解决方案 #

• 检查S3路径有效性
  确认目标S3路径下存在Glue支持的数据文件（如CSV、Parquet、JSON等），空目录或不兼容格式的文件不会触发表生成；同时检查文件ACL是否允许Glue服务读取。
• 验证文件格式与分类器
  若文件为自定义格式（如特殊分隔符的CSV），需手动添加对应分类器；检查文件是否损坏，损坏文件会导致Crawler无法解析结构。
• 确认数据库与区域匹配
  确保在正确的AWS区域（eu-west-1）查看cu_test_data_db数据库，避免跨区域或数据库名称混淆。
• 校验IAM权限细节
  通过IAM策略模拟器验证角色是否拥有glue:CreateTable、glue:UpdateTable等Catalog操作权限；同时检查S3桶策略是否存在Deny规则阻止Glue访问。
• 查看详细日志
  在CloudWatch中查找Crawler的完整日志流，排查是否存在未在BENCHMARK日志中显示的警告或错误信息。

内容的提问来源于stack exchange，提问作者Olfa2