Terraform读取.tfvars中BigQuery聚类配置失效问题排查

数据老陈

2026-7-25

问题

我尝试通过Terraform从.tfvars文件读取配置，为BigQuery表添加聚类功能，但执行terraform plan时，系统返回No changes. Your infrastructure matches the configuration.，完全忽略了daily_sales表中的clustering = ["store_identifier", "product_identifier"]配置；但如果直接在模块中设置clustering字段，Terraform就能正常识别变更并正确添加列聚类。请问问题出在哪？

相关配置如下：

main.tf:

resource "google_bigquery_dataset" "database" {
  for_each                   = { for k, v in var.dbs : k => v }
  dataset_id                 = each.value.db_id
  friendly_name              = each.value.db_name
  description                = "Dataset created by Terraform"
  location                   = "US"
  delete_contents_on_destroy = var.delete_contents_on_destroy
  labels                     = each.value.labels
}

module "tables" {
  source   = "./tables"
  for_each = { for k, v in var.dbs : k => v }
  db_id    = google_bigquery_dataset.database[each.key].dataset_id
  labels   = each.value.labels
  tables   = each.value.tables
}

variables.tf:

variable "dbs" {
  type = list(object({
    db_id   = string
    db_name = string
    labels  = map(string)
    tables = list(object({
      id          = string
      partitioned = string
      field       = string
      type        = string
      clustering  = list(string)
    }))

  }))
}

./tables/tables.tf (模块):

resource "google_bigquery_table" "tables" {
  for_each = {
    for m in var.tables : "${m.id}" => m
  }
  dataset_id          = var.db_id
  deletion_protection = false
  table_id            = each.value.id
  clustering = try(each.value.clustering, [])

  dynamic "time_partitioning" {
    for_each = each.value.partitioned == "true" ? [1] : []
    content {
      type  = each.value.type
      field = each.value.field
    }
  }

  labels = var.labels
  schema = fileexists("${path.module}/${var.db_id}/${each.value.id}.json") ? file("${path.module}/${var.db_id}/${each.value.id}.json") : null
}

dev.tfvars:

dbs = [
  {
    db_id   = "some_db"
    db_name = "some_db"
    labels = {
      environment = "development"
    }
    tables = [
      {
        id          = "daily_inventory"
        partitioned = "true"
        field       = "inventory_timestamp"
        type        = "DAY"
      },
      {
        id          = "daily_sales"
        partitioned = "true"
        field       = "sales_timestamp"
        type        = "DAY"
        clustering = ["store_identifier", "product_identifier"]
      },
      {
        id          = "stores"
        partitioned = "false"
        field       = ""
        type        = ""
      }
]

解决方案

问题出在变量定义、for_each逻辑和配置语法三个方面，具体修复步骤如下：

给clustering字段添加默认值
在variables.tf的表对象定义中，clustering字段未设置默认值，导致未配置该字段的表会缺失此属性，Terraform类型系统会判定对象结构不一致，进而忽略变更。

修改variables.tf：

variable "dbs" {
  type = list(object({
    db_id   = string
    db_name = string
    labels  = map(string)
    tables = list(object({
      id          = string
      partitioned = string
      field       = string
      type        = string
      clustering  = list(string, default = []) # 添加默认空列表
    }))
  }))
}

优化模块内的for_each逻辑
当前模块用整个表对象作为for_each的值，对象属性变化会干扰Terraform对实例标识的判断。改为仅用表ID作为唯一键，通过匹配ID获取配置：

首先补充模块的变量定义（./tables/variables.tf）：

variable "tables" {
  type = list(object({
    id          = string
    partitioned = string
    field       = string
    type        = string
    clustering  = list(string)
  }))
}

然后修改./tables/tables.tf：

resource "google_bigquery_table" "tables" {
  for_each = { for m in var.tables : m.id => m.id } # 仅用表ID作为键
  dataset_id          = var.db_id
  deletion_protection = false
  table_id            = each.value

  # 通过表ID匹配对应配置
  clustering = [for t in var.tables : t.clustering if t.id == each.value][0]

  dynamic "time_partitioning" {
    for_each = [for t in var.tables : t if t.id == each.value && t.partitioned == "true"]
    content {
      type  = time_partitioning.value.type
      field = time_partitioning.value.field
    }
  }

  labels = var.labels
  schema = fileexists("${path.module}/${var.db_id}/${each.value}.json") ? file("${path.module}/${var.db_id}/${each.value}.json") : null
}

修复dev.tfvars的语法错误
原配置中缺少tables数组和外层db对象的闭合括号，导致Terraform无法正确解析配置：

修改dev.tfvars：

dbs = [
  {
    db_id   = "some_db"
    db_name = "some_db"
    labels = {
      environment = "development"
    }
    tables = [
      {
        id          = "daily_inventory"
        partitioned = "true"
        field       = "inventory_timestamp"
        type        = "DAY"
      },
      {
        id          = "daily_sales"
        partitioned = "true"
        field       = "sales_timestamp"
        type        = "DAY"
        clustering = ["store_identifier", "product_identifier"]
      },
      {
        id          = "stores"
        partitioned = "false"
        field       = ""
        type        = ""
      }
    ] # 补充tables数组闭合括号
  } # 补充db对象闭合括号
]

完成以上修改后，先执行terraform refresh刷新状态，再运行terraform plan就能看到聚类配置的变更了。

内容的提问来源于stack exchange，提问作者MassHysteria