Terragrunt中Lakehouse模块的Provider配置覆盖问题求助

问题背景 #

当前Terragrunt目录结构如下：

├── _common
│	└── instance_template.hcl
├── prod
│	├── asia-south1
│	│	├── applications
│	│	│	├── ce-ping
│	│	│	│	├── instance
│	│	│	│	│	└── terragrunt.hcl
│	│	│	│	└── template
│	│	│	│		└── terragrunt.hcl
│	│	│	└── lakehouse
│	│	│		├── providers.hcl
│	│		│	└── terragrunt.hcl
│	│	├── infra
│	│	│	├── vpc
│	│	│	│	└── terragrunt.hcl
│	│	│	├── vpc-firewall
│	│	│	│	└── terragrunt.hcl
│	│	│	├── vpc-subnet
│	│	│	└── vpn
│	│	│		└── terragrunt.hcl
│	│	└── region.hcl
│	├── env.hcl
│	└── project.hcl
├── providers.hcl
└── terragrunt.hcl

核心问题：lakehouse模块需要使用旧版本的GCP Provider，但当前配置无法覆盖根Terragrunt生成的Provider设置。

根terragrunt.hcl内容 #

locals {
  # Automatically load account-level variables
  project_local = abspath("${get_terragrunt_dir()}/project.hcl")
  project_parent = find_in_parent_folders("project.hcl", "/safe-placeholder")
  project_path = fileexists(local.project_local) ? local.project_local : local.project_parent
  project_vars = read_terragrunt_config(local.project_path)
  
  # Automatically load region-level variables
  region_local = abspath("${get_terragrunt_dir()}/region.hcl")
  region_parent = find_in_parent_folders("region.hcl", "/safe-placeholder")
  region_path = fileexists(local.region_local) ? local.region_local : local.region_parent
  region_vars = read_terragrunt_config(local.region_path)

  # Automatically load environment-level variables
  environment_local = abspath("${get_terragrunt_dir()}/env.hcl")
  environment_parent = find_in_parent_folders("env.hcl", "/safe-placeholder")
  environment_path = fileexists(local.environment_local) ? local.environment_local : local.environment_parent
  environment_vars = read_terragrunt_config(local.environment_path)
  
  
  # Extract the variables we need for easy access
  project_id  = local.project_vars.locals.project_id
  gcp_region  = local.region_vars.locals.gcp_region
  environment = local.environment_vars.locals.environment
}

# Generate a GCP provider block
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "google" {
  region = "${local.gcp_region}"
  project = "${local.project_id}"
}
EOF
}
# Configure Terragrunt to automatically store tfstate files in a GCS bucket

remote_state {
  backend = "gcs"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }

  config = {
    project  = local.project_id 
    location = local.gcp_region 
    bucket   = "tg-state-${local.project_id}" 
    prefix   = "state/${path_relative_to_include()}/terraform.tfstate" 
  }
}

# Configure root level variables that all resources can inherit. This is especially helpful with multi-account configs
# where terraform_remote_state data sources are placed directly into the modules.
inputs = merge(
  local.project_vars.locals,
  local.region_vars.locals,
  local.environment_vars.locals,
)

lakehouse模块的terragrunt.hcl内容 #

locals {
  # Automatically load environment-level variables
  environment_vars = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  region_vars      = read_terragrunt_config(find_in_parent_folders("region.hcl"))
  project_vars     = read_terragrunt_config(find_in_parent_folders("project.hcl"))

  # Extract out common variables for reuse
  env        = local.environment_vars.locals.environment
  gcp_region = local.region_vars.locals.gcp_region
  project_id = local.project_vars.locals.project_id
}

# Terragrunt will copy the Terraform configurations specified by the source parameter, along with any files in the
# working directory, into a temporary folder, and execute your Terraform commands in that folder.
terraform {
  source = "tfr:///GoogleCloudPlatform/analytics-lakehouse/google?version=0.4.0"

  extra_arguments "provider_override" {
    commands = ["init", "apply", "plan"]
    arguments = [
      "-var-file=${get_terragrunt_dir()}/providers.hcl"
    ]
  }  
}

# Include all settings from the root terragrunt.hcl file
include {
  path = find_in_parent_folders()
}

# These are the variables we have to pass in to use the module specified in the terragrunt configuration above
inputs = {
  enable_apis         = true
  project_id          = local.project_id
  region              = local.gcp_region
  deletion_protection = false
}

lakehouse模块的providers.hcl内容 #

terraform {
  required_version = "~>1.9.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">=5.9.0, <6.0.0"
    }
  }
}

解决方案 #

你当前的错误在于用-var-file加载providers.hcl，但这个文件是Terraform的核心配置文件（包含required_providers），不是变量文件，所以无法生效。同时根Terragrunt的generate "provider"会自动生成provider.tf，可能和模块的Provider配置冲突，需要按以下步骤调整：

1. 调整lakehouse模块的terragrunt.hcl，禁用根的Provider生成 #

在lakehouse的terragrunt.hcl中，添加generate "provider"块并设置skip = true，阻止根配置生成的provider.tf覆盖模块的Provider设置：

locals {
  # 保持原有locals内容不变
  environment_vars = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  region_vars      = read_terragrunt_config(find_in_parent_folders("region.hcl"))
  project_vars     = read_terragrunt_config(find_in_parent_folders("project.hcl"))

  env        = local.environment_vars.locals.environment
  gcp_region = local.region_vars.locals.gcp_region
  project_id = local.project_vars.locals.project_id
}

# 禁用根配置的Provider生成
generate "provider" {
  skip = true
}

terraform {
  source = "tfr:///GoogleCloudPlatform/analytics-lakehouse/google?version=0.4.0"
  # 移除无效的extra_arguments配置
}

include {
  path = find_in_parent_folders()
}

inputs = {
  enable_apis         = true
  project_id          = local.project_id
  region              = local.gcp_region
  deletion_protection = false
}

2. 确保providers.hcl被Terragrunt复制到临时工作目录 #

Terragrunt默认会复制当前目录下的所有文件到临时工作目录，所以只要providers.hcl和terragrunt.hcl在同一目录，就会被自动复制。如果模块本身没有包含Provider配置，这个文件会生效。

3. （可选）在lakehouse模块中直接定义Provider配置 #

如果模块没有自带Provider配置，也可以直接在terragrunt.hcl中用generate块生成包含版本限制的Provider文件：

generate "provider" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
terraform {
  required_version = "~>1.9.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">=5.9.0, <6.0.0"
    }
  }
}

provider "google" {
  region = "${local.gcp_region}"
  project = "${local.project_id}"
}
EOF
}

这样就不需要单独的providers.hcl文件，直接在Terragrunt中生成所需的Provider配置。

4. 执行初始化命令验证 #

进入lakehouse模块目录，执行terragrunt init，检查输出是否显示正确加载了指定版本的Google Provider。如果之前有旧的状态，可能需要先执行terragrunt init -upgrade来更新Provider版本。

内容的提问来源于stack exchange，提问作者Grous