通过Terraform从外部文件为Azure DevOps构建管道配置构建变量的方案咨询

Got it, let's work through this problem! Since you need to reuse your Terraform code across different pipeline templates and can't rely on variable groups, here are two practical approaches to load pipeline variables from external files while keeping your code generic.

Approach 1: External YAML/JSON File + Dynamic Blocks #

This method lets you store all your pipeline variables in a separate YAML (or JSON) file, which you can swap out for different pipeline templates without modifying your core Terraform code.

Step 1: Create an external variables file #

First, make a pipeline-variables.yaml file in your Terraform module directory. Split regular and secret variables to keep things organized:

regular_variables:
  - name: "PipelineVariable"
    value: "Go Microsoft!"
  - name: "AnotherRegularVar"
    value: "Hello Terraform!"

secret_variables:
  - name: "PipelineSecret"
    secret_value: "ZGV2cw"
  - name: "DbPassword"
    secret_value: "cGFzc3dvcmQ="

Step 2: Update your Terraform code #

Use Terraform's yamldecode and file functions to load the external file, then use dynamic blocks to generate the variable entries automatically:

# Load variables from the external YAML file
locals {
  pipeline_vars = yamldecode(file("${path.module}/pipeline-variables.yaml"))
}

resource "azuredevops_build_definition" "build" {
  project_id = azuredevops_project.project.id
  name       = "Sample Build Definition"
  path       = "\\ExampleFolder"

  ci_trigger {
    use_yaml = true
  }

  # Generate regular variables dynamically
  dynamic "variable" {
    for_each = local.pipeline_vars.regular_variables
    content {
      name  = variable.value.name
      value = variable.value.value
    }
  }

  # Generate secret variables dynamically
  dynamic "variable" {
    for_each = local.pipeline_vars.secret_variables
    content {
      name         = variable.value.name
      secret_value = variable.value.secret_value
      is_secret    = true
    }
  }
}

Why this works: #

• The dynamic "variable" block loops through each entry in your external file and creates the corresponding Terraform resource block.
• You can create multiple versions of pipeline-variables.yaml (e.g., dev-pipeline-vars.yaml, prod-pipeline-vars.yaml) and switch between them by updating the file path in the locals block, or using a Terraform variable to reference the file name.

Approach 2: Terraform tfvars Files #

If you prefer using Terraform's native variable system, you can define variable lists in a .tfvars file and reference them in your build definition.

Step 1: Define variable schemas #

Create a variables.tf file to define the structure for your pipeline variables:

variable "pipeline_regular_variables" {
  type = list(object({
    name  = string
    value = string
  }))
  description = "List of non-secret pipeline variables"
  default     = []
}

variable "pipeline_secret_variables" {
  type = list(object({
    name         = string
    secret_value = string
  }))
  description = "List of secret pipeline variables"
  default     = []
  sensitive   = true # Mark the entire list as sensitive to avoid logs leaking secrets
}

Step 2: Populate variables in a tfvars file #

Create a pipeline-config.tfvars file with your variable values:

pipeline_regular_variables = [
  {
    name  = "PipelineVariable"
    value = "Go Microsoft!"
  },
  {
    name  = "BuildNumberPrefix"
    value = "v1.0."
  }
]

pipeline_secret_variables = [
  {
    name         = "PipelineSecret"
    secret_value = "ZGV2cw"
  }
]

Step 3: Update your build definition resource #

Use dynamic blocks to iterate over the variables:

resource "azuredevops_build_definition" "build" {
  project_id = azuredevops_project.project.id
  name       = "Sample Build Definition"
  path       = "\\ExampleFolder"

  ci_trigger {
    use_yaml = true
  }

  dynamic "variable" {
    for_each = var.pipeline_regular_variables
    content {
      name  = variable.value.name
      value = variable.value.value
    }
  }

  dynamic "variable" {
    for_each = var.pipeline_secret_variables
    content {
      name         = variable.value.name
      secret_value = variable.value.secret_value
      is_secret    = true
    }
  }
}

To use different configurations: #

Just create separate tfvars files (e.g., dev-config.tfvars, prod-config.tfvars) and apply them with:

terraform apply -var-file=dev-config.tfvars

Important Notes for Secret Variables #

• Never commit plaintext secrets to version control. Instead, use environment variables to pass secret values to your tfvars file. For example:

  # In your tfvars file
  pipeline_secret_variables = [
    {
      name         = "PipelineSecret"
      secret_value = "${var.pipeline_secret_value}"
    }
  ]
  

  Then run Terraform with:

  TF_VAR_pipeline_secret_value="your-actual-secret" terraform apply
  

• Mark secret variable lists as sensitive in your variables.tf to prevent Terraform from logging their values.

内容的提问来源于stack exchange，提问作者Devops-Learner