如何为Azure Data Factory实现灾难恢复？求含Terraform示例的参考资料

核心灾备策略 #

ADF的灾难恢复核心是保障元数据一致性和依赖资源的跨区域可用性，常用方案如下：

• 元数据同步：通过Git集成将所有ADF资源（管道、数据集、链接服务等）的元数据存储在Git仓库，主/灾备区域的ADF实例共享同一仓库，确保配置一致。
• 依赖资源灾备：对存储账户、SQL数据库等关联资源，在灾备区域部署副本并配置数据同步（如存储账户GRS异地复制、SQL主动异地副本）。
• 触发机制切换：灾备时将触发器切换到灾备区域的ADF实例，或通过Terraform批量启用/禁用主备触发器。

Terraform实现示例 #

1. 部署主/灾备区域ADF实例 #

# 主区域资源组
resource "azurerm_resource_group" "rg_primary" {
  name     = "rg-adf-primary"
  location = "East US"
}

# 灾备区域资源组
resource "azurerm_resource_group" "rg_secondary" {
  name     = "rg-adf-secondary"
  location = "West US"
}

# 主区域ADF实例
resource "azurerm_data_factory" "primary" {
  name                = "adf-primary-eastus"
  location            = azurerm_resource_group.rg_primary.location
  resource_group_name = azurerm_resource_group.rg_primary.name
}

# 灾备区域ADF实例
resource "azurerm_data_factory" "secondary" {
  name                = "adf-secondary-westus"
  location            = azurerm_resource_group.rg_secondary.location
  resource_group_name = azurerm_resource_group.rg_secondary.name
}

2. 配置Git集成（主备共享元数据） #

# 主ADF Git绑定
resource "azurerm_data_factory_git_configuration" "primary" {
  data_factory_id           = azurerm_data_factory.primary.id
  repository_url            = "https://dev.azure.com/your-org/your-project/_git/adf-metadata-repo"
  branch_name               = "main"
  root_folder               = "/"
  collaboration_branch_name = "main"
}

# 灾备ADF Git绑定（同步同一仓库）
resource "azurerm_data_factory_git_configuration" "secondary" {
  data_factory_id           = azurerm_data_factory.secondary.id
  repository_url            = "https://dev.azure.com/your-org/your-project/_git/adf-metadata-repo"
  branch_name               = "main"
  root_folder               = "/"
  collaboration_branch_name = "main"
}

3. 依赖资源灾备（以存储账户为例） #

# 主区域存储账户（启用异地冗余复制）
resource "azurerm_storage_account" "primary" {
  name                     = "storprimaryeastus001"
  resource_group_name      = azurerm_resource_group.rg_primary.name
  location                 = azurerm_resource_group.rg_primary.location
  account_tier             = "Standard"
  account_replication_type = "GRS" # 自动同步数据到配对灾备区域
}

# 灾备区域存储账户（本地冗余，用于独立访问同步后的数据）
resource "azurerm_storage_account" "secondary" {
  name                     = "storsecondarywestus001"
  resource_group_name      = azurerm_resource_group.rg_secondary.name
  location                 = azurerm_resource_group.rg_secondary.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

4. 灾备切换触发器配置 #

# 主区域触发器（正常状态启用）
resource "azurerm_data_factory_trigger_schedule" "primary_daily" {
  name                = "trigger-daily-ingest"
  data_factory_id     = azurerm_data_factory.primary.id
  frequency           = "Day"
  interval            = 1
  start_time          = "2024-01-01T00:00:00Z"
  end_time            = "2099-12-31T23:59:59Z"
  enabled             = true

  pipeline {
    name = azurerm_data_factory_pipeline.ingest_pipeline.name
  }
}

# 灾备区域触发器（灾备时启用）
resource "azurerm_data_factory_trigger_schedule" "secondary_daily" {
  name                = "trigger-daily-ingest"
  data_factory_id     = azurerm_data_factory.secondary.id
  frequency           = "Day"
  interval            = 1
  start_time          = "2024-01-01T00:00:00Z"
  end_time            = "2099-12-31T23:59:59Z"
  enabled             = false

  pipeline {
    name = azurerm_data_factory_pipeline.ingest_pipeline.name
  }
}

关键注意事项 #

• 元数据一致性：所有ADF变更必须通过Git提交，禁止直接在ADF实例上修改，避免主备配置差异。
• RPO/RTO匹配：根据业务需求选择依赖资源的同步策略（如SQL主动复制实现低RPO，存储GRS实现异步同步）。
• 定期测试：通过Terraform执行切换演练，验证灾备实例的资源可用性和业务流程完整性。

内容的提问来源于stack exchange，提问作者One Developer