Terraform创建ADF源数据集报JSON解析无效字符'Â'错误

我尝试使用Terraform创建指向Azure Data Lake Storage Gen2（ADLS Gen2）中指定文件的Azure Data Factory（ADF）源数据集，编写的Terraform配置代码如下：

初始资源组配置 #

resource "azurerm_resource_group" "myresourcegroup" {
    name     = "${var.applicationName}-${var.environment}-rg"
    location = var.location
    tags = {
        environment = var.environment
    }
}

初始ADLS Gen2 链接服务配置 #

resource "azurerm_data_factory_linked_custom_service" "storagels" {
  name                 = "${var.applicationName}-${var.environment}-storagels"
  data_factory_id      = azurerm_data_factory.adf.id
  type                 = "AzureBlobFS"
  description          = "storage connection linked service"
  integration_runtime {
    name = azurerm_data_factory_integration_runtime_self_hosted.ir.name
  }
  type_properties_json = <<JSON
  {
        "url": "https://cdfstagesrg.dfs.core.windows.net",
            "accountKey": {
                "type": "AzureKeyVaultSecret",
                "store": {
                    "referenceName": "${azurerm_data_factory_linked_service_key_vault.akvls.name}",
                    "type": "LinkedServiceReference"
                },
                "secretName": "storageaccesskey"
            }
        },
       "connectVia":{
      "referenceName": "${azurerm_data_factory_integration_runtime_self_hosted.ir.name}",
      "type": "IntegrationRuntimeReference"
       }    
JSON

  depends_on = [azurerm_data_factory_integration_runtime_self_hosted.ir]
}

初始数据集创建配置 #

resource "azurerm_data_factory_custom_dataset" "datalakelookup" {
  name            = "${var.dataset_name}-datalakelkp"
  data_factory_id = azurerm_data_factory.adf.id
  type            = "LinkedServiceReference"

  linked_service {
    name = "${var.applicationName}-${var.environment}-storagels"
    
  }
  type_properties_json = <<JSON
  {
    "properties": {
        "linkedServiceName": {
            "referenceName": "${var.applicationName}-${var.environment}-storagels",
            "type": "LinkedServiceReference"
        },
        "annotations": [],
        "type": "Excel",
        "typeProperties": {
            "sheetName": "Datalake",
            "location": {
                "type": "AzureBlobFSLocation",
                "fileName": "tablesnames.xlsx",
                "folderPath": "caseact/Input",
                "fileSystem": "cdfstage"
            },
            "firstRowAsHeader": true
        },
        "schema": []
    }
}
JSON

    depends_on = [azurerm_resource_group.myresourcegroup,azurerm_data_factory.adf,azurerm_data_factory_integration_runtime_self_hosted.ir,azurerm_data_factory_linked_custom_service.clihive-linked-service]
}

初始报错情况 #

除数据集外的其余资源均可正常创建，执行数据集创建操作时返回如下错误：

Error parsing JSON: invalid character 'Â' looking for beginning of value.

问题更新 #

使用jsonencode函数处理JSON配置内容后，出现其他不同报错，报错截图如下：

• invalid character 'Â'报错由两类问题触发：一是Terraform配置文件保存为带BOM的UTF-8编码，或heredoc块内混入了不可见特殊字符、全角空格；二是手写JSON存在语法错误，比如链接服务配置中accountKey字段闭合后多余的逗号。
• 数据集配置存在结构错误：azurerm_data_factory_custom_dataset资源的type_properties_json字段不需要额外包裹外层properties节点，该字段传入的内容会自动映射到API要求的properties结构下，多嵌套一层会触发结构校验失败；同时资源的type字段应填数据集实际类型Excel，而非LinkedServiceReference。
• 数据集配置的depends_on依赖项错误，引用了不存在的clihive-linked-service资源，会导致资源创建顺序异常。
• 手动拼接heredoc JSON极易出现语法、编码、转义问题，统一使用Terraform内置jsonencode函数构造JSON内容可从根源避免此类问题。

修复后数据集配置 #

resource "azurerm_data_factory_custom_dataset" "datalakelookup" {
  name            = "${var.dataset_name}-datalakelkp"
  data_factory_id = azurerm_data_factory.adf.id
  type            = "Excel"

  linked_service {
    name = azurerm_data_factory_linked_custom_service.storagels.name
  }

  type_properties_json = jsonencode({
    linkedServiceName = {
      referenceName = azurerm_data_factory_linked_custom_service.storagels.name
      type          = "LinkedServiceReference"
    }
    annotations = []
    type        = "Excel"
    typeProperties = {
      sheetName        = "Datalake"
      firstRowAsHeader = true
      location = {
        type       = "AzureBlobFSLocation"
        fileName   = "tablesnames.xlsx"
        folderPath = "caseact/Input"
        fileSystem = "cdfstage"
      }
    }
    schema = []
  })

  depends_on = [
    azurerm_resource_group.myresourcegroup,
    azurerm_data_factory.adf,
    azurerm_data_factory_integration_runtime_self_hosted.ir,
    azurerm_data_factory_linked_custom_service.storagels
  ]
}

额外优化建议 #

链接服务的type_properties_json也替换为jsonencode写法，避免手写JSON的语法错误，示例如下：

resource "azurerm_data_factory_linked_custom_service" "storagels" {
  name                 = "${var.applicationName}-${var.environment}-storagels"
  data_factory_id      = azurerm_data_factory.adf.id
  type                 = "AzureBlobFS"
  description          = "storage connection linked service"
  integration_runtime {
    name = azurerm_data_factory_integration_runtime_self_hosted.ir.name
  }
  type_properties_json = jsonencode({
    url = "https://cdfstagesrg.dfs.core.windows.net"
    accountKey = {
      type = "AzureKeyVaultSecret"
      store = {
        referenceName = azurerm_data_factory_linked_service_key_vault.akvls.name
        type          = "LinkedServiceReference"
      }
      secretName = "storageaccesskey"
    }
  })

  depends_on = [azurerm_data_factory_integration_runtime_self_hosted.ir]
}

操作前将所有Terraform配置文件重新保存为无BOM的UTF-8编码，清除代码中残留的不可见控制字符。

内容的提问来源于stack exchange，提问作者venkat