使用Terraform创建Databricks元数据存储时遇无效JSON错误

我正按照Databricks官方文档，使用Terraform创建Azure上的Databricks元数据存储，尝试了连接配置文件、token、指定配置文件路径等多种认证方式，但执行时仍报错：

Error: cannot create metastore: Invalid JSON received (1040 bytes):
<!doctype html>Databricks - Sign
inYou need to enable JavaScript
to run this app.

我的Terraform代码如下：

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.74.0"
    }
    databricks = {
      source  = "databricks/databricks"
    }
  }
}
provider "databricks" {
   profile = var.databricks_connection_profile (DEFAULT conn profile was valid)
   host = data.azurerm_databricks_workspace.dbwdata.workspace_url
   #token= "*****" (I tried this)
   #config_file = <path of .databrickscfg>  (& this as well)
}

resource "databricks_metastore" "this" {
       provider = databricks.accounts
       name     = "mymeta"
       storage_root = format("abfss://%s@%s.dfs.core.windows.net/sandbox",
         "container",
         "samplestorage")
       force_destroy = true
       region        = var.azure_region
       depends_on = [
         module.databricks
        ]
       }

问题原因 #

返回登录页面HTML而非API响应，说明请求发往了Databricks工作区URL，但databricks_metastore是账户级资源，必须使用Azure Databricks账户的全局URL（格式为https://accounts.azuredatabricks.net），且需要配置对应的账户级Provider实例。

解决步骤 #

1. 新增账户级Databricks Provider配置，指定账户级URL和合法的账户级认证方式
2. 确保databricks_metastore资源正确关联账户级Provider

修改后的代码示例：

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.74.0"
    }
    databricks = {
      source  = "databricks/databricks"
      version = ">=1.20.0" # 建议指定兼容版本
    }
  }
}

# 工作区级Provider（用于操作工作区内资源，如需要）
provider "databricks" {
  host = data.azurerm_databricks_workspace.dbwdata.workspace_url
  # 可使用工作区token或配置文件认证
  # token = var.databricks_workspace_token
  # config_file = "~/.databrickscfg"
}

# 账户级Provider（专门用于创建元数据存储等账户级资源）
provider "databricks" {
  alias    = "accounts"
  host     = "https://accounts.azuredatabricks.net" # 固定账户级URL
  # 使用Azure服务主体认证（推荐）
  auth_type           = "azure_service_principal"
  azure_client_id     = var.azure_client_id
  azure_client_secret = var.azure_client_secret
  azure_tenant_id     = var.azure_tenant_id
}

resource "databricks_metastore" "this" {
  provider       = databricks.accounts
  name           = "mymeta"
  storage_root   = format("abfss://%s@%s.dfs.core.windows.net/sandbox", "container", "samplestorage")
  force_destroy  = true
  region         = var.azure_region
  depends_on     = [module.databricks]
}

关键注意点 #

• 账户级Provider的host必须是https://accounts.azuredatabricks.net，不能用单个工作区的URL
• 用于认证的Azure服务主体必须拥有Databricks账户的管理员权限，否则无法创建元数据存储
• 如果使用配置文件认证，需在.databrickscfg中添加账户级的profile，指定host = https://accounts.azuredatabricks.net和对应的认证信息

内容的提问来源于stack exchange，提问作者Aniket Karajgikar