如何用Terraform配置Databricks Token ACL？求替代临时令牌方案

在使用Terraform配置Databricks Token ACL时需要用到令牌，我当前的方案是在初次部署时创建一个有效期为1天的临时令牌，但该方案被认为不够规范、缺乏可持续性，因此需要寻找替代方案。

当前使用的Python脚本如下：

import requests
import json

DATABRICKS_INSTANCE = "https://XXXXXXXXXXXXXX.azuredatabricks.net" 
DATABRICKS_TOKEN = "dapiXXXXXXXXXXXXXXXXXXX"

def create_dummy_token():
    url = f"{DATABRICKS_INSTANCE}/api/2.0/token/create"
    headers = {
        "Authorization": f"Bearer {DATABRICKS_TOKEN}",
        "Content-Type": "application/json"
    }
    payload = {
        "comment": "Dummy token",
        "lifetime_seconds": 86400  # 1 day
    }

    response = requests.post(url, headers=headers, data=json.dumps(payload))
    if response.status_code == 200:
        token_info = response.json()
        with open("dummy_token.json", "w") as token_file:
            json.dump({"token_value": token_info["token_value"]}, token_file)
        print("Dummy token created successfully.")
    else:
        print(f"Failed to create token: {response.text}")

if __name__ == "__main__":
    create_dummy_token()

在main.tf中调用该脚本的方式如下：

// Create a dummy token using an external data source
provider "local" {}

resource "null_resource" "create_dummy_token" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "python3 databricks_dummy_token.py"
  }
}

data "external" "dummy_token" {
  depends_on = [null_resource.create_dummy_token]
  program    = ["bash", "-c", "cat dummy_token.json"]
}

output "dummy_token_result" {
  value = data.external.dummy_token.result
}

output "dummy_token_new" {
  value = data.external.dummy_token.result["token_value"]
}

方案一：使用官方Databricks Terraform Provider内置认证 #

无需手动生成临时令牌，直接通过Provider支持的标准化认证方式完成Token ACL配置，这是最规范且可持续的方案：

1. Azure环境下用服务主体认证 #

针对Azure Databricks，直接绑定Azure服务主体凭据到Databricks Provider，无需依赖个人令牌：

provider "azurerm" {
  features {}
}

provider "databricks" {
  azure_workspace_resource_id = azurerm_databricks_workspace.example.id
  azure_client_id             = var.azure_client_id
  azure_client_secret         = var.azure_client_secret
  azure_tenant_id             = var.azure_tenant_id
}

# 示例：配置Token ACL
resource "databricks_token_permission" "example" {
  token_id         = databricks_token.example.id
  permission_level = "CAN_USE"
  principal_id     = databricks_group.example.id
}

2. 创建专用Databricks服务主体令牌 #

生成长期有效的服务主体令牌（可按安全要求设置有效期），专门用于Terraform操作：

# 创建Databricks服务主体
resource "databricks_service_principal" "terraform_sp" {
  display_name = "terraform-service-principal"
}

# 为服务主体生成令牌
resource "databricks_token" "terraform_token" {
  comment                = "Token for Terraform operations"
  lifetime_seconds       = 31536000 # 1年，可按需调整
  service_principal_id   = databricks_service_principal.terraform_sp.id
}

# 绑定令牌到Provider
provider "databricks" {
  host  = var.databricks_host
  token = databricks_token.terraform_token.token_value
}

方案二：优化现有临时令牌方案 #

如果必须保留临时令牌模式，可从以下维度提升规范性：

• 移除硬编码敏感信息：将脚本中的DATABRICKS_INSTANCE和DATABRICKS_TOKEN改为从环境变量读取，示例：

  DATABRICKS_INSTANCE = os.getenv("DATABRICKS_INSTANCE")
  DATABRICKS_TOKEN = os.getenv("DATABRICKS_ADMIN_TOKEN")
  

• 控制令牌生成频率：删除null_resource的always_run = timestamp()触发器，改为绑定Workspace资源ID，仅在初次部署时生成：

  resource "null_resource" "create_dummy_token" {
    triggers = {
      workspace_id = azurerm_databricks_workspace.example.id
    }
  
    provisioner "local-exec" {
      command = "python3 databricks_dummy_token.py"
    }
  }
  

• 自动清理临时令牌：添加销毁阶段脚本，在Terraform销毁时自动删除令牌：

  resource "null_resource" "create_dummy_token" {
    # 省略其他配置...
  
    provisioner "local-exec" {
      when        = destroy
      command     = "python3 databricks_delete_token.py ${data.external.dummy_token.result["token_value"]}"
    }
  }
  

  对应的删除脚本（databricks_delete_token.py）：

  import requests
  import json
  import os
  import sys
  
  DATABRICKS_INSTANCE = os.getenv("DATABRICKS_INSTANCE")
  DATABRICKS_TOKEN = os.getenv("DATABRICKS_ADMIN_TOKEN")
  token_to_delete = sys.argv[1]
  
  def delete_token():
      url = f"{DATABRICKS_INSTANCE}/api/2.0/token/delete"
      headers = {
          "Authorization": f"Bearer {DATABRICKS_TOKEN}",
          "Content-Type": "application/json"
      }
      payload = {"token_value": token_to_delete}
      response = requests.post(url, headers=headers, data=json.dumps(payload))
      if response.status_code == 200:
          print("Dummy token deleted successfully.")
      else:
          print(f"Failed to delete token: {response.text}")
  
  if __name__ == "__main__":
      delete_token()
  

• 隐藏敏感输出：移除output "dummy_token_result"和output "dummy_token_new"，避免令牌泄露到控制台。

方案三：用Databricks CLI替代自定义脚本 #

利用官方Databricks CLI完成令牌管理，Terraform调用CLI命令更符合官方规范：

resource "null_resource" "create_dummy_token" {
  triggers = {
    workspace_id = azurerm_databricks_workspace.example.id
  }

  provisioner "local-exec" {
    command = <<EOT
      databricks configure --token --host ${var.databricks_host} --token ${var.databricks_admin_token}
      databricks tokens create --comment "Terraform temp token" --lifetime-seconds 86400 > dummy_token.json
    EOT
  }
}

内容的提问来源于stack exchange，提问作者Sumit Bhardwaj