Azure HDInsight Kafka Terraform脚本：指定Worker数与禁用公网IP

问题描述 #

使用Terraform创建Azure HDInsight Kafka集群时，尝试在azurerm_hdinsight_kafka_cluster根块中添加以下参数来设置5个Kafka Worker节点并禁用公网IP：

worker_node_count   = 5
enable_gateway      = false

执行terraform plan时触发如下错误：

on hdinsight.tf line 44, in resource "azurerm_hdinsight_kafka_cluster" "kafka_cluster":
│   44:   worker_node_count   = 5
│ 
│ An argument named "worker_node_count" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on hdinsight.tf line 45, in resource "azurerm_hdinsight_kafka_cluster" "kafka_cluster":
│   45:   enable_gateway      = false
│ 
│ An argument named "enable_gateway" is not expected here.

解决方案 #

1. 配置Kafka Worker（Broker）节点数量 #

HDInsight Kafka集群的Worker节点数量需在roles下的worker_node块中通过target_instance_count参数设置，而非根块的worker_node_count。将该参数值改为5即可实现5个Broker节点的需求：

roles {
  # ... 其他角色配置 ...
  worker_node {
    virtual_network_id = azurerm_virtual_network.virtual_network.id
    subnet_id = azurerm_subnet.subnet.id
    vm_size                  = "Standard_D3_V2"
    username                 = "my-username"
    password                 = "my-password"
    number_of_disks_per_node = 3
    target_instance_count    = 5  # 修改此处为5
  }
  # ... 其他角色配置 ...
}

2. 禁用集群公网IP #

要关闭集群的公网网关（禁用公网IP），直接删除根块中的gateway配置块即可。因为当未配置gateway块时，HDInsight集群的公网网关默认处于禁用状态，无需使用enable_gateway参数。

修改后的完整配置示例 #

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "rg" {
  name     = "my-resource-group"
  location = "eastus"
}

resource "azurerm_virtual_network" "virtual_network" {
    resource_group_name = azurerm_resource_group.rg.name
    name = "my-vnet"
    location = "eastus"
    address_space = ["10.136.82.0/24"]
}

resource "azurerm_subnet" "subnet" {
  name                 = "subnet-3"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes     = ["10.136.82.64/27"]
}

resource "azurerm_storage_account" "storage_account" {
  name                     = "my-storage-account"
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_storage_container" "storage_container" {
  name                  = "hdinsight"
  storage_account_name  = azurerm_storage_account.storage_account.name
  container_access_type = "private"
}

resource "azurerm_hdinsight_kafka_cluster" "kafka_cluster" {
  name                = "my-hdicluster"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  cluster_version     = "4.0"
  tier                = "Standard"

  component_version {
    kafka = "2.4"
  }

  storage_account {
    storage_container_id = azurerm_storage_container.storage_container.id
    storage_account_key  = azurerm_storage_account.storage_account.primary_access_key
    is_default           = true
  }

  roles {
    head_node {
      virtual_network_id = azurerm_virtual_network.virtual_network.id
      subnet_id = azurerm_subnet.subnet.id
      vm_size  = "Standard_D3_V2"
      username = "my-username"
      password = "my-password"
    }

    worker_node {
      virtual_network_id = azurerm_virtual_network.virtual_network.id
      subnet_id = azurerm_subnet.subnet.id
      vm_size                  = "Standard_D3_V2"
      username                 = "my-username"
      password                 = "my-password"
      number_of_disks_per_node = 3
      target_instance_count    = 5  # 配置5个Worker节点
    }

    zookeeper_node {
      virtual_network_id = azurerm_virtual_network.virtual_network.id
      subnet_id = azurerm_subnet.subnet.id
      vm_size  = "Standard_D3_V2"
      username = "my-username"
      password = "my-password"
    }
  }
}

额外提示：原配置中azurerm_subnet的resource_group_name使用了硬编码值，改为引用azurerm_resource_group.rg.name可提升配置的一致性和可维护性。

内容的提问来源于stack exchange，提问作者Eugene Goldberg