Azure中无法创建新数据收集终结点及配置规则，求指导

一、现有已完成操作 #

我已通过以下Pipeline命令创建了Log Analytics工作区和自定义表：

az monitor log-analytics workspace create --resource-group $(ResourceGroupName) --workspace-name $(LogAnalyticsWorkspaceName)
az monitor log-analytics workspace table create --resource-group $(ResourceGroupName) \
            --workspace-name $(LogAnalyticsWorkspaceName) -n $(LogAnalyticsGWCustomTableName)  \
            --columns RawData=string TimeGenerated=datetime LogLevel=string --retention-time 90

注：上述命令中的变量已在环境文件中定义。

之后尝试通过以下命令创建数据收集规则（DCR），但当前使用的是现有数据收集终结点（DCE），希望新建一个专属DCE并完成关联配置：

az monitor data-collection rule create --resource-group $(ResourceGroupName) --location $(Location) --name 'gw-d-dcr' --rule-file '/resourcegroup-deployments/custom-loganalytics/customlogsgatewayvm-$(parameters.environment).parameters.json'

对应的原有DCR规则文件内容如下：

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
    "dataCollectionRules_gw_d_dcr_name": {
        "defaultValue": "gw-d-dcr",
        "type": "String"
    },
    "dataCollectionEndpoints_cre_gw_d_dce_externalid": {
        "defaultValue": "/subscriptions/ea273087-6293-4fc2-bfe7-aa29ea8ab4bf/resourceGroups/lz-cre-d-rg/providers/Microsoft.Insights/dataCollectionEndpoints/cre-gw-d-dce",
        "type": "String"
    },
    "workspaces_cre_d_law_externalid": {
        "defaultValue": "/subscriptions/ea273087-6293-4fc2-bfe7-aa29ea8ab4bf/resourceGroups/lz-cre-d-rg/providers/Microsoft.OperationalInsights/workspaces/cre-d-law",
        "type": "String"
    }
},
"variables": {},
"resources": [
    {
        "type": "Microsoft.Insights/dataCollectionRules",
        "apiVersion": "2023-03-11",
        "name": "[parameters('dataCollectionRules_gw_d_dcr_name')]",
        "location": "westeurope",
        "tags": {
            "AppName": "CRE",
            "Billing code": "NL02264",
            "Business Application CI": "CI0030939",
            "CIA": "112",
            "ContactMail": "laurens.de.vries@nl.abnamro.com",
            "ContactPhone": "+31624536181",
            "Environment": "Development",
            "Owner": "laurens.de.vries@nl.abnamro.com",
            "Provider": "CBSP Azure"
        },
        "kind": "Linux",
        "properties": {
            "dataCollectionEndpointId": "[parameters('dataCollectionEndpoints_cre_gw_d_dce_externalid')]",
            "streamDeclarations": {
                "Custom-Text-gw_customlogs_CL": {
                    "columns": [
                        {
                            "name": "TimeGenerated",
                            "type": "datetime"
                        },
                        {
                            "name": "RawData",
                            "type": "string"
                        }
                    ]
                }
            },
            "dataSources": {
                "logFiles": [
                    {
                        "streams": [
                            "Custom-Text-gw_customlogs_CL"
                        ],
                        "filePatterns": [
                            "/data/web/cre/LogFiles/ACBS/*.log",
                            "/data/web/cre/LogFiles/ACBS/*.err",
                            "/data/web/cre/LogFiles/CRAS/*.log",
                            "/data/web/cre/LogFiles/CRAS/*.err",
                            "/data/web/cre/LogFiles/RAPID/*.log",
                            "/data/web/cre/LogFiles/RAPID/*.err",
                            "/home/ctrmazure/software/ctm/dailylog/daily*"
                        ],
                        "format": "text",
                        "settings": {
                            "text": {
                                "recordStartTimestampFormat": "ISO 8601"
                            }
                        },
                        "name": "Custom-Text-gw_customlogs_CL"
                    }
                ]
            },
            "destinations": {
                "logAnalytics": [
                    {
                        "workspaceResourceId": "[parameters('workspaces_cre_d_law_externalid')]",
                        "name": "la--771413343"
                    }
                ]
            },
            "dataFlows": [
                {
                    "streams": [
                        "Custom-Text-gw_customlogs_CL"
                    ],
                    "destinations": [
                        "la--771413343"
                    ],
                    "transformKql": "source",
                    "outputStream": "Custom-gw_customlogs_CL"
                }
            ]
        }
    }
] }

二、新建数据收集终结点（DCE）并关联DCR的完整配置步骤 #

1. 新建数据收集终结点（DCE） #

提供两种创建方式，可根据需求选择：

方式一：Azure CLI命令创建DCE

直接通过CLI快速创建，后续手动关联DCR：

az monitor data-collection endpoint create \
  --resource-group $(ResourceGroupName) \
  --location $(Location) \
  --name 'new-gw-d-dce' \
  --tags AppName="CRE" Environment="Development" Owner="laurens.de.vries@nl.abnamro.com"

注：替换new-gw-d-dce为自定义DCE名称，标签可根据实际需求调整。

创建完成后，获取DCE的资源ID用于后续关联：

az monitor data-collection endpoint show \
  --resource-group $(ResourceGroupName) \
  --name 'new-gw-d-dce' \
  --query 'id' -o tsv

方式二：在原有ARM模板中添加DCE资源

修改原DCR规则文件，将DCE资源加入，实现DCE与DCR的一键部署：

{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
    "dataCollectionRules_gw_d_dcr_name": {
        "defaultValue": "gw-d-dcr",
        "type": "String"
    },
    "dataCollectionEndpoints_new_gw_d_dce_name": {
        "defaultValue": "new-gw-d-dce",
        "type": "String"
    },
    "workspaces_cre_d_law_externalid": {
        "defaultValue": "/subscriptions/ea273087-6293-4fc2-bfe7-aa29ea8ab4bf/resourceGroups/lz-cre-d-rg/providers/Microsoft.OperationalInsights/workspaces/cre-d-law",
        "type": "String"
    }
},
"variables": {},
"resources": [
    // 新增数据收集终结点资源
    {
        "type": "Microsoft.Insights/dataCollectionEndpoints",
        "apiVersion": "2023-03-11",
        "name": "[parameters('dataCollectionEndpoints_new_gw_d_dce_name')]",
        "location": "westeurope",
        "tags": {
            "AppName": "CRE",
            "Billing code": "NL02264",
            "Business Application CI": "CI0030939",
            "CIA": "112",
            "ContactMail": "laurens.de.vries@nl.abnamro.com",
            "ContactPhone": "+31624536181",
            "Environment": "Development",
            "Owner": "laurens.de.vries@nl.abnamro.com",
            "Provider": "CBSP Azure"
        },
        "properties": {
            "networkAcls": {
                "publicNetworkAccess": "Enabled" // 根据需求设置为Enabled或Disabled，禁用需配置专用链接
            }
        }
    },
    // 修改原有DCR资源，关联新建的DCE
    {
        "type": "Microsoft.Insights/dataCollectionRules",
        "apiVersion": "2023-03-11",
        "name": "[parameters('dataCollectionRules_gw_d_dcr_name')]",
        "location": "westeurope",
        "tags": {
            "AppName": "CRE",
            "Billing code": "NL02264",
            "Business Application CI": "CI0030939",
            "CIA": "112",
            "ContactMail": "laurens.de.vries@nl.abnamro.com",
            "ContactPhone": "+31624536181",
            "Environment": "Development",
            "Owner": "laurens.de.vries@nl.abnamro.com",
            "Provider": "CBSP Azure"
        },
        "kind": "Linux",
        "properties": {
            "dataCollectionEndpointId": "[resourceId('Microsoft.Insights/dataCollectionEndpoints', parameters('dataCollectionEndpoints_new_gw_d_dce_name'))]",
            "streamDeclarations": {
                "Custom-Text-gw_customlogs_CL": {
                    "columns": [
                        {
                            "name": "TimeGenerated",
                            "type": "datetime"
                        },
                        {
                            "name": "RawData",
                            "type": "string"
                        }
                    ]
                }
            },
            "dataSources": {
                "logFiles": [
                    {
                        "streams": [
                            "Custom-Text-gw_customlogs_CL"
                        ],
                        "filePatterns": [
                            "/data/web/cre/LogFiles/ACBS/*.log",
                            "/data/web/cre/LogFiles/ACBS/*.err",
                            "/data/web/cre/LogFiles/CRAS/*.log",
                            "/data/web/cre/LogFiles/CRAS/*.err",
                            "/data/web/cre/LogFiles/RAPID/*.log",
                            "/data/web/cre/LogFiles/RAPID/*.err",
                            "/home/ctrmazure/software/ctm/dailylog/daily*"
                        ],
                        "format": "text",
                        "settings": {
                            "text": {
                                "recordStartTimestampFormat": "ISO 8601"
                            }
                        },
                        "name": "Custom-Text-gw_customlogs_CL"
                    }
                ]
            },
            "destinations": {
                "logAnalytics": [
                    {
                        "workspaceResourceId": "[parameters('workspaces_cre_d_law_externalid')]",
                        "name": "la--771413343"
                    }
                ]
            },
            "dataFlows": [
                {
                    "streams": [
                        "Custom-Text-gw_customlogs_CL"
                    ],
                    "destinations": [
                        "la--771413343"
                    ],
                    "transformKql": "source",
                    "outputStream": "Custom-gw_customlogs_CL"
                }
            ]
        },
        "dependsOn": [
            "[resourceId('Microsoft.Insights/dataCollectionEndpoints', parameters('dataCollectionEndpoints_new_gw_d_dce_name'))]"
        ]
    }
] }

2. 部署修改后的配置 #

• 若使用CLI创建DCE，执行以下命令更新DCR关联：

az monitor data-collection rule update \
  --resource-group $(ResourceGroupName) \
  --name 'gw-d-dcr' \
  --data-collection-endpoint-id <新建DCE的资源ID>

• 若使用修改后的ARM模板，直接执行原有部署命令即可：

az monitor data-collection rule create --resource-group $(ResourceGroupName) --location $(Location) --name 'gw-d-dcr' --rule-file '/resourcegroup-deployments/custom-loganalytics/customlogsgatewayvm-$(parameters.environment).parameters.json'

3. 验证配置 #

• 登录Azure门户，确认新建的DCE已创建成功
• 检查DCR的「数据收集终结点」字段已关联新建的DCE
• 等待数分钟后，查看Log Analytics自定义表中是否有日志数据流入

内容的提问来源于stack exchange，提问作者Aviator