Azure Data Factory每日Dataverse至本地SQL Server同步偶发失败求助

我们有一个每日凌晨3:00执行的Azure Data Factory管道，用于将Dataverse（Dynamics CRM）的数据复制到本地SQL Server。该管道部分日期运行正常，部分日期执行失败，报错信息如下：

Type=Microsoft.PowerPlatform.Dataverse.Client.Utils.DataverseConnectionException,
Message=Failed to connect to Dataverse,
Source=Microsoft.PowerPlatform.Dataverse.Client,''
Type=System.Exception,
Message=ExternalTokenManagement Authentication Requested but not configured correctly.
Source=Microsoft.PowerPlatform.Dataverse.Client,''
Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,
Message=An error occurred while sending the request.
Source=Microsoft.DataTransfer.Common,''
Type=System.Net.Http.HttpRequestException,
Message=An error occurred while sending the request.
Source=mscorlib,'' Type=System.Net.WebException,
Message=The underlying connection was closed: An unexpected error occurred on a send.
Source=System,''  Type=System.IO.IOException,
Message=Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.,
Source=System,''   Type=System.Net.Sockets.SocketException,
Message=An existing connection was forcibly closed by the remote host

单独测试Dataverse和SQL Server的连接均正常，仅管道执行时出现问题。已将巴西南部区域Azure Data Factory的IP地址段添加至SQL Server所在服务器的防火墙，但问题仍未解决：

{
      "name": "DataFactory.BrazilSouth",
      "id": "DataFactory.BrazilSouth",
      "properties": {
        "changeNumber": 6,
        "region": "brazilsouth",
        "regionId": 9,
        "platform": "Azure",
        "systemService": "DataFactory",
        "addressPrefixes": [
          "4.203.145.160/27",
          "20.206.179.144/29",
          "191.233.205.160/28",
          "191.234.137.32/29",
          "191.234.142.64/26",
          "191.234.143.0/24",
          "191.234.149.0/28",
          "191.234.157.0/28",
          "191.235.224.128/25",
          "191.235.225.0/26",
          "2603:1050:6:1::480/121",
          "2603:1050:6:1::500/122",
          "2603:1050:6:1::700/121",
          "2603:1050:6:1::780/122",
          "2603:1050:6:402::330/124",
          "2603:1050:6:802::210/124",
          "2603:1050:6:c02::210/124"
        ],
        "networkFeatures": [
          "API",
          "NSG"
        ]
      }
    }

排查建议 #

• 认证配置核查：报错核心提示ExternalTokenManagement Authentication Requested but not configured correctly，需重点检查：
  • 若使用服务主体认证，确认客户端密钥是否过期、是否拥有Dataverse的必要权限（如系统管理员、数据读取权限）
  • 排查租户是否启用了条件访问策略或MFA，导致服务主体在凌晨时段的请求被拦截
• 网络稳定性排查：连接被远程主机强制关闭可能与时段性网络问题有关：
  • 检查Dataverse租户的流量限制，确认凌晨时段是否有其他批量任务占用资源触发限流
  • 核实本地SQL Server到Azure的网络链路，是否存在凌晨时段的带宽波动、防火墙临时规则变更
  • 若使用自托管集成运行时，检查其凌晨时段是否有重启、资源不足（CPU/内存过高）情况
• IP段有效性确认：Azure服务IP会定期更新，需确认当前添加的巴西南部ADF IP段是否为最新版本
• 超时设置调整：检查管道复制活动的超时配置，凌晨时段数据量可能增大，适当延长超时时间避免连接被强制关闭
• 日志深度分析：开启ADF详细日志，提取失败时段的请求ID，结合Dataverse审计日志，定位请求被拒绝的具体原因

内容的提问来源于stack exchange，提问作者Julián Meneses