Azure消费计划下DrainMode引发ServiceBus监听TaskCanceledException日志泛滥

我们在Azure Functions消费计划中运行多个ServiceBus监听器，当平台释放资源时会触发DrainMode，此时所有批量监听器会输出大量无意义的TaskCanceledException错误日志，占用AppInsights存储空间。

已尝试但无效的方案：

• 为所有ServiceBus函数添加try/catch捕获异常
• 按照微软文档及社区方案处理CancellationToken

现寻求无需修改日志级别的解决方案。

错误堆栈信息：

[Error]   sbt-alarms/Subscriptions/sbs-alarms-log-41ea7d07-9a99-4ffe-9de9-e91713d7d6d7: ReceiveBatchAsync Exception: System.Threading.Tasks.TaskCanceledException: A task was canceled.

   at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.Azure.Amqp.ReceivingAmqpLink.EndReceiveMessages(IAsyncResult result, IEnumerable`1& messages)

   at Microsoft.Azure.Amqp.ReceivingAmqpLink.<>c.<ReceiveMessagesAsync>b__27_1(IAsyncResult r)

   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)

--- End of stack trace from previous location ---

   at Azure.Messaging.ServiceBus.Amqp.AmqpReceiver.ReceiveMessagesAsyncInternal(Int32 maxMessages, Nullable`1 maxWaitTime, TimeSpan timeout, CancellationToken cancellationToken)

   at Azure.Messaging.ServiceBus.Amqp.AmqpReceiver.<>c.<<ReceiveMessagesAsync>b__41_0>d.MoveNext()

--- End of stack trace from previous location ---

   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.RunOperation[T1,TResult](Func`4 operation, T1 t1, TransportConnectionScope scope, CancellationToken cancellationToken, Boolean logRetriesAsVerbose)

   at Azure.Messaging.ServiceBus.ServiceBusRetryPolicy.RunOperation[T1,TResult](Func`4 operation, T1 t1, TransportConnectionScope scope, CancellationToken cancellationToken, Boolean logRetriesAsVerbose)

   at Azure.Messaging.ServiceBus.Amqp.AmqpReceiver.ReceiveMessagesAsync(Int32 maxMessages, Nullable`1 maxWaitTime, CancellationToken cancellationToken)

   at Azure.Messaging.ServiceBus.ServiceBusReceiver.ReceiveMessagesAsync(Int32 maxMessages, Nullable`1 maxWaitTime, Boolean isProcessor, CancellationToken cancellationToken).

可行解决方案 #

1. AppInsights自定义日志过滤规则 #

直接在AppInsights层面过滤这类无意义异常：

• 进入目标AppInsights资源，打开“日志”页面
• 创建并保存过滤规则，针对exceptions表排除符合以下特征的记录：

  exceptions
  | where type == "System.Threading.Tasks.TaskCanceledException"
  | where stacktrace contains "ReceiveBatchAsync"
  | where stacktrace contains "Azure.Messaging.ServiceBus.Amqp.AmqpReceiver.ReceiveMessagesAsync"
  

• 将此规则设为默认过滤，避免这类日志占用存储空间和干扰错误统计。

2. 自定义日志拦截器（基于Serilog） #

通过第三方日志框架在日志输出前拦截过滤：

• 配置Serilog作为Functions的日志提供方，添加异常过滤逻辑：

  Log.Logger = new LoggerConfiguration()
      .Filter.ByExcluding(evt => 
          evt.Level == LogEventLevel.Error &&
          evt.Exception is TaskCanceledException &&
          evt.MessageTemplate.Text.Contains("ReceiveBatchAsync Exception")
      )
      .WriteTo.ApplicationInsights(TelemetryConfiguration.CreateDefault(), TelemetryConverter.Traces)
      .CreateLogger();
  

• 在Program.cs中注册Serilog替代默认日志系统：

  var host = new HostBuilder()
      .UseSerilog()
      .ConfigureFunctionsWorkerDefaults()
      .Build();
  host.Run();
  

3. 调整ServiceBus触发器的DrainMode参数 #

通过缩短Drain等待时间减少异常触发次数：

• 在host.json中修改ServiceBus扩展配置：

  {
    "version": "2.0",
    "extensions": {
      "serviceBus": {
        "messageHandlerOptions": {
          "drainMode": true,
          "waitTimeForMessagesToComplete": "00:00:30"
        }
      }
    }
  }
  

  缩短waitTimeForMessagesToComplete可以让Drain过程更快结束，降低因超时引发的取消异常频率。

内容的提问来源于stack exchange，提问作者Kars