You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

ASP.NET 6中OpenTelemetry OTLP Exporter内存访问异常求助

问题描述

我有一个ASP.NET 6应用,已集成OpenTelemetry并配置OTLP Exporter,配置代码如下:

services.AddOpenTelemetryTracing(tracerProviderBuilder =>
        {
            tracerProviderBuilder
                .AddOtlpExporter(opt =>
                {
                    opt.Protocol = OtlpExportProtocol.Grpc;
                    opt.BatchExportProcessorOptions = new BatchExportProcessorOptions<Activity>
                    {
                        ////MaxExportBatchSize = 16 // Set a low batch size
                    };
                })
                .AddSource(serviceName)
                .AddSource("MySqlConnector")        // Or remove this
                .SetResourceBuilder(
                    ResourceBuilder.CreateDefault()
                        .AddService(serviceName: serviceName, serviceVersion: serviceVersion)
                        .AddDetector(new AWSEC2ResourceDetector()))
                .AddAWSInstrumentation()
                .AddHttpClientInstrumentation()
                .AddAspNetCoreInstrumentation()
                .AddRedisInstrumentation();
        });

使用的依赖包及版本:

<PackageReference Include="OpenTelemetry" Version="1.4.0-alpha.2" />
<PackageReference Include="OpenTelemetry.Contrib.Extensions.AWSXRay" Version="1.2.0" />
<PackageReference Include="OpenTelemetry.Contrib.Instrumentation.AWS" Version="1.0.1" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.3.0" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.0.0-rc9.6" />
<PackageReference Include="OpenTelemetry.Instrumentation.StackExchangeRedis" Version="1.0.0-rc9.7" />

运行集成测试时触发致命异常:

Exit code is -1073741819 (Fatal error. System.AccessViolationException: Attempted to 
read or write protected memory. This is often an indication that other memory is 
corrupt.
   at System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
   at OpenTelemetry.Internal.TagTransformer`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryTransformTag(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>, System.__Canon ByRef)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState.ForEach(System.Collections.Generic.KeyValuePair`2<System.String,System.Object>)
   at DynamicClass.AllocationFreeForEachDelegate(System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.String,System.Object>>, TagEnumerationState ByRef, ForEachDelegate<System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.String,System.Object>>,System.Collections.Generic.KeyValuePair`2<System.String,System.Object>,TagEnumerationState>)
   at OpenTelemetry.Trace.ActivityHelperExtensions+ActivityTagsEnumeratorFactory`1[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]].Enumerate(System.Diagnostics.ActivityEvent, TagEnumerationState ByRef)
   at OpenTelemetry.Trace.ActivityHelperExtensions.EnumerateTags[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+TagEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]](System.Diagnostics.ActivityEvent, TagEnumerationState ByRef)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions.ToOtlpEvent(System.Diagnostics.ActivityEvent)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+EventEnumerationState.ForEach(System.Diagnostics.ActivityEvent)
   at OpenTelemetry.Trace.ActivityHelperExtensions+ActivityEventsEnumeratorFactory`1[[OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions+EventEnumerationState, OpenTelemetry.Exporter.OpenTelemetryProtocol, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7bd6737fe5b67e3c]].ForEachEventCallback(EventEnumerationState ByRef, System.Diagnostics.ActivityEvent)
   at DynamicClass.AllocationFreeForEachDelegate(System.Collections.Generic.IEnumerable`1<System.Diagnostics.ActivityEvent>, EventEnumerationState ByRef, ForEachDelegate<System.Collections.Generic.IEnumerable`1<System.Diagnostics.ActivityEvent>,System.Diagnostics.ActivityEvent,EventEnumerationState>)
   at OpenTelemetry.Exporter.OpenTelemetryProtocol.Implementation.ActivityExtensions.AddBatch(Opentelemetry.Proto.Collector.Trace.V1.ExportTraceServiceRequest, Opentelemetry.Proto.Resource.V1.Resource, OpenTelemetry.Batch`1<System.Diagnostics.Activity> ByRef)
   at OpenTelemetry.Exporter.OtlpTraceExporter.Export(OpenTelemetry.Batch`1<System.Diagnostics.Activity> ByRef)
   at OpenTelemetry.BatchExportProcessor`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ExporterProc()
   at System.Threading.Thread.StartCallback())

已知以下操作可临时规避问题:

  • 将MaxExportBatchSize设置为较小值(如16,默认512)
  • 移除"MySqlConnector"数据源(推测因减少了追踪导出数量)
解决方案

1. 统一OpenTelemetry依赖版本

当前依赖包版本混杂,既有Alpha版核心包,又有不同版本的工具包,版本不匹配是引发内存访问异常的核心原因之一。将所有OpenTelemetry相关包升级到稳定且统一的版本,示例配置如下:

<PackageReference Include="OpenTelemetry" Version="1.5.1" />
<PackageReference Include="OpenTelemetry.Contrib.Extensions.AWSXRay" Version="1.3.1" />
<PackageReference Include="OpenTelemetry.Contrib.Instrumentation.AWS" Version="1.1.0" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.5.1" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.5.1" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.5.1" />
<PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.5.1" />
<PackageReference Include="OpenTelemetry.Instrumentation.StackExchangeRedis" Version="1.5.1" />

2. 修复批量导出选项的初始化方式

原代码直接新建BatchExportProcessorOptions<Activity>实例的方式可能存在内存管理问题,建议通过配置委托修改默认选项,而非完全替换:

.AddOtlpExporter(opt =>
{
    opt.Protocol = OtlpExportProtocol.Grpc;
    // 修改默认批量导出参数,无需新建实例
    opt.BatchExportProcessorOptions.MaxExportBatchSize = 64; // 可根据实际流量调整,无需过小
    opt.BatchExportProcessorOptions.MaxQueueSize = 2048;
})

3. 过滤MySqlConnector追踪的异常标签

异常发生在标签转换阶段,推测MySqlConnector生成的追踪标签中存在特殊类型或未正确释放的对象。可通过自定义处理器过滤无效标签:

// 注册自定义标签过滤处理器
services.AddOpenTelemetryTracing(tracerProviderBuilder =>
{
    // ... 其他配置
    .AddProcessor(new InvalidTagFilterProcessor())
});

// 自定义处理器实现
public class InvalidTagFilterProcessor : BaseProcessor<Activity>
{
    public override void OnEnd(Activity activity)
    {
        // 移除非基础类型的标签,避免转换时内存异常
        foreach (var tag in activity.Tags.ToList())
        {
            if (tag.Value == null || !IsValidTagType(tag.Value))
            {
                activity.RemoveTag(tag.Key);
            }
        }
        base.OnEnd(activity);
    }

    private bool IsValidTagType(object value)
    {
        return value is string || value is int || value is long || value is double || value is bool;
    }
}

4. 禁用分配自由枚举特性

该异常与OpenTelemetry的分配自由枚举机制相关,可通过环境变量临时禁用此特性验证:
在应用启动前添加:

Environment.SetEnvironmentVariable("OTEL_DOTNET_TRACES_ALLOCATION_FREE_ENABLED", "false");

若禁用后异常消失,说明是该机制的内存问题,后续可等待OpenTelemetry官方修复。


内容的提问来源于stack exchange,提问作者Mihai

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.20 11:42:23