RTX A6000启动及GPU负载时PCI AER修正错误的根源排查问询

我遇到了NVIDIA RTX A6000的一个问题：每次机器启动或者GPU处于负载状态时，都会触发PCI AER（高级错误报告）相关的修正错误。以下是详细的信息和我已经尝试过的排查步骤，希望能得到大家的分析建议。

问题现象与日志详情 #

通过dmesg命令可以看到，系统报告了三个PCI设备的AER缓冲区溢出错误：

• 41:00.1 音频设备：NVIDIA Corporation GA102 High Definition Audio Controller (rev a1)
• 41:00.0 VGA兼容控制器：NVIDIA Corporation GA102GL [RTX A6000] (rev a1)
• 40:01.1 PCI桥：Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge

这些错误都被标记为已修正，但日志中也显示snd_hda_intel 0000:41:00.1受到了该问题的影响。

错误日志片段 #

[    5.301395] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301397] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301399] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301401] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301402] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301403] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301405] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301405] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301406] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301407] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301408] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301409] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301410] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301411] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301411] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301413] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301414] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301414] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301416] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301416] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301417] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301418] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301419] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301420] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301421] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301422] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301422] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301424] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301424] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301425] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301426] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301427] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301428] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301429] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301430] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301430] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301432] snd_hda_intel 0000:41:00.1: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301432] snd_hda_intel 0000:41:00.1:    [ 0] RxErr                  (First)
[    5.301433] snd_hda_intel 0000:41:00.1: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301435] nvidia 0000:41:00.0: AER: aer_status: 0x00000001, aer_mask: 0x00000000
[    5.301435] nvidia 0000:41:00.0:    [ 0] RxErr                  (First)
[    5.301436] nvidia 0000:41:00.0: AER: aer_layer=Physical Layer, aer_agent=Receiver ID
[    5.301437] pcieport 0000:40:01.1: AER: aer_status: 0x00001000, aer_mask: 0x00000000
[    5.301438] pcieport 0000:40:01.1:    [12] Timeout
[    5.301439] pcieport 0000:40:01.1: AER: aer_layer=Data Link Layer, aer_agent=Transmitter ID
[    5.301440] pcieport 0000:40:01.1: AER: aer_status: 0x00001000, aer_mask: 0x00000000
[    5.301441] pcieport 0000:40:01.1:    [12] Timeout
[    5.301442] pcieport 0000:40:01.1: AER: aer_layer=Data Link Layer, aer_agent=Transmitter ID

修正错误日志示例 #

[   10.419954] {3}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 512
[   10.419957] {3}[Hardware Error]: It has been corrected by h/w and requires no further action
[   10.419958] {3}[Hardware Error]: event severity: corrected
[   10.419959] {3}[Hardware Error]:  Error 0, type: corrected
[   10.419960] {3}[Hardware Error]:   section_type: PCIe error
[   10.419960] {3}[Hardware Error]:   port_type: 4, root port
[   10.419961] {3}[Hardware Error]:   version: 0.2
[   10.419961] {3}[Hardware Error]:   command: 0x0407, status: 0x0010
[   10.419962] {3}[Hardware Error]:   device_id: 0000:40:01.1
[   10.419963] {3}[Hardware Error]:   slot: 0
[   10.419964] {3}[Hardware Error]:   secondary_bus: 0x41
[   10.419964] {3}[Hardware Error]:   vendor_id: 0x1022, device_id: 0x1483
[   10.419965] {3}[Hardware Error]:   class_code: 060400
[   10.419966] {3}[Hardware Error]:   bridge: secondary_status: 0x0000, control: 0x0012

已尝试的排查与解决措施 #

和供应商配合排查后，我已经尝试了以下方法来定位问题：

• 移除RTX A6000显卡后，AER错误完全消失
• 更新了机器内两块Western Digital SN850X NVMe/SSD的固件
• 将系统安装到其他型号的SSD上，排除WD SN850X引发问题的可能性
• 确认PNY提供的RTX A6000没有可用的BIOS更新（已更新过显卡BIOS）
• 在Windows系统下运行未检测到特定硬件问题
• 测试了Linux 6.2版本内核，确保硬件组件都能被正确适配
• 在GRUB启动菜单中关闭了ASPM（BIOS中仅存储设备有ASPM控制选项，GPU无相关设置）
• 学生使用该机器运行计算任务未报告异常；Windows下FurMark和Ubuntu下GPUburn压力测试均能正常完成，说明错误确实被硬件自动修正了

我的疑问 #

虽然目前使用过程中没有出现功能性问题，但我还是希望能搞清楚这些AER错误的根源——不确定这是Linux系统层面的软件问题，还是RTX A6000显卡本身的硬件问题。更重要的是，我想确认这些已修正的错误会不会对未来的计算工作产生潜在影响，毕竟这台机器主要用于计算任务。

感谢大家的帮助！

备注：内容来源于stack exchange，提问作者jamboNum5