ASM1062 PCI-E SATA扩展卡连接硬盘出现WRITE FPDMA QUEUED错误导致RAID1故障的排查咨询
设备配置情况
我最近折腾了一台家用服务器,用的是ASRock QC5000M主板,装了Debian 12系统,目标是给两块硬盘做RAID 1。但这主板只有2个SATA口,所以我加了一块chenyang SA-208-CY PCIe SATA扩展卡,多出来2个SATA口。
用到的硬盘是两块2TB的西部数据蓝盘WDC WD20EZRZ-00Z,从2017年用到现在,用smartctl测过SMART数据,全是0错误,状态正常。
具体连接方式:
- 主板SATA1口接1块SSD(系统盘)
- 主板SATA2口接1块HDD(RAID成员盘)
- PCIe扩展卡接1块HDD(RAID成员盘)
遇到的问题
我把两块HDD组成RAID 1之后,每次重启服务器,文件系统都会损坏!每次都得手动运行fsck.ext4 -y /dev/md0修复才行。一开始只是少量inode错误,后来情况越来越糟,修复的时候直接删掉了我几乎所有文件(还好之前做了备份,没丢重要数据)。
没办法,我只能把扩展卡上的那块HDD从RAID阵列里移除,只剩主板上的单盘RAID 1才能正常工作,这显然完全失去了RAID 1的冗余意义。
我查了dmesg日志,发现了这些关键错误:
ata4.00: failed command: WRITE FPDMA QUEUEData4: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
我换了一根SATA线试了试,还是出现同样的错误。
然后我用lspci查了SATA控制器的状态,发现扩展卡的控制器没标注AHCI模式:
lspci -nn | grep -i sata 00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7801] (rev 40) 05:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
我想请教:是不是扩展卡没工作在AHCI模式导致RAID 1故障?有没有办法在Debian 12里配置解决这个问题?另外我看到Ubuntu社区的相关回答说可能是SATA供电或者电源的问题,但我现在不在服务器旁边,暂时没法测试这个方向。
后续排查更新
我后来搜了一下“WRITE FPDMA QUEUED”和“ASM1062”的相关问题,发现有人提到:只有把硬盘接在ASM1062扩展卡的COM4口时才会出问题,接另一个COM3口就不会报错。我打算尽快远程操作试试这个方案。
更新2:换接口后问题依旧
我把硬盘换到扩展卡的另一个接口后,错误还是出现了!只是日志里的错误变成了ata3.00上的WRITE FPDMA QUEUED和NCQ相关错误。
附:完整dmesg错误日志
[ 107.152069] ata4.00: exception Emask 0x10 SAct 0x2 SErr 0x400000 action 0x6 frozen [ 107.152113] ata4.00: irq_stat 0x08000000, interface fatal error [ 107.152129] ata4: SError: { Handshk } [ 107.152148] ata4.00: failed command: WRITE FPDMA QUEUED [ 107.152162] ata4.00: cmd 61/01:08:08:08:00/00:00:00:00:00/40 tag 1 ncq dma 512 out res 40/00:0c:08:08:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 107.152207] ata4.00: status: { DRDY } [ 107.152232] ata4: hard resetting link [ 107.627952] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 107.629311] ata4.00: configured for UDMA/133 [ 107.629360] ata4: EH complete [ 107.696032] ata4.00: exception Emask 0x10 SAct 0x81000 SErr 0x400000 action 0x6 frozen [ 107.696076] ata4.00: irq_stat 0x08000000, interface fatal error [ 107.696092] ata4: SError: { Handshk } [ 107.696113] ata4.00: failed command: WRITE FPDMA QUEUED [ 107.696127] ata4.00: cmd 61/01:60:08:08:00/00:00:00:00:00/40 tag 12 ncq dma 512 out res 40/00:9c:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 107.696173] ata4.00: status: { DRDY } [ 107.696189] ata4.00: failed command: READ FPDMA QUEUED [ 107.696201] ata4.00: cmd 60/08:98:00:00:00/00:00:00:00:00/40 tag 19 ncq dma 4096 in res 40/00:9c:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 107.696250] ata4.00: status: { DRDY } [ 107.696273] ata4: hard resetting link [ 108.167983] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 108.169348] ata4.00: configured for UDMA/133 [ 108.169417] sd 3:0:0:0: [sdc] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s [ 108.169435] sd 3:0:0:0: [sdc] tag#19 Sense Key : Illegal Request [current] [ 108.169447] sd 3:0:0:0: [sdc] tag#19 Add. Sense: Unaligned write command [ 108.169460] sd 3:0:0:0: [sdc] tag#19 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [ 108.169468] I/O error, dev sdc, sector 0 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2 [ 108.169535] ata4: EH complete [ 108.207911] md: recovery of RAID array md0 [ 108.331923] ata4.00: exception Emask 0x10 SAct 0x400000 SErr 0x400000 action 0x6 frozen [ 108.331970] ata4.00: irq_stat 0x08000000, interface fatal error [ 108.331988] ata4: SError: { Handshk } [ 108.332012] ata4.00: failed command: WRITE FPDMA QUEUED [ 108.332027] ata4.00: cmd 61/00:b0:00:10:04/0a:00:00:00:00/40 tag 22 ncq dma 1310720 ou res 40/00:b4:00:10:04/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 108.332079] ata4.00: status: { DRDY } [ 108.332101] ata4: hard resetting link [ 108.811925] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 108.813266] ata4.00: configured for UDMA/133 [ 108.813317] ata4: EH complete [ 108.887995] ata4: limiting SATA link speed to 3.0 Gbps [ 108.888020] ata4.00: exception Emask 0x10 SAct 0x3c SErr 0x400000 action 0x6 frozen [ 108.888053] ata4.00: irq_stat 0x08000000, interface fatal error [ 108.888069] ata4: SError: { Handshk } [ 108.888090] ata4.00: failed command: WRITE FPDMA QUEUED [ 108.888105] ata4.00: cmd 61/01:10:08:08:00/00:00:00:00:00/40 tag 2 ncq dma 512 out res 40/00:2c:00:1a:04/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 108.888156] ata4.00: status: { DRDY } [ 108.888172] ata4.00: failed command: READ FPDMA QUEUED [ 108.888186] ata4.00: cmd 60/08:18:08:08:00/00:00:00:00:00/40 tag 3 ncq dma 4096 in res 40/00:2c:00:1a:04/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 108.888233] ata4.00: status: { DRDY } [ 108.888248] ata4.00: failed command: WRITE FPDMA QUEUED [ 108.888262] ata4.00: cmd 61/00:20:00:10:04/0a:00:00:00:00/40 tag 4 ncq dma 1310720 ou res 40/00:2c:00:1a:04/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 108.888309] ata4.00: status: { DRDY } [ 108.888324] ata4.00: failed command: WRITE FPDMA QUEUED [ 108.888338] ata4.00: cmd 61/80:28:00:1a:04/00:00:00:00:00/40 tag 5 ncq dma 65536 out res 40/00:2c:00:1a:04/00:00:00:00:00/40 Emask 0x10 (ATA bus error) [ 108.888384] ata4.00: status: { DRDY } [ 108.888408] ata4: hard resetting link [ 109.364012] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 320) [ 109.365269] ata4.00: configured for UDMA/133 [ 109.365334] ata4: EH complete
附:完整lspci SATA控制器输出
$ lspci -nnk | grep --after-context=3 SATA 00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7801] (rev 40) Subsystem: ASRock Incorporation QC5000-ITX/PH [1849:7801] Kernel driver in use: ahci Kernel modules: ahci -- 05:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02) Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:1060] Kernel driver in use: ahci Kernel modules: ahci
备注:内容来源于stack exchange,提问作者A.L

