ZFS存储组内多块磁盘UUID相同是否正常?新盘校验和异常及池降级问题求助
首先直接给你吃个定心丸:你看到的所有ZFS成员分区的UUID="10204323147149787819"完全正常!这个UUID是你整个nas池的全局UUID,不是单个磁盘的唯一标识。真正用来区分每个vdev成员的是UUID_SUB字段——你看你的blkid输出里,每个磁盘的UUID_SUB都是独一无二的,这才是ZFS用来识别不同设备的关键ID,所以这部分完全不用担心。
接下来咱们聊聊你遇到的实际问题:
问题分析
从你的zpool status输出能看到:
- 你的
nas池处于DEGRADED状态,有一个设备因标签丢失/无效不可用 - 两块新增的3.7T磁盘(
scsi-350000397083ac015和scsi-35000039708425461)都出现了26个校验和错误,数量完全一致
这种成对出现的校验和错误,大概率不是磁盘本身的问题,更可能是共享链路/控制器的故障,比如:
- 两块新盘接在了同一个有故障的HBA卡、扩展背板或线缆上,导致数据传输时出现一致性错误
- 新盘的固件存在bug,导致数据校验时误报错误
- 也不排除磁盘本身的硬件问题,但两块盘同时出问题的概率相对较低
另外那个不可用的设备17872382825322943538,提示是原来的/dev/disk/by-id/scsi-35000c500aeccd07b-part1,可能是这块旧盘故障离线,或者你在替换时操作有误,导致ZFS无法识别它的标签。
排查&解决建议
1. 先做数据备份(重中之重)
虽然你的池是raidz3有冗余,但处于DEGRADED状态下风险极高,先把重要数据备份到其他存储介质,避免意外。
2. 检查硬件链路
- 把两块新盘换到其他正常的SATA/SAS接口或线缆上,排除接触不良或链路故障
- 检查HBA卡/磁盘控制器的状态,看是否有报错信息
3. 检测磁盘健康状态
用smartctl工具检查新盘的SMART数据,看是否有硬件异常:
smartctl -a /dev/sdg smartctl -a /dev/sdi
重点关注Reallocated_Sector_Ct(重新分配扇区数)、Current_Pending_Sector(待映射扇区数)、Offline_Uncorrectable(离线无法校正扇区数)这些指标,如果数值不为0,说明磁盘有硬件故障。
4. 处理不可用设备
确认旧盘scsi-35000c500aeccd07b已经物理移除后,用新盘替换这个不可用的设备:
# 一定要用磁盘的by-id路径,不要用/dev/sdx这种会变化的路径 zpool replace nas 17872382825322943538 /dev/disk/by-id/scsi-35000039708xxxxxxx
替换完成后,等待ZFS完成 resilver(数据重建),然后运行一次scrub:
zpool scrub nas
5. 观察校验和错误是否复现
scrub完成后,再次运行zpool status nas,看两块新盘的校验和错误是否消失,或者是否继续增加。如果错误消失,说明是链路问题;如果错误持续增加,可能需要联系磁盘厂商售后,排查磁盘固件或硬件问题。
附:你提供的原始诊断信息
zpool status输出:
$ zpool status pool: nas state: DEGRADED status: One or more devices could not be used because the label is missing or invalid. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the device using 'zpool replace'. scan: scrub repaired 0B in 0 days 12:53:46 with 0 errors on Mon Aug 7 06:59:53 2023 config: NAME STATE READ WRITE CKSUM nas DEGRADED 0 0 0 raidz3-0 DEGRADED 0 0 0 scsi-35000cca05884aa50 ONLINE 0 0 0 scsi-35000cca0588494a0 ONLINE 0 0 0 scsi-35000cca058852aa0 ONLINE 0 0 0 scsi-35000cca058852098 ONLINE 0 0 0 scsi-35000c50093a9182b ONLINE 0 0 0 scsi-350000397083ac015 ONLINE 0 0 26 scsi-35000039708425461 ONLINE 0 0 26 17872382825322943538 UNAVAIL 0 0 0 was /dev/disk/by-id/scsi-35000c500aeccd07b-part1 errors: No known data errors
blkid输出:
/dev/sdf1: LABEL="System Reserved" UUID="A2885ECD885EA019" TYPE="ntfs" /dev/sdf2: UUID="5E2E62DB2E62ABA9" TYPE="ntfs" /dev/sdf3: UUID="226e836d-7b8e-424c-b0a0-0397ee458c7c" TYPE="xfs" /dev/sdf5: UUID="60c94586-7d6a-4e8a-b350-04719990cb69" TYPE="xfs" /dev/sdf6: UUID="4d91f3bb-8c97-43c8-acea-fb1dd1fe0ed7" TYPE="swap" /dev/sdf7: UUID="bbc69fc6-12fa-499a-a0c6-e0f65e248ce2" TYPE="xfs" /dev/sda1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="6584635425097243078" TYPE="zfs_member" PARTLABEL="zfs-132734cdf8c9f4c2" PARTUUID="6c32ae56-c2aa-a744-b387-6c15664472ae" /dev/sde1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="7415027662888351724" TYPE="zfs_member" PARTLABEL="zfs-376a6cd58d69e2f2" PARTUUID="87b4ec37-cc21-544a-9186-2a14bf6ef88c" /dev/sdb1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="9819349529238006405" TYPE="zfs_member" PARTLABEL="zfs-335c2de359086356" PARTUUID="bc0eba1e-6ec0-ac41-9254-1622f38e168f" /dev/sdc1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="11038151487099663370" TYPE="zfs_member" PARTLABEL="zfs-51d5d511e56fdb77" PARTUUID="d2caa7bf-48ce-2d4d-8b39-6664f1f513a4" /dev/sdd1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="10380994937332854296" TYPE="zfs_member" PARTLABEL="zfs-122d2866eacc955b" PARTUUID="7ef16354-aab4-3048-91e2-2bd1f9d46723" /dev/sdg1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="6768350752206510228" TYPE="zfs_member" PARTLABEL="zfs-b0cecfa2ba1e7d10" PARTUUID="c0204fc5-58d1-ca4e-bd8e-73b2ff6700f5" /dev/sdi1: LABEL="nas" UUID="10204323147149787819" UUID_SUB="9578689271328227841" TYPE="zfs_member" PARTLABEL="zfs-f317a6736bc4baaa" PARTUUID="48cdd444-4af8-6942-8081-ea4ad3bc631b" /dev/sda9: PARTUUID="13d570a2-d306-194d-84fa-2bb119176f45" /dev/sde9: PARTUUID="ae55d3b9-ba7c-3d41-956d-21f0f1c0bba1" /dev/sdb9: PARTUUID="69dff684-38bc-e04d-bce5-9fc17ea9ac80" /dev/sdc9: PARTUUID="6d6fc257-f983-e540-948d-6fb955439781" /dev/sdd9: PARTUUID="be43f88a-e235-6641-9dbf-4f1c8d13c6a2" /dev/sdg9: PARTUUID="a4f58548-c276-c24a-bfcf-776e208f7c8c" /dev/sdi9: PARTUUID="1fb21d63-5781-eb48-872d-2f7672c682ab"
lsblk输出:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 2.7T 0 disk ├─sda1 8:1 0 2.7T 0 part └─sda9 8:9 0 8M 0 part sdb 8:16 0 2.7T 0 disk ├─sdb1 8:17 0 2.7T 0 part └─sdb9 8:25 0 8M 0 part sdc 8:32 0 2.7T 0 disk ├─sdc1 8:33 0 2.7T 0 part └─sdc9 8:41 0 8M 0 part sdd 8:48 0 2.7T 0 disk ├─sdd1 8:49 0 2.7T 0 part └─sdd9 8:57 0 8M 0 part sde 8:64 0 2.7T 0 disk ├─sde1 8:65 0 2.7T 0 part └─sde9 8:73 0 8M 0 part sdf 8:80 0 465.8G 0 disk ├─sdf1 8:81 0 100M 0 part ├─sdf2 8:82 0 122G 0 part ├─sdf3 8:83 0 500M 0 part /boot ├─sdf4 8:84 0 1K 0 part ├─sdf5 8:85 0 100.3G 0 part /home ├─sdf6 8:86 0 5.9G 0 part [SWAP] └─sdf7 8:87 0 40G 0 part / sdg 8:96 0 3.7T 0 disk ├─sdg1 8:97 0 3.7T 0 part └─sdg9 8:105 0 8M 0 part sdh 8:112 0 3.7T 0 disk sdi 8:128 0 3.7T 0 disk ├─sdi1 8:129 0 3.7T 0 part └─sdi9 8:137 0 8M 0 part
备注:内容来源于stack exchange,提问作者Brian Thomas

