LVM管理的RAID6存储上XFS写入性能异常缓慢问题求助

I’ve seen this exact scenario play out a few times with XFS on RAID6 + LVM—especially on older CentOS 7 kernels. Let’s break down the most likely fixes to get your write performance up to par with EXT4 or raw LVM:

1. Double-Check Alignment Across the Stack #

XFS is extremely sensitive to misalignment with RAID stripe sizes, which can tank write performance. Here’s how to verify each layer:

• RAID Stripe Confirmation: First, confirm your mdadm RAID6 is using the 256k stripe you configured:

  mdadm --detail /dev/md0  # Replace /dev/md0 with your actual RAID device
  

  Look for Chunk Size: 256K in the output to confirm.

• LVM PE Size Matching: For RAID6, your stripe width is (number of data disks) × stripe size—with 6 disks total, that’s 4 data disks, so stripe width = 4×256k = 1024k (1GB). Your LVM Physical Extent (PE) size should match this to avoid splitting writes across stripes. Check your PE size with:

  vgdisplay | grep "PE Size"
  

  If it’s not 1024M, you’ll need to recreate the volume group with -s 1024M (PE size can’t be changed after creation).

• XFS Stripe Alignment: When creating the XFS filesystem, you must explicitly set the stripe unit and width to match your RAID. For your setup, the correct create command would be:

  mkfs.xfs -d su=256k,sw=4 /dev/mapper/your_vg-your_lv
  

  su = stripe unit (256k), sw = stripe width (4 data disks). To check an existing XFS filesystem’s alignment:

  xfs_info /dev/mapper/your_vg-your_lv
  

  Look for sunit=512 (XFS uses 512-byte blocks, so 256k / 512 = 512) and swidth=2048 (4×512).

2. Tune XFS Write Optimizations #

XFS has several knobs that can drastically improve write performance, especially on RAID setups:

• Disable Write Barriers (If Safe): If your RAID controller has a battery-backed write cache (BBU), you can safely disable XFS write barriers, which add unnecessary overhead. Check if barriers are enabled:

  xfs_admin -u /dev/mapper/your_vg-your_lv
  

  If it returns UUID flags = 0x0, barriers are on. Disable them with:

  xfs_admin -U 0 /dev/mapper/your_vg-your_lv
  

  Warning: Only do this if you have a working BBU—otherwise, you risk data loss on power failure.

• Adjust XFS Log Size: The XFS journal (log) can become a bottleneck for large writes. By default, CentOS 7 uses a small log size. When creating XFS, set a larger log (64M or 128M is a good start):

  mkfs.xfs -l size=64m -d su=256k,sw=4 /dev/mapper/your_vg-your_lv
  

  For existing filesystems, you can’t resize the log in-place, but you can move it to a faster device (like an SSD) if available.

• Tune VM Writeback Cache: Adjust the kernel’s dirty page thresholds to allow more data to be cached before flushing to disk. For a server with 16GB+ RAM, try these values:

  echo 67108864 > /proc/sys/vm/dirty_background_bytes  # 64MB
  echo 268435456 > /proc/sys/vm/dirty_bytes            # 256MB
  

  Make these persistent by adding to /etc/sysctl.conf:

  vm.dirty_background_bytes = 67108864
  vm.dirty_bytes = 268435456
  

3. Check Kernel and XFS Patches #

CentOS 7.4’s 3.10.0-693 kernel is quite old, and there were several XFS write performance fixes backported in later updates. Even if you said the system is updated, double-check you’re on the latest kernel for 7.4:

yum update kernel

If that doesn’t help, consider enabling the ELRepo repository to get a newer stable kernel (like 5.x), which includes significant XFS improvements.

4. Validate Benchmark Methodology #

Make sure your performance tests are consistent across filesystems. Use fio with identical parameters for XFS, EXT4, and raw LVM to eliminate variables. For example:

fio --name=raid_write_test --ioengine=libaio --rw=write --bs=256k --numjobs=4 --size=20G --iodepth=32 --direct=1 --runtime=300 --time_based

Direct I/O (--direct=1) bypasses the page cache, so you’re testing raw filesystem/RAID performance rather than cache hits.

Start with alignment checks first—misalignment is the most common cause of XFS write lag on RAID. Once that’s fixed, tweak the other parameters to squeeze out more performance.

内容的提问来源于stack exchange，提问作者teissler