如何解决CI/CD中随机失败的Ruby RSpec排序测试？

问题描述 #

我的Ruby RSpec测试在CI/CD环境中约70%概率随机失败，但本地运行完全正常。失败点集中在供应商名称降序排序的断言，返回的列表顺序偶尔不符合预期。

测试代码 #

require "rails_helper"

RSpec.describe "Admin::Vendors", type: :request do
  let!(:vendors) {
    create_list(:vendor, 5, type: 1) do |vendor|
      create(:vendor_user, vendor_id: vendor.id, role: "vendor")
    end

    create_list(:vendor, 5, type: 2) do |vendor|
      create(:vendor_user, vendor_id: vendor.id, role: "vendor")
    end

    create_list(:vendor, 5, status: "waiting_for_approval") do |vendor|
      create(:vendor_user, vendor_id: vendor.id, role: "vendor")
    end

    create_list(:vendor, 5, status: "inactive") do |vendor|
      create(:vendor_user, vendor_id: vendor.id, role: "vendor")
    end
  end
  
  describe "index" do
    context "(success - display the list of vendors by vendor name asc)" do
      before { get "/api/v1/vendor_management/admin/vendors?sort_column=vendor_name&sort_by=asc", headers: headers[:auth] }

      it "returns list of vendors in asc order" do
        asc_vendor_names = json["vendors"].pluck("name")
        expect(json).not_to be_empty
        expect(json["vendors"]).not_to be_empty
        expect(response).to have_http_status(200)
        expect(asc_vendor_names).to eq(asc_vendor_names.sort)
      end
    end

    context "(success - display the list of vendors by vendor name desc)" do
      before { get "/api/v1/vendor_management/admin/vendors?sort_column=vendor_name&sort_by=desc", headers: headers[:auth] }

      it "returns list of vendors in desc order" do
        desc_vendor_names = json["vendors"].pluck("name")
        expect(json).not_to be_empty
        expect(json["vendors"]).not_to be_empty
        expect(response).to have_http_status(200)
        expect(desc_vendor_names).to eq(desc_vendor_names.sort.reverse)
      end
    end
  end

end

失败信息 #

Failures:

  1) Admin::VendorManagement::VendorController index (success - display the list of vendors by vendor name desc) returns list of vendors in desc order
     Failure/Error: expect(desc_vendor_names).to eq(desc_vendor_names.sort.reverse)
     
       expected: ["Wyatt Conn", "Vikki Rowe VM", "Rocco Bauch", "Randal Muller", "Noe Koch", "Natashia Stark", "Miss L...oe Walter", "Dwain Stoltenberg", "Chara Orn", "Benito Hegmann", "Armida Kuphal", "Agustin Mosciski"]
            got: ["Wyatt Conn", "Vikki Rowe VM", "Rocco Bauch", "Randal Muller", "Noe Koch", "Natashia Stark", "Miss L...h Murazik", "Dwain Stoltenberg", "Chara Orn", "Benito Hegmann", "Armida Kuphal", "Agustin Mosciski"]
     
       (compared using ==)

问题根源与解决方案 #

根源分析 #

问题出在单字段排序的不稳定性：当多个供应商的vendor_name排序后被判定为相等（或存在重复值），数据库不会保证这些记录的返回顺序。本地环境可能因缓存、数据量小等因素偶然保持顺序一致，但CI环境每次都是全新数据库实例，查询执行的细微差异会导致重复值的顺序随机变化，触发断言失败。

解决方案 #

1. 添加稳定的二级排序字段（推荐）

修改后端控制器的排序逻辑，在vendor_name基础上添加一个唯一字段（比如id）作为二级排序，确保结果始终稳定：

def index
  sort_column = params[:sort_column] || 'vendor_name'
  sort_direction = params[:sort_by] || 'asc'
  # 先按指定字段排序，再按id同方向排序保证稳定性
  vendors = Vendor.order("#{sort_column} #{sort_direction}", id: sort_direction)
  # 后续序列化返回逻辑...
end

2. 调整测试断言逻辑

如果暂时无法修改后端，可调整测试断言，只验证排序趋势而非精确顺序：

it "returns list of vendors in desc order" do
  desc_vendor_names = json["vendors"].pluck("name")
  expect(json).not_to be_empty
  expect(response).to have_http_status(200)
  # 验证每个元素大于等于下一个元素，符合降序趋势
  desc_vendor_names.each_cons(2) do |a, b|
    expect(a).to be >= b
  end
end

3. 确保测试数据名称唯一

检查FactoryBot的vendor工厂，生成唯一的名称避免排序冲突：

FactoryBot.define do
  factory :vendor do
    name { Faker::Name.unique.name } # 使用unique保证名称不重复
    # 其他属性配置...
  end
end

同时在rails_helper.rb中添加重置逻辑，避免重复测试时Faker唯一值耗尽：

RSpec.configure do |config|
  config.before(:suite) do
    Faker::UniqueGenerator.clear
  end
end

内容的提问来源于stack exchange，提问作者RanH