MS.Test V3.4.3 DataTestMethod在Azure Pipeline中WebAPI环境仅执行单条测试

已将旧版单元测试平台升级至MS.TestFramework V3.4.3，用于通过[DataTestMethod]和[DataRow]特性编写数据驱动测试。测试框架采用抽象架构，可同时运行API和Web服务测试，通过TestCategory特性定义测试执行方式，并在Pipeline层面做过滤。

测试类代码 #

[TestClass]
public class ApiHistoryTests : AllApiTestBase
{
    public ApiHistoryTests()
    {
    }

    [DataTestMethod]
    [TestCategory("WebAPI")]
    [TestCategory("SOAP")]
    [Priority(1)]
    [DataRow("History - Document Name", "test2", "test1")]
    [DataRow("History - Document Name", "test1", null)]
    public void X_Y_Z_TEST(string titleSetting, string field1, string field2)
    {
        // 测试逻辑省略
    }
}

异常现象 #

在Visual Studio中，两个DataRow对应的测试用例均可正常显示和执行，但在Azure Pipeline执行时出现差异：按环境过滤后，SOAP环境能发现并执行两个测试用例，而WebAPI环境仅执行第一个测试用例。

当前环境：.NET 4.7.2、Microsoft.Net.Test.SDK 17.10

执行日志对比 #

WebAPI阶段日志 #

024-06-19T07:37:56.1873197Z Discovering tests from sources
2024-06-19T07:37:56.1969717Z Using new test platform for test execution
2024-06-19T07:37:56.2088724Z Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper instance will be created
2024-06-19T07:38:01.2795503Z **************** Starting test execution *********************
2024-06-19T07:38:01.2844479Z Using new test platform for test execution
2024-06-19T07:38:01.2844880Z Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper instance will be created

SOAP阶段日志 #

**************** Starting test execution *********************
2024-06-18T09:40:48.2708107Z [command]C:\TestAgent\_work\_tool\VsTest\17.1.0\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@C:\TestAgent\_work\_temp\xdn1tlpbrvz.tmp"
2024-06-18T09:40:48.3381835Z Microsoft (R) Test Execution Command Line Tool Version 17.1.0
2024-06-18T09:40:48.3382290Z Copyright (c) Microsoft Corporation.  All rights reserved.
2024-06-18T09:40:48.3416326Z vstest.console.exe "C:\Test\API\Therefore.Test.ApiCore.dll"

Pipeline配置对比 #

WebAPI阶段配置 #

variables:
  TestPlatformVersion: '17.1.0'

steps:
- task: VisualStudioTestPlatformInstaller@1
  displayName: 'Install Visual Studio Test Platform'
  inputs:
    versionSelector: specificVersion
    testPlatformVersion: '$(TestPlatformVersion)'

- task: VSTest@2
  displayName: 'Execute Tests on Agent - Part 1'
  inputs:
    testAssemblyVer2: Test.ApiCore.dll
    searchFolder: 'C:\Test\API'
    testFiltercriteria: '$(TestFilterCriteria)'
    vsTestVersion: toolsInstaller
    runSettingsFile: '$(TestRunArchitecture).runsettings'
    codeCoverageEnabled: false
    distributionBatchType: basedOnExecutionTime
    dontDistribute: false
    testRunTitle: 'Web API Tests ($(TestRunArchitecture)) Part 1'
    platform: '$(ConfigurationName)'
    configuration: 'WebAPI ($(TestRunArchitecture)) @ $(OSInfo)'
    rerunFailedTests: true
    rerunType: basedOnTestFailureCount
    rerunFailedTestCasesMaxLimit: 10
    rerunMaxAttempts: 1
  continueOnError: true
  timeoutInMinutes: 40

注：原配置中testFiltercriteria缺少闭合引号，已修正

SOAP阶段配置 #

variables:
  TestPlatformVersion: '17.1.0'

steps:
- task: VisualStudioTestPlatformInstaller@1
  displayName: 'VsTest Platform Installer'
  inputs:
    versionSelector: specificVersion
    testPlatformVersion: '$(TestPlatformVersion)'

- task: VSTest@2
  displayName: 'Execute Tests on Agent '
  inputs:
    testAssemblyVer2: Test.ApiCore.dll
    searchFolder: 'C:\Test\API'
    testFiltercriteria: '$(TestFilterCriteria)'
    vsTestVersion: toolsInstaller
    runSettingsFile: '$(TestRunArchitecture).runsettings'
    dontDistribute: true
    testRunTitle: 'API Tests ($(TestRunArchitecture)) Part 1 - $(Agent.Name)'
    platform: '$(ConfigurationName)'
    configuration: 'API ($(TestRunArchitecture)) @ $(OSInfo)'
    rerunFailedTests: true
    rerunType: basedOnTestFailureCount
    rerunFailedTestCasesMaxLimit: 10
    rerunMaxAttempts: 1
  continueOnError: true

注：原配置中runSettingsFile缺少起始引号，已修正

已尝试的操作 #

• 修改项目构建输出目录至\API\，仅保留MSTest.TestAdapter，问题未解决

期望结果 #

WebAPI和SOAP两个阶段能发现并执行完全相同的测试用例，寻求可行的解决方法。

内容的提问来源于stack exchange，提问作者matahari32