如何解决MSTest并行运行Chrome实例执行Feature场景的异常？

我使用MSTest框架执行UI测试，现有10个feature文件，每个文件至少包含20个测试场景（scenarios）。目前测试能通过流水线正常运行，但执行时间长达4-5小时。因此我希望拆分测试用例，在同一代理上同时启动多个Chrome实例并行运行（无需跨浏览器）。

仓库结构 #

• Feature文件
  • 步骤定义文件（steps file）
    • 页面对象模型（Page object model）

示例代码 #

Login.feature #

Scenario: Verify if user is able to login with valid credentials
    Given I have opened a web app in new browser
    And I have navigated to url
    When I enter valid credentials
    Then I should be signed in successfully

Scenario: Verify if user is able to login with invalid credentials
    Given I have opened a web app in new browser
    And I have navigated to url
    When I enter invalid credentials
    Then an error message regarding invalid credentials

LoginSteps.cs #

[Given(@"I have opened a web app in new browser")]
public void OpenBrowser()
{
    _loginPage.OpenBrowser();
}

[Given(@"I have navigated to url")]
public void NavigateTo()
{
_loginPage.NavigateTo();
}

[When("I enter valid credentials")]
public void EnterCredentials()
{
    _loginPage.EnterCredentials();
}

[When("I enter invalid credentials")]
public void EnterInvalidCredentials()
{
    _loginPage.EnterInvalidCredentials();
}

[Then("I should be signed in successfully")]
public void SigninSucceessful()
{
    _loginPage.SigninSucceessful();
}

[Then("an error message regarding invalid credentials")]
public void SigninError()
{
    _loginPage.SigninError();
}

LoginPOM.cs #

public void navigateTo()
{
   
}

public void OpenBrowser()
{
  //code to open chrome browser
}

public void EnterCredentials(){
 // Browser.sendkeys() used here
}

尝试并行后的错误 #

我在某一步骤定义文件中添加了以下并行配置代码：

using Microsoft.VisualStudio.TestTools.UnitTesting;
[assembly: Parallelize(Workers = 2, Scope=ExecutionScope.MethodLevel)];

但运行测试时出现以下错误：

System.ArgumentNullException: Value cannot be null. (Parameter 'ScenarioContext')
TestCleanup method TM.UserFunctionalTests.Features.InventoryFeature.TestTearDown threw exception. System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object..

Hooks.cs代码 #

using BoDi;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using TechTalk.SpecFlow;
using TM.Test.Core.DataContexts;
using TM.Test.Core.Helpers;
using TM.Test.Core.Setup;
using TM.Test.Shared.Logging;

namespace TM.Test.Shared.Hooks
{
[Binding]
public class ScenarioHooks : TechTalk.SpecFlow.Steps
{
private readonly TestContext _testContext;
private readonly ScreenShotSteps _screenShotSteps;
private readonly PerformanceInfoContext _startInfo;
private readonly IObjectContainer _container;

public ScenarioHooks(TestContext testContext, ScreenShotSteps screenShotSteps, PerformanceInfoContext startInfo, IObjectContainer container)
{
_testContext = testContext;
_screenShotSteps = screenShotSteps;
_startInfo = startInfo;
_container = container;
}

[BeforeTestRun]
public static void BeforeTestRun()
{
try
{
                PInvoke.AllowSetForegroundWindow(Process.GetCurrentProcess().Id);
}
catch (Exception ex)
{
Debug.WriteLine($"Failed to do PInvoke -> {nameof(PInvoke.AllowSetForegroundWindow)}");
Debug.WriteLine(ex);
}
}

[BeforeScenario(Order = int.MinValue)]
public async Task BeforeScenarioAsync()
{
Console.WriteLine($"|||| Current CPU load: {_startInfo.GetCPULoad()}, available memory: {_startInfo.GetAvailableMemory()} ");

await _testContext.SetUpNewScenarioRun();

Console.WriteLine($"|||| Current CPU load: {_startInfo.GetCPULoad()}, available memory: {_startInfo.GetAvailableMemory()} ");
Console.WriteLine("= SCENARIO: " + ScenarioContext.ScenarioInfo.Title);
Console.WriteLine("= FEATURE: " + FeatureContext.FeatureInfo.Title);


//Initialize Serilog
var logService = new LogService();
_container.RegisterInstanceAs(logService, typeof(ILogService));
}

[AfterScenario(Order = int.MinValue)]
public void AfterScenarioBeforeTearDown()
{
var lastError = ScenarioContext.TestError;
var result = lastError == null ? "PASSED" : "FAILED";

Console.WriteLine("EXECUTED SCENARIO: " + ScenarioContext.ScenarioInfo.Title);
Console.WriteLine("EXECUTED FEATURE: " + FeatureContext.FeatureInfo.Title);
Console.WriteLine($"RESULTS OF EXECUTION: {result}");
if (lastError != null)
{
Console.WriteLine($"ERROR MESSAGE: {lastError.Message}");
Console.WriteLine($"ERROR SOURCE: {lastError.Source}");
Console.WriteLine($"ERROR STACKTRACE: {lastError.StackTrace}");

_screenShotSteps.TakeAScreenShotWithContextData();
}
           
//Dispose of Serilog
var logService = _container.Resolve<ILogService>();
logService.CloseAndFlush();
}

[AfterScenario(Order = int.MaxValue)]
private async Task AfterScenarioTearDown()
{
var stopwatch = Stopwatch.StartNew();
try
{
await _testContext.TearDownAfterScenarioRun();
Debug.WriteLine($"Tear down time: {stopwatch.Elapsed.TotalSeconds:N1} seconds.");
Debug.WriteLine($"Total scenario run time: {_testContext.ScenarioStopwatch.Elapsed.TotalSeconds:N1} seconds.");
}
catch (Exception ex)
{
Debug.WriteLine("Failed to tear down after scenario.");
Debug.WriteLine(ex);
}
}
}
}

问题定位 #

我发现问题出在Feature文件自动生成的.cs文件中，TestTearDown方法抛出异常：

[Microsoft.VisualStudio.TestTools.UnitTesting.TestCleanupAttribute()]
public void TestTearDown()
{
    testRunner.OnScenarioEnd();
}

内容的提问来源于stack exchange，提问作者User123456789