如何在单个浏览器会话中执行SpecFlow Scenario Outline的所有示例？

问题描述 #

我使用Selenium结合SpecFlow的Scenario Outline进行网页测试，Outline包含大量Examples测试数据。出于性能优化需求，希望只打开一个浏览器窗口并在其中运行所有测试用例，而非为每行测试数据单独打开新窗口。

可复现示例 #

被测应用 #

这是一个基础网页，支持输入两个数字并计算乘积：

<!DOCTYPE html>
<html>
<body>
    <!-- Multiply this number... -->
    <input type="text" id="operand1" />
    *
    <!-- ...by this number... -->
    <input type="text" id="operand2" />
    =
    <!-- ...and the result should be displayed here... -->
    <span id="product"></span>
    <div>
        <!-- ...when I click this button -->
        <input type="button" id="calculate" value="Calculate" onclick="calculate()" />
    </div>

    <!-- import jQuery so that I can be confident about browser compatibility -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.1/jquery.min.js"
            integrity="sha512-aVKKRRi/Q/YV+4mjoKBsE4x3H+BkegoM/em46NNlCqNTmUYADjBbeNefNxYV7giUp0VxICtqdrbqU7iVaeZNXA=="
            crossorigin="anonymous"
            referrerpolicy="no-referrer">
    </script>

    <!-- the script which does the multiplication -->
    <script>
        function calculate() {
            var operand1 = $('#operand1').val();
            var operand2 = $('#operand2').val();
            var product = operand1 * operand2;
            $('#product').text(product);
        }
    </script>
</body>
</html>

Feature文件 #

Feature: Multiplication

As someone who's no good at mental arithmetic
I want something to multiply one number by another number
So that I can know the answer without working it out myself

Background:
    Given I am on the multiplication page

Scenario Outline: Multiply two numbers
    Given the first number is <operand1>
    And the second number is <operand2>
    When I click the calculate button
    Then the result should be <product>
    Examples:
    | operand1 | operand2 | product |
    | 1        | 2        | 2       |
    | 2        | 2        | 4       |
    | 3        | 2        | 6       |
    | 4        | 2        | 8       |
    # 此处省略数百行测试数据

步骤定义 #

using OpenQA.Selenium;

[Binding]
public sealed class MultiplicationSteps : Steps
{
    private readonly IWebDriver driver;

    public MultiplicationSteps(ScenarioContext context)
    {
        this.driver = (IWebDriver)context["driver"];
    }

    [Given(@"I am on the multiplication page")]
    public void GivenIAmOnTheMultiplicationPage()
    {
        this.driver.Navigate().GoToUrl(@"file:///C:/path/to/multiplier.html");
    }

    [Given("the first number is (.*)")]
    public void GivenTheFirstNumberIs(int number)
    {
        var operand1Box = this.driver.FindElement(By.Id("operand1"));
        operand1Box.Clear();
        operand1Box.SendKeys(number.ToString());
    }

    [Given("the second number is (.*)")]
    public void GivenTheSecondNumberIs(int number)
    {
        var operand2Box = this.driver.FindElement(By.Id("operand2"));
        operand2Box.Clear();
        operand2Box.SendKeys(number.ToString());
    }

    [When("I click the calculate button")]
    public void WhenIClickTheCalculateButton()
    {
        var calculateButton = this.driver.FindElement(By.Id("calculate"));
        calculateButton.Click();
    }

    [Then("the result should be (.*)")]
    public void ThenTheResultShouldBe(int expectedProduct)
    {
        var productDiv = this.driver.FindElement(By.Id("product"));
        var product = productDiv.Text;
        Assert.Equal(expectedProduct.ToString(), product);
    }
}

Hooks代码 #

using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;

[Binding]
public class Hooks : Steps
{
    [BeforeScenario]
    public void BeforeScenario()
    {
        if (!this.ScenarioContext.ContainsKey("driver"))
        {
            this.ScenarioContext["driver"] = new ChromeDriver();
        }
    }

    [AfterScenario]
    public void AfterScenario()
    {
        var driver = this.ScenarioContext["driver"] as IWebDriver;
        if (driver != null)
        {
            driver.Quit();
            driver.Dispose();
        }
    }
}

当前问题 #

测试功能正常，但性能极差。因为[BeforeScenario]钩子会为Scenario Outline中Examples的每一行单独执行，而非针对整个Outline执行一次，导致每行测试数据都打开新浏览器窗口，大幅增加总执行时间。

已尝试的方案 #

方案1：使用BeforeFeature/AfterFeature钩子 #

我需要类似[BeforeScenarioOutline]的钩子，但该钩子不存在，于是尝试将IWebDriver存储在FeatureContext而非ScenarioContext中：

[BeforeFeature]
public void BeforeFeature()
{
    this.FeatureContext["driver"] = new ChromeDriver();
}

[AfterFeature]
public void AfterFeature()
{
    var driver = this.FeatureContext["driver"] as IWebDriver;
    if (driver != null)
    {
        driver.Quit();
        driver.Dispose();
    }
}

但出现编译警告和运行时错误：

The binding methods for before/after feature and before/after test run events must be static!

若将方法改为static，则无法访问实例属性this.FeatureContext，BeforeTestRun和AfterTestRun钩子也存在同样问题。

方案2：静态类存储IWebDriver #

创建静态类初始化IWebDriver：

public static class TheDriver
{
    private static readonly IWebDriver driver = new ChromeDriver();
    public static IWebDriver Driver => driver;
}

修改步骤定义构造函数：

public MultiplicationSteps()
{
    this.driver = TheDriver.Driver;
}

使用[AfterTestRun]钩子在测试结束时关闭浏览器：

[AfterTestRun]
public static void AfterTestRun()
{
    if (TheDriver.Driver != null)
    {
        TheDriver.Driver.Quit();
        TheDriver.Driver.Dispose();
    }
}

该方案仅在单个Feature文件测试时有效，但非线程安全，并行运行多个Feature时会因共享浏览器窗口导致测试失败。

技术栈 #

• .NET 6.0
• Selenium.WebDriver 4.4.0
• SpecFlow 3.9.40
• xUnit 2.4.1

内容的提问来源于stack exchange，提问作者sbridewell