Playwright C#中无法稳定遍历分页行的技术问题

我使用Playwright C#遍历某网站的分页设备数据，该网站共有8页，每页最多展示10台设备。我通过页面上的总设备数计算出总页数，再用for循环点击每个分页按钮以获取所有设备数据，但循环执行不稳定——有时到第6页就提前结束，有时能正常遍历到第8页。我推测是Playwright的异步特性导致的，原以为加await关键字就能保证顺序执行，但实际循环速度还是过快。拆分异步函数也没能解决问题，只有在循环末尾加Thread.Sleep能略有改善，但不确定这是不是正确的处理方式。

using System.Threading.Tasks;
using Microsoft.Playwright;

namespace PlaywrightTests.Pom;
public class Devices
{
  private readonly IPage _page;
  private readonly ILocator _searchField;
  private readonly ILocator _goButton;
  private readonly ILocator _previousPageButton;
  private readonly ILocator _table;
  private readonly ILocator _totalNumberOfDevicesElement;

  public Devices(IPage page)
  {
    _page = page;
    _searchField = page.GetByPlaceholder("Search");
    _goButton = page.GetByRole(AriaRole.Button, new() { Name = "Go" });
    _previousPageButton = page.GetByLabel("Previous");
    _table = page.Locator("//*[@id='tblList']");
    _totalNumberOfDevicesElement = page.Locator("//div[@class='row ng-star-inserted']/div/span");
  }

  public async Task<double> CalculateNumberOfPages(string numberOfDevices) {
      // Dummy click on an element which succeeds the element containing the number of devices, to make sure that the number of devices item loaded completely
      await _previousPageButton.ClickAsync();
      int pFrom = numberOfDevices.IndexOf("of ") + "of ".Length;
      int pTo = numberOfDevices.LastIndexOf(" items");
      numberOfDevices = numberOfDevices.Substring(pFrom, pTo - pFrom);
      double devicesNumber = Convert.ToDouble(numberOfDevices);
      double numberOfPages = Math.Ceiling(devicesNumber / 10);
      return numberOfPages;
  }

  public async Task IterateOverPages(double nmbrOfPages) {
      for (int i = 1; i <= nmbrOfPages; i++) {
       var singlePage = _page.GetByRole(AriaRole.Link, new() { Name = i.ToString() , Exact = true});
       await singlePage.ClickAsync();
      }
  }

  public async Task GoThroughPages() {
      string totalNumberOfDevices = await _totalNumberOfDevicesElement.InnerTextAsync();
      double convertedTotalNumber = await CalculateNumberOfPages(totalNumberOfDevices);
      await IterateOverPages(convertedTotalNumber);
  }
}

1. 用Playwright原生等待替代Thread.Sleep #

Thread.Sleep是阻塞式等待，会卡死整个线程，完全不符合异步编程的思路。Playwright提供了更智能的等待机制，推荐用以下方式：

• 点击分页后等待页面加载完成：await _page.WaitForLoadStateAsync(LoadState.NetworkIdle);，确保网络请求全部完成
• 等待当前分页按钮处于激活状态（比如带有active类），确认页面已切换到目标页

修改后的IterateOverPages方法示例：

public async Task IterateOverPages(double nmbrOfPages) {
    for (int i = 1; i <= nmbrOfPages; i++) {
        var pageLink = _page.GetByRole(AriaRole.Link, new() { Name = i.ToString(), Exact = true });
        // 等待分页按钮可见再点击
        await pageLink.WaitForAsync(new LocatorWaitForOptions { State = WaitForSelectorState.Visible });
        await pageLink.ClickAsync();
        
        // 等待页面加载稳定，或者等待当前页成为激活状态
        await _page.WaitForLoadStateAsync(LoadState.NetworkIdle);
        await pageLink.WaitForAsync(new LocatorWaitForOptions { HasClass = "active" });
        
        // 这里添加获取当前页设备数据的逻辑
    }
}

2. 修正总页数计算逻辑 #

原代码中点击_previousPageButton的“dummy点击”可能导致页面跳转，反而干扰总页数的获取。直接等待总设备数元素加载完成即可：

public async Task<double> CalculateNumberOfPages() {
    // 等待总设备数元素加载完成
    await _totalNumberOfDevicesElement.WaitForAsync();
    string totalText = await _totalNumberOfDevicesElement.InnerTextAsync();
    
    int fromIndex = totalText.IndexOf("of ") + "of ".Length;
    int toIndex = totalText.LastIndexOf(" items");
    if (fromIndex <= 0 || toIndex <= fromIndex) {
        throw new InvalidOperationException("无法解析页面上的总设备数");
    }
    
    if (!double.TryParse(totalText.Substring(fromIndex, toIndex - fromIndex), out double deviceCount)) {
        throw new InvalidOperationException("总设备数字解析失败");
    }
    
    return Math.Ceiling(deviceCount / 10);
}

对应的GoThroughPages方法也需要调整：

public async Task GoThroughPages() {
    double totalPages = await CalculateNumberOfPages();
    await IterateOverPages(totalPages);
}

3. 处理分页按钮折叠的情况 #

如果网站页数较多时，分页栏会隐藏部分页码（用省略号代替），直接点击数字链接会失败。这种情况需要先点击“下一页”按钮，或者展开隐藏的页码区域，再进行点击。可以先判断目标页码是否可见，不可见就点击分页栏的“更多”或“下一页”按钮，直到目标页码出现。

4. 添加异常捕获与重试 #

在循环中添加try-catch块，对点击失败的页码进行重试，避免单次失败导致整个循环终止：

public async Task IterateOverPages(double nmbrOfPages) {
    for (int i = 1; i <= nmbrOfPages; i++) {
        int retryCount = 3;
        while (retryCount > 0) {
            try {
                var pageLink = _page.GetByRole(AriaRole.Link, new() { Name = i.ToString(), Exact = true });
                await pageLink.WaitForAsync(new LocatorWaitForOptions { State = WaitForSelectorState.Visible, Timeout = 5000 });
                await pageLink.ClickAsync();
                await _page.WaitForLoadStateAsync(LoadState.NetworkIdle);
                break;
            } catch (PlaywrightException) {
                retryCount--;
                if (retryCount == 0) throw;
                await _page.WaitForTimeoutAsync(1000); // 短暂等待后重试
            }
        }
    }
}

内容的提问来源于stack exchange，提问作者apploid