Playwright修改页面元素后解析显示未选中问题求助

我用Playwright操作目标网站（截图：https://i.sstatic.net/jm9FP.png），选中了name为selectFirstVariationGroup[]的下拉框中value为182695的选项，视觉上已经看到选中效果，但用requests获取页面内容再用BeautifulSoup解析后，select标签的选中项还是默认的“Seçiniz”（value=-1），求解决办法。

相关代码 #

def run(playwright: Playwright) -> None:
browser = playwright.firefox.launch(headless=False)
context = browser.new_context()

page = context.new_page()

page.goto("https://www.shopier.com/ShowProductNew/products.php?id=12617395")
r = requests.get(page.url)

page.locator('select[name="selectFirstVariationGroup[]"]').select_option(value='182695')

page.wait_for_load_state('networkidle')
page.wait_for_timeout(4000)

soup = BeautifulSoup(r.content, 'html.parser')

select_tag = soup.find_all("select", {'type':'hidden'})
print(select_tag)

执行输出 #

[<select class="custom-select btn-block" id="selectFirstVariationGroup" name="selectFirstVariationGroup[]" onchange="singleVariationFirstVariationClick(this)"> 

<option selected="" value="-1">Seçiniz</option>
<option value="182693">M</option>
<option value="182694">L</option>
<option value="182695">XL</option>
<option value="182696">XXL</option> </select>]

问题核心是你用requests.get(page.url)单独发起了页面请求，这个请求完全独立于Playwright的浏览器上下文，拿到的是页面初始的静态HTML，自然看不到Playwright操作后的DOM变化。

正确做法是从Playwright当前页面中获取实时DOM内容，而非重新请求：

修改步骤 #

1. 删除r = requests.get(page.url)这一行代码
2. 操作完下拉框后，用page.content()获取当前页面的HTML，再传入BeautifulSoup
3. 修正BeautifulSoup的定位条件：原代码用type='hidden'定位，但目标select标签无此属性，改用name='selectFirstVariationGroup[]'更准确

修改后的代码 #

def run(playwright: Playwright) -> None:
    browser = playwright.firefox.launch(headless=False)
    context = browser.new_context()

    page = context.new_page()
    page.goto("https://www.shopier.com/ShowProductNew/products.php?id=12617395")

    # 选中目标选项
    page.locator('select[name="selectFirstVariationGroup[]"]').select_option(value='182695')

    page.wait_for_load_state('networkidle')
    page.wait_for_timeout(4000)

    # 获取Playwright当前页面的实时HTML
    current_html = page.content()
    soup = BeautifulSoup(current_html, 'html.parser')

    # 用正确的属性定位select标签
    select_tag = soup.find_all("select", {'name': 'selectFirstVariationGroup[]'})
    print(select_tag)

内容的提问来源于stack exchange，提问作者Adrian Demir