如何使用Playwright捕获页面重定向并获取最终跳转URL
问题描述
访问短链 https://scnv.io/760y 时,页面会先运行一段JavaScript代码,随后自动重定向到PDF文件地址 https://qcg-media.s3.amazonaws.com/media/uploads/72778/2022/06/20220622_663043_221.pdf。
使用如下Python版Playwright代码尝试获取最终URL未成功:
from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.launch() page = browser.new_page() page.goto("https://scnv.io/760y", wait_until="networkidle") print(page.url) page.close()
代码中已设置wait_until="networkidle"参数等待网络空闲后打印页面URL,仍无法获取重定向后的最终PDF地址。
失效原因
原代码拿不到目标地址的核心原因有两点:
wait_until="networkidle"的判定规则为页面连续500ms无网络请求即触发,这类短链的JS跳转通常带有1-3秒的固定延时(比如跳转倒计时),networkidle触发时跳转逻辑还未执行。- 跳转目标为PDF文件时,Chrome会调用内置PDF阅读器加载资源,此时主框架的URL更新逻辑和普通网页导航存在差异,直接读取
page.url无法及时同步到最终地址。
实现方案
方案1:监听网络响应捕获地址(兼容性最优)
不依赖页面加载状态和page.url更新,直接监听页面所有网络响应,匹配到PDF资源时直接记录地址,适用于PDF在线预览的场景:
from playwright.sync_api import sync_playwright with sync_playwright() as p: browser = p.chromium.launch() page = browser.new_page() final_pdf_url = None def capture_pdf_url(response): nonlocal final_pdf_url content_type = response.headers.get("content-type", "") # 匹配规则:响应类型为PDF,或响应地址属于目标PDF存储域名 if "application/pdf" in content_type or "qcg-media.s3.amazonaws.com" in response.url: final_pdf_url = response.url page.on("response", capture_pdf_url) page.goto("https://scnv.io/760y", wait_until="domcontentloaded") # 等待JS跳转执行,可根据短链实际跳转延时调整时长 page.wait_for_timeout(5000) print(final_pdf_url) browser.close()
方案2:监听下载事件捕获地址
如果浏览器对该PDF响应触发下载行为而非在线预览,可以通过监听下载事件直接拿到源地址:
from playwright.sync_api import sync_playwright with sync_playwright() as p: # 开启下载权限 browser = p.chromium.launch(accept_downloads=True) page = browser.new_page() # 等待下载事件触发 with page.expect_download(timeout=10000) as download_task: page.goto("https://scnv.io/760y") download = download_task.value print(download.url) browser.close()
内容的提问来源于stack exchange,提问作者chhenning
相关产品推荐
相关产品推荐

