Selenium技术咨询:无需点击JS调用链接获取最终href/链接
Absolutely, you can get those target PDF URLs without triggering downloads or clicking the links—JavaScript-powered links almost always leave clues we can exploit. Here are the most common approaches, ordered by ease of implementation:
Check for hidden data attributes
Many sites store the actual URL in a dedicated data attribute instead of putting it directly in thehref. Look for attributes likedata-url,data-pdf,data-src, or evendata-id(which might be used to build the URL). For example, if your element looks like this:<a href="javascript:;" class="pdf-link" data-pdf="/assets/reports/Q3_2024.pdf">Download Report</a>You can grab the URL directly with:
pdf_url = element.get_attribute('data-pdf')This is the simplest case—always check for these first using your browser's dev tools (Inspect Element).
Parse the JavaScript function call
If thehreforonclickattribute contains a function call with the URL as an argument, you can extract it with string manipulation or regex. For example, if the element is:<a href="javascript:openPDF('/assets/reports/Q3_2024.pdf')">Download Report</a>Grab the
hrefvalue, then use regex to pull out the URL:import re js_code = element.get_attribute('href') # Match single-quoted strings inside the function call url_match = re.search(r"'(https?://.*?|/.*?)'", js_code) if url_match: pdf_url = url_match.group(1)Adjust the regex if the URL uses double quotes instead of single quotes.
Extract from global JavaScript state
Modern sites often load data (including PDF URLs) into a global variable in the page's<script>tags—look for variables likewindow.__DATA__,window.__INITIAL_STATE__, or something similar. For example, you might find a script block like:window.__PDF_REPORTS__ = [ {id: 123, url: "/assets/reports/Q3_2024.pdf", name: "Q3 2024 Report"}, {id: 124, url: "/assets/reports/Q4_2024.pdf", name: "Q4 2024 Report"} ]You can use BeautifulSoup to find this script tag, extract its content, parse it as JSON (or convert the JS object to JSON), and then map the URLs to their corresponding table rows.
Replicate the JavaScript URL-building logic
If the URL is generated dynamically (e.g., using an element's ID or a hidden value to build the path), inspect the JavaScript function in your browser's dev tools (Sources tab) to see how it constructs the URL. For example, if the function looks like:function downloadReport(reportId) { const baseUrl = "/assets/reports/"; window.location.href = baseUrl + reportId + ".pdf"; }You can grab the
reportIdfrom the element (maybe adata-idattribute) and build the URL yourself:report_id = element.get_attribute('data-id') pdf_url = f"/assets/reports/{report_id}.pdf"
Pro tip: Use your browser's Network tab to monitor what happens when you do click a link (in a private window, so you don't actually download the file) — this can reveal the exact URL being requested, which will help you reverse-engineer how to get it without clicking.
内容的提问来源于stack exchange,提问作者Stanford Wong

