如何通过URL参数设置伦敦证交所页面的Show500News下拉选项

Hey Daz, sorry to hear your Android scraper stopped working after the LSE site’s dropdown update—let’s break down practical technical directions to get that 500-news view loading automatically again, plus a feasibility check.

1. Reverse-Engineer Network Requests for Hidden Parameters #

The LSE site is almost certainly a dynamic single-page application (SPA), which is why you didn’t see the dropdown code in the static page source. Here’s how to find the right URL parameters:

• Open Chrome DevTools (F12) on the news page
• Switch to the Network tab, filter requests by XHR/Fetch
• Manually select "Show500News" from the dropdown, then watch what request fires to load more results
• Look for parameters in the request URL or body—common candidates are limit=500, pageSize=500, count=500, or something structured like pagination[size]=500
• Your original URL had results=500, but that parameter might have been renamed, or now requires a companion parameter (like a sort or tab value) to work
• Test any new parameters you find by appending them to your original URL to see if it loads 500 results directly

2. Check Browser Storage for Saved Preferences #

Many sites store user preferences like news display count in local storage, session storage, or cookies. After selecting "Show500News":

• Go to DevTools’ Application tab
• Scan Local Storage, Session Storage, and Cookies for keys related to result limits (e.g., news_per_page, lse_results_limit, display_size)
• If you find a key-value pair like news_per_page: "500", you can update your Android app’s HTTP client to send this cookie, or inject code to set the storage value before loading the page in a WebView

3. Automate Dropdown Selection with JavaScript #

If URL parameters and storage tricks don’t work, you can simulate user interaction via JavaScript in your app’s WebView:

• First, find the dropdown’s selector using DevTools’ Elements tab (right-click the dropdown after it renders and select "Inspect")
• Use JS to trigger the selection—here are two common scenarios:

  // For a standard <select> element
  const selectElement = document.querySelector('select[name="resultsPerPage"]');
  selectElement.value = "500";
  selectElement.dispatchEvent(new Event('change'));
  
  // For a custom SPA dropdown
  document.querySelector('.dropdown-toggle').click();
  document.querySelector('.dropdown-item[data-value="500"]').click();
  

• In Android, run this code using webView.evaluateJavascript() once the page finishes loading

This is totally feasible. Almost all modern sites provide a way to control such display preferences—either via URL parameters, browser storage, or frontend interaction. The key is to reverse-engineer how the site tracks the "Show500News" choice. The steps above should help you identify the exact method to automate it.

内容的提问来源于stack exchange，提问作者Daz