Hey there! Let's figure out why your auto-click script is loading the first page repeatedly, and how to fix it to get the full content for your date range, plus some alternative methods to make this easier.

Why Your Original Script Isn't Working #

The issue with your setInterval code is that it clicks the button every second regardless of whether the previous content has finished loading. Many sites have safeguards against rapid repeated clicks—either they ignore extra requests, or the pagination state hasn't updated yet, so you end up getting the same first page over and over. Also, sometimes after loading new content, the button might get re-rendered (even if the ID stays the same), making your clicks less reliable.

Fixed Button-Click Script #

Here's an improved script that checks if the button is actually ready to be clicked, and waits for content to load before the next action:

let autoLoadTimer;

function loadMoreContent() {
  const loadButton = document.getElementById("paginationShowMoreText");
  
  // Check if the button exists, isn't disabled, and isn't in a loading state
  if (!loadButton || loadButton.disabled || loadButton.textContent.includes("加载中")) {
    clearTimeout(autoLoadTimer);
    console.log("所有内容已加载完成！停止自动点击。");
    saveFullPageHTML();
    return;
  }
  
  // Click the button only if it's ready
  loadButton.click();
}

function startAutoLoading() {
  autoLoadTimer = setTimeout(function autoClickLoop() {
    loadMoreContent();
    // Adjust the delay based on how fast the site loads content (1.5s is a starting point)
    autoLoadTimer = setTimeout(autoClickLoop, 1500);
  }, 1500);
}

function stopAutoLoading() {
  clearTimeout(autoLoadTimer);
  saveFullPageHTML();
}

function saveFullPageHTML() {
  // Grab the entire page HTML
  const fullHTML = document.documentElement.outerHTML;
  // Create a download link to save it
  const blob = new Blob([fullHTML], { type: "text/html" });
  const downloadURL = URL.createObjectURL(blob);
  const downloadLink = document.createElement("a");
  
  downloadLink.href = downloadURL;
  downloadLink.download = "完整分析页面.html";
  downloadLink.click();
  
  // Clean up the temporary URL
  URL.revokeObjectURL(downloadURL);
}

// Start the auto-loader
startAutoLoading();
// Optional: Stop after a fixed time (e.g., 5 minutes) in case the button never disappears
setTimeout(stopAutoLoading, 5 * 60 * 1000);

This script uses setTimeout instead of setInterval to ensure we only click after the previous load might have finished, and it checks the button's state to avoid useless clicks.

Alternative Methods to Load & Save Full Content #

If the button-based approach still gives you trouble, try these options:

• Simulate Scroll-to-Bottom Loading
  Some sites load new content when you scroll to the bottom instead of clicking a button. Use this scroll-based script:

  function autoScrollToLoad() {
    const currentScrollHeight = document.body.scrollHeight;
    // Scroll to the bottom of the page
    window.scrollTo(0, currentScrollHeight);
    
    setTimeout(() => {
      const newScrollHeight = document.body.scrollHeight;
      // If the scroll height didn't change, we've reached the end
      if (currentScrollHeight === newScrollHeight) {
        console.log("已滚动到页面底部！开始保存页面。");
        saveFullPageHTML();
        return;
      }
      // Keep scrolling if new content loaded
      autoScrollToLoad();
    }, 1500);
  }
  
  // Start scrolling
  autoScrollToLoad();
  

• Use Browser Extensions (No Code Needed)
  Extensions like SingleFile or Scraper can automatically scroll through the entire page and save a complete HTML copy. Just set your date range, run the extension, and it handles all the waiting and loading logic for you.

• Fetch Data Directly via API
  For a more reliable approach, check if the site uses an API to load analytics data. Open your browser's DevTools (F12) → Network tab, then click the "Show More" button. Look for XHR/fetch requests that load new data. You can replicate these requests (with your date range parameters) using tools like curl or Postman, then parse the JSON response into your preferred format. This avoids page rendering issues and is faster than scrolling.

Final Tips #

The key to avoiding repeated first-page loads is waiting for content to finish loading before triggering the next action. If the button-based script still doesn't work, check if the button's class or text changes when loading (e.g., becomes "加载中") and adjust the condition in loadMoreContent() to match that.

内容的提问来源于stack exchange，提问作者backlog