带分页HTML表格全量导出CSV需求及代码问题咨询

Got it, let's fix this so you can export all 200 pages instead of just the visible one. Your current code works great for the current page, but we need to automate page navigation, wait for each page's data to load, and combine all that data into one CSV. Here's how to do it:

Step 1: Refactor Your Code to Reuse Table Data Extraction #

First, let's pull out the table-scraping logic into a separate function so we can call it for every page:

// Extracts data from the currently visible table
function getCurrentPageTableData() {
    const rows = document.querySelectorAll("table tr");
    const pageData = [];

    for (let i = 0; i < rows.length; i++) {
        const row = [];
        const cols = rows[i].querySelectorAll("td, th");
        
        for (let j = 0; j < cols.length; j++) {
            // Handle text with commas (wrap in quotes to avoid CSV formatting issues)
            const cellText = cols[j].innerText.trim();
            row.push(cellText.includes(",") ? `"${cellText}"` : cellText);
        }
        
        pageData.push(row.join(","));
    }

    return pageData;
}

Step 2: Build the Full Export Function #

Next, we'll create an async function that cycles through pages 1 to 200, waits for each page to load, grabs its data, and builds the full CSV. You'll need to adjust the page-navigation part to match how your site handles pagination (I'll include common examples below):

// Export all pages (1 to 200) to a single CSV
async function exportAllPagesToCSV(filename) {
    let fullCsvData = [];
    const totalPages = 200;

    for (let pageNum = 1; pageNum <= totalPages; pageNum++) {
        try {
            // --------------------------
            // 🔧 UPDATE THIS PART FOR YOUR PAGINATION
            // --------------------------
            // Example 1: If using a page number input field
            // const pageInput = document.getElementById("page-number-input");
            // pageInput.value = pageNum;
            // pageInput.dispatchEvent(new Event("change")); // Trigger page load

            // Example 2: If using clickable page buttons
            // const pageButton = document.querySelector(`.pagination-links button[data-page="${pageNum}"]`);
            // if (pageButton) pageButton.click();

            // Example 3: If data loads via API (better for performance!)
            // const response = await fetch(`/api/data?page=${pageNum}`);
            // const pageJson = await response.json();
            // Convert JSON to CSV rows here instead of scraping the table
            // --------------------------

            // Wait for the page to load its data (adjust timeout if needed)
            // For better reliability, replace this with a listener for table load completion
            await new Promise(resolve => setTimeout(resolve, 1200));

            // Grab data from the current page
            const pageData = getCurrentPageTableData();

            // Add header only once (from the first page)
            if (pageNum === 1) {
                fullCsvData = [...fullCsvData, ...pageData];
            } else {
                // Skip header for subsequent pages
                fullCsvData = [...fullCsvData, ...pageData.slice(1)];
            }

            console.log(`Page ${pageNum} data collected`);
        } catch (error) {
            console.error(`Failed to collect page ${pageNum}:`, error);
            // Keep going even if one page fails
            continue;
        }
    }

    // Export the combined CSV
    downloadCSV(fullCsvData.join("\n"), filename);
}

// Updated download function with memory cleanup
function downloadCSV(csv, filename) {
    const csvBlob = new Blob([csv], { type: "text/csv;charset=utf-8;" });
    const downloadLink = document.createElement("a");
    const blobUrl = window.URL.createObjectURL(csvBlob);

    downloadLink.href = blobUrl;
    downloadLink.download = filename;
    downloadLink.style.display = "none";

    document.body.appendChild(downloadLink);
    downloadLink.click();
    document.body.removeChild(downloadLink);
    window.URL.revokeObjectURL(blobUrl); // Free up memory
}

Key Notes to Make This Work: #

• Pagination Logic: The most critical part is updating the page-navigation code to match your site. If your table loads data via an API, skip scraping the table entirely—call the API directly for each page, convert the JSON to CSV rows, and combine them. This is faster and more reliable.
• Wait Time: The setTimeout(1200) is a simple way to wait for the page to load, but for production, you should replace it with a listener that detects when the table has finished updating (e.g., watching for changes to the table's tr elements).
• CSV Formatting: We added logic to wrap text with commas in quotes—this prevents the CSV from splitting those cells into multiple columns.
• Error Handling: The function will keep running even if one page fails to load, so you won't lose all your progress if a page times out.

To use this, just call exportAllPagesToCSV("full-table-data.csv") from your button click event!

内容的提问来源于stack exchange，提问作者Danny76