如何通过Nginx服务器实现浏览器扩展提取非当前URL的HTML字符串

Alright, since you’ve already got your AWS Nginx server up and running, let’s walk through the exact steps to make your extension pull HTML from inactive tabs. The key here is using your server as a reverse proxy to get around browser cross-origin restrictions—you’re already halfway there!

Step 1: Configure Nginx as a Reverse Proxy with CORS Support #

First, you need to set up Nginx to forward requests from your extension to the target URLs, while also sending the right CORS headers so your extension can access the response.

Update your Nginx config (usually at /etc/nginx/sites-available/your-domain.conf) with this location block:

server {
    listen 443 ssl; # Use HTTPS—critical for modern extensions
    server_name your-aws-domain.com;

    # Add SSL certs here (use Let's Encrypt for free ones)
    ssl_certificate /path/to/your/cert.pem;
    ssl_certificate_key /path/to/your/key.pem;

    location /fetch-html {
        # Forward the request to the target URL passed as a query parameter
        proxy_pass $arg_url;
        
        # Allow your extension to access this endpoint
        add_header Access-Control-Allow-Origin "chrome-extension://YOUR_CHROME_EXTENSION_ID";
        # For Firefox, add this line too (swap in your Firefox extension ID)
        # add_header Access-Control-Allow-Origin "moz-extension://YOUR_FIREFOX_EXTENSION_ID";
        
        add_header Access-Control-Allow-Methods "GET, OPTIONS";
        add_header Access-Control-Allow-Headers "Content-Type";
    }
}

• Replace your-aws-domain.com with your actual AWS server domain.
• Swap in your extension IDs: In Chrome, enable Developer Mode in the Extensions page to see your ID; Firefox shows it in the Add-ons Manager under "Debug Add-ons".
• Test the config with sudo nginx -t, then restart Nginx with sudo systemctl restart nginx.

Step 2: Build the Extension Logic #

Let’s use Manifest V3 (the latest standard for Chrome/Firefox) for this example.

Manifest.json #

First, define permissions and your extension’s basic info:

{
    "manifest_version": 3,
    "name": "Inactive Tab HTML Fetcher",
    "version": "1.0",
    "permissions": ["tabs"],
    "host_permissions": ["https://your-aws-domain.com/*"],
    "action": {
        "default_popup": "popup.html"
    }
}

Popup.html #

Create a simple popup with a button to trigger the fetch:

<!DOCTYPE html>
<html>
<body style="width: 300px; padding: 1rem;">
    <button id="fetchBtn" style="width: 100%; padding: 0.5rem; cursor: pointer;">Fetch Inactive Tabs HTML</button>
    <div id="results" style="margin-top: 1rem;"></div>
    <script src="popup.js"></script>
</body>
</html>

Popup.js #

Add the logic to grab inactive tabs, send requests to your proxy, and display results:

document.getElementById('fetchBtn').addEventListener('click', async () => {
    const resultsDiv = document.getElementById('results');
    resultsDiv.innerHTML = '<p>Fetching HTML...</p>';

    // Get all tabs that aren't currently active
    const inactiveTabs = await chrome.tabs.query({ active: false });

    resultsDiv.innerHTML = '';

    for (const tab of inactiveTabs) {
        // Skip browser internal pages (like chrome://settings) since they can't be fetched
        if (!tab.url || tab.url.startsWith('chrome://') || tab.url.startsWith('moz-extension://')) {
            continue;
        }

        try {
            // Encode the target URL to pass as a query parameter
            const encodedUrl = encodeURIComponent(tab.url);
            const proxyUrl = `https://your-aws-domain.com/fetch-html?url=${encodedUrl}`;

            // Fetch HTML via your Nginx proxy
            const response = await fetch(proxyUrl);
            const html = await response.text();

            // Display a snippet of the HTML (customize this to extract your target string)
            const resultItem = document.createElement('div');
            resultItem.style.marginBottom = '1rem';
            resultItem.innerHTML = `
                <strong>Tab URL:</strong> ${tab.url}<br>
                <strong>HTML Snippet:</strong> ${html.substring(0, 400)}...
                <hr>
            `;
            resultsDiv.appendChild(resultItem);
        } catch (err) {
            // Handle errors (e.g., target site blocks the proxy, network issues)
            const errorItem = document.createElement('div');
            errorItem.style.color = 'red';
            errorItem.textContent = `Failed to fetch ${tab.url}: ${err.message}`;
            resultsDiv.appendChild(errorItem);
        }
    }
});

Key Notes to Avoid Headaches #

• HTTPS is mandatory: Modern browsers block HTTP requests from HTTPS extension pages (which Manifest V3 uses by default). Make sure your Nginx server has a valid SSL certificate.
• CORS restrictions: Don’t use Access-Control-Allow-Origin: * in production—lock it down to your specific extension IDs to prevent misuse.
• Anti-scraping measures: Some sites block requests from servers. If you hit issues, add proxy headers to mimic a browser request (e.g., proxy_set_header User-Agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"; in your Nginx location block).
• Firefox compatibility: Replace chrome.tabs.query with browser.tabs.query if you need full Firefox support, or use a polyfill.

内容的提问来源于stack exchange，提问作者Foobar