React应用刷新触发多API调用及Puppeteer重复执行问题求助

我在React应用中遇到一个问题：刷新浏览器时，前端会向后端发起多次API调用，触发Puppeteer脚本重复执行，给后端带来额外负载，拖慢了应用速度。

我的技术栈：

• React函数组件，通过空依赖数组的useEffect在组件挂载时发起API请求
• API请求触发后端Puppeteer脚本，启动无头浏览器执行爬取任务

预期刷新时API仅调用一次，但实际会发送多个请求，且该问题在不同浏览器和设备上均存在。相关代码如下：

前端代码（src/components/NewsFeed.jsx） #

import { useState, useEffect } from 'react';
import { getNews } from '../../apiClient/apiClient';
import Cardcomp from './Card';

const NewsFeed = () => {
    const [news, setNews] = useState({});
    const [error, setError] = useState(null);
    const [isLoading, setIsLoading] = useState(false);

    useEffect(() => {
        console.log('Fetching news on component mount');
        setIsLoading(true);
        const fetchNews = async () => {
            try {
                const response = await getNews();
                setNews(response);
                setIsLoading(false);
            } catch (error) {
                setError(error.message);
                setIsLoading(false);
            }
        };
    
        fetchNews();
    }, []); 
    
    if (error) {
        return <div>Error: {error}</div>;
    }
    
    if (isLoading) {
        return <div>Loading...</div>;
    }
    
    if (!news.coindesk || !news.dailycoin) {
        return null; // Return null to prevent rendering the card component
    }
    
    return (
        <div>
            <h2>News Feed</h2>
            <ul>
               <li>
                    <Cardcomp name="CoinDes" title={news.coindesk.title} para={news.coindesk.para} />
                </li>
                <li>
                    <Cardcomp name="DailyCoin" title={news.dailycoin.title} para={news.dailycoin.para} />
                </li>
            </ul>
        </div>
    );
   
};

export default NewsFeed;

后端代码（index.js） #

const express = require('express');
const cors = require('cors');
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
const { scrapeCoinDesk, scrapeDailyCoin } = require('./script.js');

puppeteer.use(StealthPlugin());

const app = express();
const port = 3000;


app.use(cors());

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.get('/scrape', async (req, res) => {
    console.log('Scrape API called');
    try {
        const browser = await puppeteer.launch({ headless: false });

        const results = await Promise.all([
            scrapeCoinDesk(browser),
            scrapeDailyCoin(browser),
        ]);

        await browser.close();

        res.json({
            coindesk: results[0],
            dailycoin: results[1],
        });
    } catch (error) {
        console.error('Error during scraping:', error);
        res.status(500).json({ error: 'Failed to scrape data' });
    }
});


app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
});

爬取脚本（server/script.js） #

const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');

puppeteer.use(StealthPlugin());

async function scrapeCoinDesk(browser) {
    const page = await browser.newPage();
    await page.goto('https://www.coindesk.com/', { timeout: 60000 });
    await page.waitForSelector('.featured-cardstyles__FeaturedCardWrapper-sc-caozbq-2 .card-title h3');
    const title = await page.$eval('.featured-cardstyles__FeaturedCardWrapper-sc-caozbq-2 .card-title h3', el => el.textContent.trim());
    await page.waitForSelector('.featured-cardstyles__FeaturedCardWrapper-sc-caozbq-2 .featured-cardstyles__FeaturedCardDescription-sc-caozbq-0 p');
    const para = await page.$eval('.featured-cardstyles__FeaturedCardWrapper-sc-caozbq-2 .featured-cardstyles__FeaturedCardDescription-sc-caozbq-0 p', el => el.textContent.trim());
    await page.close();
    return { title, para };
}

async function scrapeDailyCoin(browser) {
    const page = await browser.newPage();
    await page.goto('https://dailycoin.com/', { timeout: 60000 });
    await page.waitForSelector('.post-content-wrapper .dc-title-link');
    const title = await page.$eval('.post-content-wrapper .dc-title-link', el => el.textContent.trim());
    await page.waitForSelector('.post-content-wrapper .post-excerpt p');
    const para = await page.$eval('.post-content-wrapper .post-excerpt p', el => el.textContent.trim());
    await page.close();
    return { title, para };
}
module.exports = { scrapeCoinDesk, scrapeDailyCoin  };

问题根源排查 #

1. React开发环境Strict Mode双重渲染 #

这是最常见的原因：React开发环境下，Strict Mode会刻意执行两次组件挂载、useEffect等生命周期方法，用于检测潜在bug（比如未清理的副作用）。你的useEffect是空依赖数组，在开发环境中会被触发两次，直接导致两次API调用和Puppeteer执行。

2. 组件被多次挂载 #

检查NewsFeed组件是否在应用的多个位置被渲染，或者路由配置是否导致组件在刷新时重复挂载。

3. 未处理的重复请求触发 #

如果getNews函数内部没有防重复逻辑，或者组件状态更新引发意外重渲染，也可能导致重复请求。

解决方案 #

1. 临时关闭Strict Mode（开发环境） #

若只是想在开发阶段避免该问题，可以移除index.js中的<React.StrictMode>包裹。注意生产环境下Strict Mode不会触发双重渲染，上线前可重新添加。

2. 给API请求添加防重复逻辑 #

在useEffect中增加请求状态标记，避免同一时间发起多个请求，同时防止组件卸载后更新状态引发内存泄漏：

useEffect(() => {
    let isMounted = true;
    if (isLoading) return; // 正在请求时直接返回

    console.log('Fetching news on component mount');
    setIsLoading(true);
    const fetchNews = async () => {
        try {
            const response = await getNews();
            if (isMounted) { // 确保组件仍挂载时再更新状态
                setNews(response);
                setIsLoading(false);
            }
        } catch (error) {
            if (isMounted) {
                setError(error.message);
                setIsLoading(false);
            }
        }
    };

    fetchNews();

    return () => {
        isMounted = false; // 组件卸载时标记
    };
}, []);

3. 后端添加缓存机制（核心优化） #

Puppeteer爬取成本极高，每次请求都启动浏览器会严重拖慢后端响应。建议添加缓存，设定过期时间（比如5分钟），相同请求直接返回缓存结果：

// 后端index.js中添加内存缓存
let cachedData = null;
let cacheExpiry = 0;
const CACHE_DURATION = 5 * 60 * 1000; // 5分钟

app.get('/scrape', async (req, res) => {
    console.log('Scrape API called');
    const now = Date.now();
    // 检查缓存是否有效
    if (cachedData && now < cacheExpiry) {
        return res.json(cachedData);
    }

    try {
        const browser = await puppeteer.launch({ headless: false });
        const results = await Promise.all([
            scrapeCoinDesk(browser),
            scrapeDailyCoin(browser),
        ]);
        await browser.close();

        const data = {
            coindesk: results[0],
            dailycoin: results[1],
        };
        // 更新缓存
        cachedData = data;
        cacheExpiry = now + CACHE_DURATION;

        res.json(data);
    } catch (error) {
        console.error('Error during scraping:', error);
        res.status(500).json({ error: 'Failed to scrape data' });
    }
});

4. 检查组件挂载情况 #

确认NewsFeed组件仅在应用中渲染一次，排查路由配置或父组件的渲染逻辑，避免组件重复挂载。

内容的提问来源于stack exchange，提问作者shyam sharma