NextJS应用因大文件上传内存溢出崩溃，求优化方案

你的核心问题是大文件上传时，请求体被完整加载到内存中，导致Docker内存耗尽崩溃。以下是针对性的解决方案：

1. 优化Multer配置，强制流式写入磁盘 #

当前代码用了diskStorage但仍可能存在内存缓存，通过以下配置确保文件直接落地磁盘，不经过内存缓冲：

import type { NextApiRequest, NextApiResponse } from "next";
import { createRouter } from "next-connect";
import multer from "multer";
// 扩展NextApiRequest类型，解决file属性的类型报错，无需//@ts-ignore
declare module 'next' {
  interface NextApiRequest {
    file?: Express.Multer.File;
  }
}

export const config = {
  api: {
    bodyParser: false,
  },
};

const upload = multer({
  storage: multer.diskStorage({
    destination: "./uploads",
    filename(req, file, callback) {
      callback(null, file.originalname);
    },
  }),
  // 设置内存阈值，超过该值直接写入磁盘
  limits: {
    fileSize: 10 * 1024 * 1024 * 1024, // 允许最大10GB文件（按需调整）
    fieldSize: 256, // 限制表单字段大小，减少内存占用
  },
  // 冗余配置确保文件直接写入磁盘
  dest: "./uploads",
});

const uploadMiddleware = upload.single("file");
const router = createRouter<NextApiRequest, NextApiResponse>();

router.use(uploadMiddleware);

router.post(async (req, res) => {
  res.status(200).json({ data: "Success" });
});

export default router.handler();

2. 实现分块上传（断点续传） #

这是解决大文件内存问题的根本方案，将大文件分割为10MB左右的小分片逐个上传，后端接收后合并：

前端Axios示例 #

// 分片上传函数
async function uploadFileChunk(file, chunkIndex, totalChunks, chunkSize = 10 * 1024 * 1024) {
  const start = chunkIndex * chunkSize;
  const end = Math.min(file.size, start + chunkSize);
  const chunk = file.slice(start, end);
  const formData = new FormData();
  formData.append('file', chunk);
  formData.append('chunkIndex', chunkIndex);
  formData.append('totalChunks', totalChunks);
  formData.append('fileName', file.name);
  
  await axios.post('/api/upload-chunk', formData, {
    headers: { 'Content-Type': 'multipart/form-data' },
  });
}

// 触发上传
const file = document.querySelector('input[type="file"]').files[0];
const chunkSize = 10 * 1024 * 1024;
const totalChunks = Math.ceil(file.size / chunkSize);
for (let i = 0; i < totalChunks; i++) {
  await uploadFileChunk(file, i, totalChunks, chunkSize);
}
// 分片上传完成后请求合并文件
await axios.post('/api/merge-chunks', { fileName: file.name, totalChunks });

后端Next.js API路由 #

// pages/api/upload-chunk.ts
import type { NextApiRequest, NextApiResponse } from "next";
import { createRouter } from "next-connect";
import multer from "multer";
import path from "path";

export const config = { api: { bodyParser: false } };

// 临时存储分片
const chunkStorage = multer.diskStorage({
  destination: "./temp-chunks",
  filename(req, file, callback) {
    const { fileName, chunkIndex } = req.body;
    callback(null, `${fileName}.chunk${chunkIndex}`);
  },
});

const uploadChunk = multer({ storage: chunkStorage }).single('file');
const router = createRouter<NextApiRequest, NextApiResponse>();

router.use(uploadChunk);

router.post(async (req, res) => {
  res.status(200).json({ success: true });
});

export default router.handler();

// pages/api/merge-chunks.ts
import type { NextApiRequest, NextApiResponse } from "next";
import fs from "fs";
import path from "path";

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
  const { fileName, totalChunks } = req.body;
  const outputPath = path.join("./uploads", fileName);
  const writeStream = fs.createWriteStream(outputPath);

  // 合并所有分片
  for (let i = 0; i < totalChunks; i++) {
    const chunkPath = path.join("./temp-chunks", `${fileName}.chunk${i}`);
    const chunkData = fs.readFileSync(chunkPath);
    writeStream.write(chunkData);
    fs.unlinkSync(chunkPath); // 删除临时分片
  }

  writeStream.end();
  res.status(200).json({ success: true, filePath: `/uploads/${fileName}` });
}

3. 调整Docker与Node.js内存配置 #

• 限制Node.js堆内存，避免占用过多容器内存：
  在Docker启动命令中添加参数：

  node --max-old-space-size=1536 node_modules/.bin/next start
  

  （设置为1.5GB，预留500MB给系统进程）

• 给Docker容器添加swap空间，防止内存不足直接崩溃：
  在docker-compose.yml中配置：

  services:
    your-nextjs-app:
      image: your-app-image
      deploy:
        resources:
          limits:
            memory: 2g
          reservations:
            memory: 1g
      volumes:
        - ./uploads:/app/uploads
        - ./temp-chunks:/app/temp-chunks
      command: sh -c "dd if=/dev/zero of=/swapfile bs=1M count=2048 && mkswap /swapfile && swapon /swapfile && node --max-old-space-size=1536 node_modules/.bin/next start"
  

4. 基础环境准备 #

确保上传目录存在并拥有写入权限：

mkdir -p ./uploads ./temp-chunks
chmod 755 ./uploads ./temp-chunks # 生产环境使用更严格的权限

内容的提问来源于stack exchange，提问作者Drogo