React Native流式读取本地大视频至内存以上传AWS S3的解决方案求助

问题背景 #

开发React Native应用时，需要支持上传500MB-1GB的大视频文件，但移动端内存有限。目前采用Blob切片+FileReader分块读取的方式，运行一段时间后应用仍会崩溃，核心需求是实现从磁盘流式读取本地视频至内存，配合AWS S3 SDK完成上传。

现有实现代码 #

1. Expo选择视频 #

const pickVideo = async () => {
  let result = await ImagePicker.launchImageLibraryAsync({
    mediaTypes: ImagePicker.MediaTypeOptions.Videos,
    aspect: [4, 3],
    quality: 1,
  });

  handleVideoPicked(result);
};

2. 通过fetch获取Blob对象 #

const fetchVideoFromUri = async (uri: any) => {         
  const response = await fetch(uri);        
  const blob = await response.blob();         
  return blob;     
};

3. 分块读取尝试 #

const chunk = file.slice(start, chunkEnd);
const reader = new FileReader();

reader.loadend = () => {
  // 处理读取结果
  reader.result
}
reader.readAsDataURL(chunk);

核心问题分析 #

• 使用fetch(uri).blob()会将整个视频文件加载到内存，即使后续切片，初始的Blob已经占用了大量内存，这是崩溃的主要原因。
• FileReader在处理多个chunk时，若未及时释放引用或清理内存，容易导致内存堆积。

解决方案 #

1. 改用Expo FileSystem流式读取文件片段 #

避免加载整个文件到内存，直接从磁盘读取指定范围的字节：

import * as FileSystem from 'expo-file-system';

// 读取指定范围的文件内容（转换为ArrayBuffer适配AWS SDK）
const readFileChunk = async (uri: string, start: number, end: number) => {
  const fileInfo = await FileSystem.getInfoAsync(uri);
  if (!fileInfo.exists) throw new Error('文件不存在');

  const chunkSize = end - start;
  const base64Chunk = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
    position: start,
    length: chunkSize,
  });

  return Buffer.from(base64Chunk, 'base64');
};

2. 配合AWS S3分段上传（Multipart Upload） #

AWS S3支持分段上传，将大文件拆分为多个部分（推荐100MB左右/段），逐个上传后合并：

import { S3Client, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, AbortMultipartUploadCommand } from '@aws-sdk/client-s3';

const s3Client = new S3Client({
  region: '你的AWS区域',
  credentials: {
    accessKeyId: '你的AccessKey',
    secretAccessKey: '你的SecretKey',
  },
});

const uploadLargeVideoToS3 = async (videoUri: string, bucketName: string, key: string) => {
  const fileInfo = await FileSystem.getInfoAsync(videoUri);
  const fileSize = fileInfo.size || 0;
  const chunkSize = 100 * 1024 * 1024; // 100MB每段
  const partCount = Math.ceil(fileSize / chunkSize);
  const uploadParts = [];

  // 初始化分段上传
  const createUploadCmd = new CreateMultipartUploadCommand({
    Bucket: bucketName,
    Key: key,
    ContentType: 'video/mp4', // 根据实际视频类型调整
  });
  const createUploadResult = await s3Client.send(createUploadCmd);
  const uploadId = createUploadResult.UploadId;

  try {
    // 逐个上传分段
    for (let partNumber = 1; partNumber <= partCount; partNumber++) {
      const start = (partNumber - 1) * chunkSize;
      const end = Math.min(start + chunkSize, fileSize);
      const chunkBuffer = await readFileChunk(videoUri, start, end);

      // 上传当前分段
      const uploadPartCmd = new UploadPartCommand({
        Bucket: bucketName,
        Key: key,
        UploadId: uploadId,
        PartNumber: partNumber,
        Body: chunkBuffer,
      });
      const uploadPartResult = await s3Client.send(uploadPartCmd);

      uploadParts.push({
        PartNumber: partNumber,
        ETag: uploadPartResult.ETag,
      });

      // 主动释放内存
      chunkBuffer.fill(0);
      delete chunkBuffer;
    }

    // 完成分段上传
    const completeUploadCmd = new CompleteMultipartUploadCommand({
      Bucket: bucketName,
      Key: key,
      UploadId: uploadId,
      MultipartUpload: { Parts: uploadParts },
    });
    await s3Client.send(completeUploadCmd);

    console.log('视频上传完成');
  } catch (error) {
    // 上传失败时中止分段上传，避免残留无效分段
    const abortUploadCmd = new AbortMultipartUploadCommand({
      Bucket: bucketName,
      Key: key,
      UploadId: uploadId,
    });
    await s3Client.send(abortUploadCmd);
    throw error;
  }
};

3. 内存优化细节 #

• 每次上传完分段后，主动清空Buffer并删除引用，帮助RN垃圾回收机制释放内存。
• 避免在循环中保留不必要的变量引用或创建大量闭包。
• 建议使用AWS SDK v3版本，其流式处理逻辑更轻量高效。

内容的提问来源于stack exchange，提问作者Yoel