Node.js+AWS获取MP4中间片段并保留元数据的技术问题

当从AWS S3获取MP4视频的中间片段（start≠0）时，返回内容缺少视频开头的元数据块（ftyp、moov等），HTML视频播放器因无法识别元数据而关闭流。原函数在start=0时正常，但分段获取时失效：

export async function* initiateObjectStream(
  Key: string,
  start: number,
  end: number,
): AsyncGenerator<any, any, unknown> {
  const streamRange = `bytes=${start}-${end}`;

  const getObjectCommand = new GetObjectCommand({
    Bucket: bucket,
    Key,
    Range: streamRange,
  });

  const { Body: chunks} = await s3Client.send(getObjectCommand);

  for await (const chunk of chunks) {
    yield chunk;
  }
}

尝试手动构造元数据块拼接、使用ffmpeg处理流均未成功：

手动构造元数据的失败尝试 #

export async function* initiateObjectStream(
  Key: string,
  start: number,
  end: number,
): AsyncGenerator<any, any, unknown> {
  const streamRange = `bytes=${start}-${end}`;

  const getObjectCommand = new GetObjectCommand({
    Bucket: bucket,
    Key,
    Range: streamRange,
  });

  const { Body: chunks } = await s3Client.send(getObjectCommand);

  //@ts-ignore
  const passThroughStream = new PassThrough();

  const ftypChunk = Buffer.alloc(28);
  ftypChunk.writeUInt32BE(28, 0);
  ftypChunk.write('ftyp', 4);
  ftypChunk.write('mmp4', 8);
  ftypChunk.write('isom', 12);
  ftypChunk.write('iso2', 16);
  ftypChunk.write('mp41', 20);
  ftypChunk.write('mp42', 24);

  const mdatChunk1 = Buffer.alloc(8);
  mdatChunk1.writeUInt32BE(8, 0);
  mdatChunk1.write('mdat', 4);

  const mdatChunk2Size = 303739;
  const mdatChunk2 = Buffer.alloc(8 + mdatChunk2Size, 0x01);
  mdatChunk2.writeUInt32BE(8 + mdatChunk2Size, 0);
  mdatChunk2.write('mdat', 4);

  const moovChunkSize = 6202;
  const moovChunk = Buffer.alloc(8 + moovChunkSize, 0x02);
  moovChunk.writeUInt32BE(8 + moovChunkSize, 0);
  moovChunk.write('moov', 4);

  passThroughStream.write(ftypChunk);
  passThroughStream.write(mdatChunk1);
  passThroughStream.write(mdatChunk2);
  passThroughStream.write(moovChunk);
  passThroughStream.end();

  for await (const chunk of passThroughStream) {
    yield chunk;
  }

  //@ts-ignore
  for await (const chunk of chunks) {
    yield chunk;
  }
}

FFmpeg处理流的失败尝试 #

const ffmpegStream = ffmpeg()
    .input(chunks)
    .format('mp4')
    .addOutputOptions(
      '-movflags +frag_keyframe+separate_moof+omit_tfhd_offset+empty_moov',
    )
    .on('error', function (err) {
      console.log('An error occurred: ' + err.message);
    })
    .on('end', function () {
      console.log('Processing finished !');
    });

const ffstream = ffmpegStream.pipe().on('data', function (chunk) {
  console.log('ffmpeg just wrote ' + chunk.length + ' bytes');
});

解决方案 #

方法1：预提取原文件元数据并拼接 #

MP4的元数据（ftyp+moov块）通常集中在文件开头，先单独获取这部分内容，再和目标片段拼接：

import { GetObjectCommand } from "@aws-sdk/client-s3";

export async function* initiateObjectStream(
  Key: string,
  start: number,
  end: number,
): AsyncGenerator<Buffer, void, unknown> {
  let metadataChunks: Buffer[] = [];

  // 仅在start≠0时获取元数据
  if (start > 0) {
    // 请求前100KB（足够覆盖绝大多数MP4的元数据区域）
    const metadataCmd = new GetObjectCommand({
      Bucket: bucket,
      Key,
      Range: "bytes=0-102400",
    });
    const { Body: metadataStream } = await s3Client.send(metadataCmd);

    // 读取并提取完整的ftyp+moov块
    let buffer = Buffer.from([]);
    for await (const chunk of metadataStream) {
      buffer = Buffer.concat([buffer, chunk]);
      let offset = 0;
      // 遍历MP4块结构：4字节大小 + 4字节类型
      while (offset + 8 <= buffer.length) {
        const chunkSize = buffer.readUInt32BE(offset);
        const chunkType = buffer.toString("ascii", offset + 4, offset + 8);
        if (chunkType === "moov") {
          // 提取从开头到moov块结束的所有内容
          metadataChunks.push(buffer.slice(0, offset + chunkSize));
          buffer = Buffer.from([]);
          break;
        }
        offset += chunkSize;
      }
      if (metadataChunks.length > 0) break;
    }
  }

  // 获取目标片段
  const streamRange = `bytes=${start}-${end}`;
  const contentCmd = new GetObjectCommand({
    Bucket: bucket,
    Key,
    Range: streamRange,
  });
  const { Body: contentStream } = await s3Client.send(contentCmd);

  // 先返回元数据块
  for (const metaChunk of metadataChunks) {
    yield metaChunk;
  }

  // 再返回目标片段内容
  for await (const chunk of contentStream) {
    yield chunk;
  }
}

方法2：用FFmpeg生成可流式的碎片化MP4 #

调整FFmpeg参数，输出支持流式播放的碎片化MP4（fMP4），无需前置完整元数据：

import { GetObjectCommand } from "@aws-sdk/client-s3";
import ffmpeg from "fluent-ffmpeg";
import { PassThrough } from "stream";

export async function* initiateObjectStream(
  Key: string,
  start: number,
  end: number,
): AsyncGenerator<Buffer, void, unknown> {
  // 获取目标片段
  const streamRange = `bytes=${start}-${end}`;
  const contentCmd = new GetObjectCommand({
    Bucket: bucket,
    Key,
    Range: streamRange,
  });
  const { Body: s3Stream } = await s3Client.send(contentCmd);

  const passThrough = new PassThrough();

  // 配置FFmpeg：直接复制流，生成可流式的fMP4
  ffmpeg(s3Stream)
    .inputFormat("mp4")
    .outputFormat("mp4")
    .addOutputOptions([
      "-movflags", "frag_keyframe+empty_moov+faststart",
      "-c:v", "copy",
      "-c:a", "copy",
    ])
    .on("error", (err) => {
      console.error("FFmpeg处理错误:", err);
      passThrough.destroy(err);
    })
    .pipe(passThrough);

  // 流式返回处理后的内容
  for await (const chunk of passThrough) {
    yield chunk;
  }
}

关键说明 #

• MP4的moov块包含视频编码信息、轨道参数等核心元数据，播放器必须先获取才能解析后续媒体内容，手动构造元数据几乎不可行（每个视频的元数据内容都不同）。
• 碎片化MP4（fMP4）将元数据分散到每个媒体片段中，支持播放器边接收边解析，无需完整前置元数据。

内容的提问来源于stack exchange，提问作者turaiev