如何修改FFmpeg多流命令适配音视频单独/混合文件

Let's break down the issues with your original command and fix it to work with your mixed input types (audio-only, video-only, and combined).

First, let's recap your input file types (from ffprobe): #

• Area 1:
  • video1a: Audio-only
  • video1b: Both audio + video
  • video1c: Video-only
• Area 2:
  • video2a: Video-only
  • video2b: Audio-only
• Area 3:
  • video3a: Video-only
  • video3b: Audio-only
• Area 4:
  • video4a: Video-only
  • video4b: Both audio + video

Key Issues in the Original Command #

1. Invalid stream references: You tried to access video streams ([0:v], [5:v], [6:v]) from audio-only files, which don't have video streams—this is the main source of your error.
2. No audio handling: The original command ignores audio entirely, so your output would have no audio (even though several inputs have audio tracks).

Modified Working Command #

ffmpeg -i video1a -i video2a -i video3a -i video4a -i video1b -i video2b -i video3b -i video4b -i video1c \
-filter_complex "
  # Video processing: only use inputs that have video streams
  nullsrc=size=640x480 [base];
  # Area 2 (top-right): video2a (video-only)
  [1:v] setpts=PTS-STARTPTS+300/TB, scale=320x240 [2a];
  # Area 3 (bottom-left): video3a (video-only)
  [2:v] setpts=PTS-STARTPTS+400/TB, scale=320x240 [3a];
  # Area 4 (bottom-right): video4a (video-only)
  [3:v] setpts=PTS-STARTPTS+400/TB, scale=320x240 [4a];
  # Area 1 (top-left): video1b (combined)
  [4:v] setpts=PTS-STARTPTS+2500/TB, scale=320x240 [1b];
  # Area 4 (bottom-right): video4b (combined)
  [7:v] setpts=PTS-STARTPTS+800/TB, scale=320x240 [4b];
  # Area 1 (top-left): video1c (video-only)
  [8:v] setpts=PTS-STARTPTS+3000/TB, scale=320x240 [1c];

  # Overlay all valid video streams onto the base canvas
  [base][1b] overlay=eof_action=pass [o1];
  [o1][1c] overlay=eof_action=pass:shortest=1 [o1];
  [o1][2a] overlay=eof_action=pass:x=320 [o2];
  [o2][3a] overlay=eof_action=pass:y=240 [o3];
  [o3][4a] overlay=eof_action=pass:x=320:y=240 [o4];
  [o4][4b] overlay=eof_action=pass:x=320:y=240 [v_out];

  # Audio processing: mix all valid audio streams
  [0:a][4:a][5:a][6:a][7:a] amix=inputs=5:duration=longest [a_out]
" \
-c:v libx264 -c:a aac -map "[v_out]" -map "[a_out]" output.mp4

What Changed & Why #

1. Removed invalid video stream references: We dropped lines trying to access video streams from video1a ([0:v]), video2b ([5:v]), and video3b ([6:v]) since these files have no video.
2. Added audio mixing: We collect all existing audio streams (from video1a, video1b, video2b, video3b, video4b) and use the amix filter to combine them. The duration=longest ensures the audio track matches the longest video stream's length.
3. Explicit stream mapping: We use -map "[v_out]" and -map "[a_out]" to tell FFmpeg exactly which processed streams to include in the output, avoiding auto-selection issues.
4. Added AAC audio encoding: The original command didn't specify an audio codec—we added -c:a aac to ensure compatibility with the MP4 container.

Troubleshooting Notes #

• If you get an error about missing audio streams: Double-check that the input files match the types you listed (e.g., ensure video1a really has an audio stream).
• If overlays are positioned incorrectly: Adjust the x=/y= values in the overlay filters to match your desired layout.
• If timing is off: Tweak the values in setpts=PTS-STARTPTS+XXX/TB (XXX is milliseconds) to adjust when each video starts playing.

内容的提问来源于stack exchange，提问作者Robert Smith