FFmpeg使用concat拼接视频时第二个视频自动旋转的原因排查

我用以下命令拼接视频：

ffmpeg -f concat -an -i ./files.txt ztestout.mp4

原本要拼接5个视频，测试时只放了两个，结果第二个视频上下翻转了，但单独用VLC打开每个视频都正常。所有视频都是同一安卓手机拍摄的，怀疑是元数据问题？

从FFmpeg输出里看到，输入视频的side data显示有-180.00度旋转，而输出视频的旋转角度是0度：

ffmpeg version 5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with Apple clang version 14.0.0 (clang-1400.0.29.102)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/5.1.2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x138804730] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from './files.txt':
  Duration: N/A, start: 0.000000, bitrate: 17255 kb/s
  Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m, progressive), 1920x1080, 16999 kb/s, SAR 1:1 DAR 16:9, 30.01 fps, 30 tbr, 90k tbn
    Metadata:
      creation_time   : 2023-01-08T13:32:50.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -180.00 degrees
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 256 kb/s
    Metadata:
      creation_time   : 2023-01-08T13:32:50.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x137f07fe0] using SAR=1/1
[libx264 @ 0x137f07fe0] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x137f07fe0] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x137f07fe0] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=15 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'ztestout.mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuvj420p(pc, smpte170m, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 15360 tbn
    Metadata:
      creation_time   : 2023-01-08T13:32:50.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.37.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
      displaymatrix: rotation of -0.00 degrees
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x138904b40] Auto-inserting h264_mp4toannexb bitstream filterdrop=20 speed=3.67x    
frame=74497 fps=109 q=-1.0 Lsize= 1740353kB time=00:41:23.13 bitrate=5741.5kbits/s dup=0 drop=25 speed=3.64x    
video:1739531kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.047236%
[libx264 @ 0x137f07fe0] frame I:300   Avg QP:20.73  size:294110
[libx264 @ 0x137f07fe0] frame P:29265 Avg QP:23.65  size: 45500
[libx264 @ 0x137f07fe0] frame B:44932 Avg QP:27.95  size:  8045
[libx264 @ 0x137f07fe0] consecutive B-frames: 19.4%  0.4%  0.7% 79.5%
[libx264 @ 0x137f07fe0] mb I  I16..4:  5.5% 43.8% 50.7%
[libx264 @ 0x137f07fe0] mb P  I16..4:  0.2%  0.9%  0.7%  P16..4: 36.9% 13.9% 10.8%  0.0%  0.0%    skip:36.6%
[libx264 @ 0x137f07fe0] mb B  I16..4:  0.0%  0.1%  0.1%  B16..8: 30.0%  3.0%  1.3%  direct: 1.1%  skip:64.3%  L0:38.1% L1:55.8% BI: 6.1%
[libx264 @ 0x137f07fe0] 8x8 transform intra:49.7% inter:45.0%
[libx264 @ 0x137f07fe0] coded y,uvDC,uvAC intra: 84.0% 82.3% 41.9% inter: 12.8% 10.8% 0.5%
[libx264 @ 0x137f07fe0] i16 v,h,dc,p: 30% 17%  8% 45%
[libx264 @ 0x137f07fe0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 25% 12%  4%  7%  7% 12%  7%  8%
[libx264 @ 0x137f07fe0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 25% 12%  5%  7%  7% 10%  6%  8%
[libx264 @ 0x137f07fe0] i8c dc,h,v,p: 47% 22% 22%  9%
[libx264 @ 0x137f07fe0] Weighted P-Frames: Y:0.3% UV:0.0%
[libx264 @ 0x137f07fe0] ref P L0: 70.3% 10.7% 13.9%  5.0%  0.0%
[libx264 @ 0x137f07fe0] ref B L0: 91.8%  6.2%  2.0%
[libx264 @ 0x137f07fe0] ref B L1: 95.7%  4.3%
[libx264 @ 0x137f07fe0] kb/s:5738.58

请问这是什么原因导致的？是否是视频元数据有误？

这确实是视频旋转元数据处理不一致导致的问题：

1. 原因分析：

   • 安卓手机拍摄视频时，会把旋转角度（比如这里的-180度）存在视频的side data（显示矩阵）里，而非直接修改视频帧像素。VLC这类播放器会读取这种元数据并自动旋转视频显示，所以单独播放时正常。
   • 你的concat命令中，FFmpeg的concat分离器默认仅读取第一个输入视频的元数据，后续视频的旋转信息会被忽略；同时输出时FFmpeg将旋转角度重置为0度，导致第二个视频的原始帧未被旋转，直接以原始方向输出，看起来就上下翻转了。

2. 解决办法：

   • 方法一：强制FFmpeg根据元数据旋转视频帧，统一输出方向。修改命令如下：

ffmpeg -f concat -an -i ./files.txt -vf "transpose=2,transpose=2" ztestout.mp4

（transpose=2是顺时针旋转90度，两次操作等价于180度旋转，匹配-180度的需求）

• 方法二：先预处理每个视频，将旋转元数据应用到帧上，再进行拼接：
  1. 单个视频处理命令：

ffmpeg -i input.mp4 -vf "rotate=PI" -c:a copy output.mp4

2. 用`concat`命令拼接所有处理后的视频。

内容的提问来源于stack exchange，提问作者Pompey Magnus