FFmpeg技术问询：仅延迟dshow关联输入的视频流以同步音画

Hey there, let’s tackle that audio-video sync headache you’re dealing with. Looking at the partial command you shared, there are several tweaks and fixes we can apply to get your streams aligned properly. Here’s what to try:

1. Optimize Input Queue & Buffer Settings #

Your current command sets thread_queue_size for the combined video/audio input, but splitting out the audio input and giving it its own queue/buffer can prevent underruns that cause sync drift. DShow devices (especially audio interfaces like your RME Fireface) often need dedicated buffer handling to keep up with high-frame-rate video capture.

Adjust your input section to separate video and audio, each with their own queue and buffer:

ffmpeg -y -thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 ^
-f dshow -video_size 3440x1440 -rtbufsize 2147.48M -framerate 100 -pixel_format nv12 ^
-i video="Video (00 Pro Capture HDMI 4K+)" ^
-thread_queue_size 9999 -f dshow -rtbufsize 2147.48M ^
-i audio="SPDIF/ADAT (1+2) (RME Fireface UC)"

2. Reduce Encoder Latency #

You’re using h264_nvenc with the llhp (low-latency high-performance) preset, but we can add extra flags to minimize encoding-side delay, which often contributes to sync issues:

• -rc-lookahead 0: Disables lookahead for the NVENC encoder, cutting down on processing delay
• -delay 0: Ensures the encoder doesn’t introduce intentional latency

Add these to your video encoding parameters.

3. Manual Sync Offset (If Needed) #

If your hardware has inherent latency (e.g., HDMI capture has a slight delay vs. your SPDIF audio), you can manually shift one stream to match the other using -itsoffset. For example, if audio is ahead of video by 0.1 seconds, add the offset to the video input:

-itsoffset 0.1 -i video="Video (00 Pro Capture HDMI 4K+)"

Adjust the value (positive or negative) until sync is perfect.

4. Verify Stream Mapping & Sync Flags #

Make sure you’re mapping the correct streams (video from the first input, audio from the second) and add sync-related flags to help FFmpeg auto-align:

• -vsync 1: Forces video sync to audio
• -async 1: Lets FFmpeg adjust audio timing to match video

Here’s the full adjusted command incorporating all these fixes:

ffmpeg -y -thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 ^
-f dshow -video_size 3440x1440 -rtbufsize 2147.48M -framerate 100 -pixel_format nv12 ^
-i video="Video (00 Pro Capture HDMI 4K+)" ^
-thread_queue_size 9999 -f dshow -rtbufsize 2147.48M ^
-i audio="SPDIF/ADAT (1+2) (RME Fireface UC)" ^
-map 0:0 -map 1:0 -flags +cgop -force_key_frames expr:gte(t,n_forced*2) ^
-c:v h264_nvenc -preset llhp -pix_fmt nv12 -b:v 250M ^
-rc-lookahead 0 -delay 0 ^
-c:a aac -b:a 320k ^
-vsync 1 -async 1 ^
output.mp4

Quick Troubleshooting Tip #

If sync is still off, try capturing video and audio to separate files first (without mixing them), then use FFmpeg to combine them. This will tell you if the issue is happening at capture time or during encoding. For example:

# Capture video only
ffmpeg -f dshow -i video="Video (00 Pro Capture HDMI 4K+)" -c copy raw_video.mp4

# Capture audio only
ffmpeg -f dshow -i audio="SPDIF/ADAT (1+2) (RME Fireface UC)" -c copy raw_audio.wav

# Combine and test sync
ffmpeg -i raw_video.mp4 -i raw_audio.wav -map 0:v -map 1:a -c copy test_sync.mp4

If the combined file is out of sync, the problem is with your hardware setup (check if your capture card and audio interface are synced to the same clock source). If it’s in sync, the issue was with your original encoding command.

内容的提问来源于stack exchange，提问作者ninbura