WebRTC中x264编码+FFmpeg解码的H.264延迟异常问题求助

From your description, it’s clear that the H.264 pipeline develops significant latency over time (jumping from ~300ms initial to 800-900ms) while VP9 stays consistent at ~500ms. Let’s break down possible causes and actionable checks based on your x264 config and WebRTC’s behavior:

1. x264 Encoder Parameter Tweaks for Low Latency #

Your current config uses veryfast and zerolatency presets, but a few settings might be contributing to growing latency:

• Keyframe Interval: You’ve set i_keyint_max = 40 (8fps * 5s). WebRTC typically recommends keyframe intervals of 2 seconds or less (16 frames at 8fps). Larger keyframes take longer to transmit, trigger more jitter, and force the receiver’s jitter buffer to expand—once expanded, most WebRTC implementations don’t shrink buffers aggressively, leading to persistent high latency.
• Rate Control: The zerolatency preset works best with CQP (Constant Quantization Parameter) mode explicitly enabled. Add this line to lock in consistent quality without variable bitrate fluctuations that can trigger congestion:

  encoder_params.rc.i_rc_method = X264_RC_CQP;
  

• Threading: b_sliced_threads = 1 can introduce overhead in low-latency scenarios. Try disabling sliced threads and setting an explicit thread count matching your CPU cores (e.g., i_threads = 4) to improve encoding throughput and reduce queuing delays.
• Redundant Features: Double-check that no implicit lookahead or post-processing features are enabled—you’ve already disabled lookahead, but adding encoder_params.b_deblocking_filter = 0 can cut down on encoding time further.

2. WebRTC Congestion Control & Jitter Buffer Behavior #

The initial low latency followed by a spike suggests your network or WebRTC stack is reacting to congestion:

• Keyframe-Induced Congestion: Large 5-second keyframes can saturate bandwidth, triggering WebRTC’s GCC (Google Congestion Control) to throttle bitrate and expand the jitter buffer to handle packet loss/jitter. Once the buffer expands, it may not contract even after congestion subsides, leading to sustained high latency.
• Frame Queuing: If your encoding pipeline doesn’t drop old frames when under load, unencoded frames can pile up in a queue, adding end-to-end latency over time. Ensure your WebRTC integration includes logic to drop frames older than a small threshold (e.g., 100ms) before encoding.

3. FFmpeg Decoder Low-Latency Configuration #

Don’t overlook the decoding side—FFmpeg can introduce latency if not optimized:

• Disable frame caching by using flags like -flags low_delay and -async 1 when initializing the decoder to prioritize immediate frame output over smooth playback.
• Match decoder thread count to your CPU cores to avoid bottlenecks that cause frame accumulation in the decoder queue.
• Ensure the decoder isn’t waiting for redundant packets or reordering frames unnecessarily—WebRTC already handles packet reordering, so FFmpeg should decode frames as they arrive.

4. Diagnostic Steps to Pinpoint the Root Cause #

To narrow down exactly where latency is building:

• Packet Capture: Use Wireshark to analyze H.264 traffic. Look for large keyframe packets, retransmissions, or increasing RTT over time. Compare this to VP9 traffic to spot differences in packet size/interval.
• Queue Monitoring: Add logging in your encoding/decoding pipelines to track pending frame counts. If the encoding queue grows over time, the issue is sender-side; if the decoder queue grows, focus on the receiver.
• WebRTC Stats: Check built-in stats like jitterBufferDelay and jitterBufferEmittedCount to see if the jitter buffer expands and stays large after the initial phase.

内容的提问来源于stack exchange，提问作者citygenius