使用FFmpeg库通过RTP(C++)传输视频流时丢包问题求助

Hey there! Let's break down the most common causes for your receiver error when sending FLV-extracted image streams via C++, along with the best fixes you can try right away:

1. Mismatch Between SDP Declaration and Stream Encoding Format #

This is the #1 culprit for receiver decoding failures. For example, you might extract H.264 frames from the FLV but your SDP declares H.265, or you didn't include critical encoding metadata (like SPS/PPS for H.264) in the SDP.

• Why it happens: FLV image streams carry specific codec metadata, but your SDP either mislabels the codec or omits required parameters that the receiver needs to initialize its decoder.
• Fixes:
  • First confirm the codec of your FLV stream with a tool like ffprobe:

    ffprobe -v quiet -print_format json -show_streams your_video.flv
    

  • Update your SDP to match the codec: For H.264, set the media type to video/H264 and add the base64-encoded SPS/PPS in the fmtp attribute, e.g.:

    a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z0LAH9kBQBbsCAAADAAgAAAMAAADqGIUK,aM48gA==
    

  • Ensure your sender transmits codec metadata (like SPS/PPS) before sending any image frames—receivers can't decode without this initialization data.

2. Incorrect RTP Packaging of FLV Frames #

FLV stores frames in tag structures; sending raw FLV tag data directly as RTP payload violates RTP packaging rules for video codecs.

• Why it happens: You might be sending the entire FLV tag (including its 11-byte header) instead of extracting just the raw codec frame data, or not splitting/aggregating frames per the codec's RTP spec (e.g., H.264 requires NALU-based packaging per RFC 6184).
• Fixes:
  • For H.264, split FLV AVC tags into individual NALU units, then package them using valid RTP formats: single NALU, STAP-A (aggregated packets), or FU-A (fragmented packets)
  • Match the RTP payload type in your sender code to the value declared in the SDP (e.g., if SDP has m=video 5004 RTP/AVP 96, use payload type 96)
  • Verify RTP header fields: Serial numbers must increment sequentially, and timestamps should align with the frame's PTS (presentation timestamp) to avoid jitter or decoding errors.

3. Invalid Network Configuration in SDP #

Typos or mismatched network parameters in the SDP will prevent the receiver from listening for or interpreting your stream correctly.

• Why it happens: The SDP's IP address (c= field) might not match the receiver's accessible IP, the port (m= field) might differ from what the receiver is listening on, or the SDP has syntax errors missing required fields.
• Fixes:
  • Double-check the c=IN IP4 ... line: Use 127.0.0.1 for local testing, or the receiver's actual LAN/WAN IP for remote streams
  • Ensure the port in the m=video ... line matches the receiver's RTP listening port (note: RTCP usually uses RTP port +1)
  • Validate your SDP syntax with a tool like sdp-validator to catch missing fields (like m= or a= attributes) that violate RFC 4566 standards.

4. Receiver Decoder Initialization Failure #

Even if the stream is sent correctly, the receiver might fail to initialize its decoder if it doesn't process metadata first.

• Why it happens: The receiver tries to decode frames before receiving and parsing codec metadata (SPS/PPS for H.264), or its decoder parameters don't match what's declared in the SDP.
• Fixes:
  • Modify the receiver to wait for and parse metadata packets (SPS/PPS) before attempting to decode image frames
  • Ensure the receiver's decoder context uses parameters matching the SDP (e.g., H.264 profile/level must align with the SPS data)
  • If using FFmpeg for decoding, initialize the decoder context using avcodec_parameters_from_context or by extracting parameters directly from the SDP.

5. Incomplete/Corrupted FLV Frame Extraction #

Your extraction logic might be pulling extra FLV header data or missing critical frame components, leading to unreadable payloads.

• Why it happens: You might not be skipping the FLV file header (first 9 bytes) or tag headers (11 bytes per tag), so your sender is sending non-codec data. Or you're missing keyframes (I-frames) which are required for the receiver to start decoding.
• Fixes:
  • Correct your FLV parsing logic: Skip the 9-byte file header, then for each tag, skip the 11-byte tag header and only send the raw codec frame data from the tag payload
  • Test with a known-good raw stream: Use ffmpeg to extract the raw codec stream:

    ffmpeg -i your_video.flv -vcodec copy -an raw.h264
    

    If sending this raw stream works, your FLV extraction code is the issue.
  • Ensure the first frame you send is an I-frame—receivers can't start decoding from P-frames or B-frames alone.

内容的提问来源于stack exchange，提问作者Alisa456