实时解码MJPEG至YUV/RGB技术求助：摄像头直播视频处理

Great to hear you've already got the OpenGL rendering, YUV→RGB conversion, and MJPEG pointer retrieval sorted out! Let's tackle the real-time MJPEG decoding with libjpeg—since you already have the library, this should be straightforward, with a focus on buffer safety and efficient output to RGB/YUV (including YUV422).

Core libjpeg Decoding Flow (MJPEG → RGB/YUV) #

Libjpeg works with memory-based streams perfectly for real-time use cases. We'll set up a custom error handler (critical for avoiding crashes on malformed frames), feed the MJPEG pointer directly into the decoder, and handle buffer scaling to prevent out-of-bounds access.

Step 1: Set Up Custom Error Handling #

Libjpeg's default error handler aborts the program, which is bad for real-time streams. We'll override it to gracefully recover:

#include <jpeglib.h>
#include <setjmp.h>

struct my_error_mgr {
  struct jpeg_error_mgr pub;
  jmp_buf setjmp_buffer;
};

METHODDEF(void) my_error_exit(j_common_ptr cinfo) {
  struct my_error_mgr* myerr = (struct my_error_mgr*)cinfo->err;
  // Print error message (optional but helpful for debugging)
  (*cinfo->err->output_message)(cinfo);
  // Jump back to our error recovery point
  longjmp(myerr->setjmp_buffer, 1);
}

Step 2: Real-Time MJPEG to RGB Decoder #

This function takes your MJPEG data pointer, decodes it to RGB, and handles buffer scaling to avoid out-of-bounds issues. Adjust the scale_num/scale_denom values to shrink the output as needed:

int decode_mjpeg_to_rgb(unsigned char* mjpeg_data, size_t mjpeg_len, 
                        unsigned char** rgb_buf, int* out_width, int* out_height) {
  struct jpeg_decompress_struct cinfo;
  struct my_error_mgr jerr;

  // Initialize error handler
  cinfo.err = jpeg_std_error(&jerr.pub);
  jerr.pub.error_exit = my_error_exit;
  // Set up jump point for error recovery
  if (setjmp(jerr.setjmp_buffer)) {
    jpeg_destroy_decompress(&cinfo);
    if (*rgb_buf) free(*rgb_buf);
    *rgb_buf = NULL;
    return -1; // Indicate failure
  }

  // Initialize decoder
  jpeg_create_decompress(&cinfo);
  // Feed MJPEG data from memory (no file I/O needed)
  jpeg_mem_src(&cinfo, mjpeg_data, mjpeg_len);

  // Read JPEG header to get frame dimensions
  (void)jpeg_read_header(&cinfo, TRUE);

  // **Prevent buffer overflow: Scale the output**
  // Example: Shrink to 1/2 the original size (adjust as needed)
  cinfo.scale_num = 1;
  cinfo.scale_denom = 2;

  // Set output color space to RGB
  cinfo.out_color_space = JCS_RGB;

  // Start decoding
  (void)jpeg_start_decompress(&cinfo);

  // Calculate output buffer size
  *out_width = cinfo.output_width;
  *out_height = cinfo.output_height;
  int row_stride = cinfo.output_width * cinfo.output_components; // 3 bytes per RGB pixel
  *rgb_buf = (unsigned char*)malloc(row_stride * cinfo.output_height);
  if (!*rgb_buf) {
    jpeg_abort_decompress(&cinfo);
    jpeg_destroy_decompress(&cinfo);
    return -1;
  }

  // Decode line by line (efficient for real-time)
  unsigned char* row_ptr = *rgb_buf;
  while (cinfo.output_scanline < cinfo.output_height) {
    (void)jpeg_read_scanlines(&cinfo, &row_ptr, 1);
    row_ptr += row_stride;
  }

  // Cleanup
  (void)jpeg_finish_decompress(&cinfo);
  jpeg_destroy_decompress(&cinfo);

  return 0; // Success
}

Step 3: Decode Directly to YUV (or Convert to YUV422) #

If you prefer YUV output instead of RGB, adjust the color space setting:

// Set output to YUV420 (libjpeg's native YUV format)
cinfo.out_color_space = JCS_YCbCr;

Libjpeg outputs YUV420 by default, but if you need YUV422, add a lightweight conversion step after decoding:

void yuv420_to_yuv422(unsigned char* yuv420, unsigned char* yuv422, int width, int height) {
  int y_size = width * height;
  int uv_size = y_size / 4;

  unsigned char* y = yuv420;
  unsigned char* u = yuv420 + y_size;
  unsigned char* v = yuv420 + y_size + uv_size;

  int idx = 0;
  for (int i = 0; i < height; i++) {
    for (int j = 0; j < width; j += 2) {
      // Copy two Y pixels
      yuv422[idx++] = y[i * width + j];
      yuv422[idx++] = y[i * width + j + 1];
      // Copy corresponding U/V pair
      int uv_idx = (i / 2) * (width / 2) + (j / 2);
      yuv422[idx++] = u[uv_idx];
      yuv422[idx++] = v[uv_idx];
    }
  }
}

Real-Time Optimization Tips #

• Reuse the decoder: Instead of creating/destroying jpeg_decompress_struct every frame, initialize it once and reuse it (call jpeg_abort_decompress between frames to reset). This cuts down on overhead.
• Pre-allocate buffers: If you know the maximum possible output size, allocate buffers once and reuse them instead of calling malloc/free per frame.
• Validate MJPEG frames: Real-time streams may send partial frames—check for the SOI (0xFFD8) and EOI (0xFFD9) markers before decoding to avoid garbage data.
• Switch to libjpeg-turbo: If you need faster decoding later, libjpeg-turbo is API-compatible with libjpeg and delivers much better performance for real-time use.

内容的提问来源于stack exchange，提问作者Gabe Rundlett