BGRA转YUV(420V)异常：特定视频转换后像素错乱问题排查

问题描述 #

自研BGRA格式CVPixelBuffer转YUV(420V)格式的函数，处理手机相册中大部分视频时运行正常，但处理某一特定视频时，转换后的像素出现错乱。

视频媒体信息 #

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L3.1
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Format settings, GOP                     : M=1, N=15
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 6 s 623 ms
Source duration                          : 6 s 997 ms
Bit rate                                 : 4 662 kb/s
Width                                    : 884 pixels
Clean aperture width                     : 884 pixels
Height                                   : 492 pixels
Clean aperture height                    : 492 pixels
Display aspect ratio                     : 16:9
Original display aspect ratio            : 16:9
Frame rate mode                          : Variable
Frame rate                               : 57.742 FPS
Minimum frame rate                       : 20.000 FPS
Maximum frame rate                       : 100.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.186
Stream size                              : 3.67 MiB (94%)
Source stream size                       : 3.79 MiB (97%)
Title                                    : Core Media Video
Encoded date                             : UTC 2021-10-29 09:54:03
Tagged date                              : UTC 2021-10-29 09:54:03
Color range                              : Limited
Color primaries                          : Display P3
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

转换函数代码 #

CFDictionaryRef CreateCFDictionary(CFTypeRef* keys, CFTypeRef* values, size_t size) {
      return CFDictionaryCreate(kCFAllocatorDefault,
                                keys,
                                values,
                                size,
                                &kCFTypeDictionaryKeyCallBacks,
                                &kCFTypeDictionaryValueCallBacks);
    }

static void bt709_rgb2yuv8bit_TV(uint8_t R, uint8_t G, uint8_t B, uint8_t &Y, uint8_t &U, uint8_t &V)
    {
        Y =  0.183 * R + 0.614 * G + 0.062 * B + 16;
        U = -0.101 * R - 0.339 * G + 0.439 * B + 128;
        V =  0.439 * R - 0.399 * G - 0.040 * B + 128;
    }

CVPixelBufferRef RGB2YCbCr8Bit(CVPixelBufferRef pixelBuffer)
    {
        CVPixelBufferLockBaseAddress(pixelBuffer, 0);
        uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(pixelBuffer);
        int w = (int) CVPixelBufferGetWidth(pixelBuffer);
        int h = (int) CVPixelBufferGetHeight(pixelBuffer);
//        int stride = (int) CVPixelBufferGetBytesPerRow(pixelBuffer) / 4;

        OSType pixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange;

        CVPixelBufferRef pixelBufferCopy = NULL;
        const size_t attributes_size = 1;
        CFTypeRef keys[attributes_size] = {
            kCVPixelBufferIOSurfacePropertiesKey,
        };
        CFDictionaryRef io_surface_value = CreateCFDictionary(nullptr, nullptr, 0);
        CFTypeRef values[attributes_size] = {io_surface_value};
        
        CFDictionaryRef attributes = CreateCFDictionary(keys, values, attributes_size);
        CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault,
                                              w,
                                              h,
                                              pixelFormat,
                                              attributes,
                                              &pixelBufferCopy);
        if (status != kCVReturnSuccess) {
            std::cout << "YUVBufferCopyWithPixelBuffer :: failed" << std::endl;
            return nullptr;
        }
        if (attributes) {
            CFRelease(attributes);
            attributes = nullptr;
        }
        
        CVPixelBufferLockBaseAddress(pixelBufferCopy, 0);

        size_t y_stride = CVPixelBufferGetBytesPerRowOfPlane(pixelBufferCopy, 0);
        size_t uv_stride  = CVPixelBufferGetBytesPerRowOfPlane(pixelBufferCopy, 1);
        
        int plane_h1 = (int) CVPixelBufferGetHeightOfPlane(pixelBufferCopy, 0);
        int plane_h2 = (int) CVPixelBufferGetHeightOfPlane(pixelBufferCopy, 1);
        
        uint8_t *y = (uint8_t *) CVPixelBufferGetBaseAddressOfPlane(pixelBufferCopy, 0);
        memset(y, 0x80, plane_h1 * y_stride);
        
        uint8_t *uv = (uint8_t *) CVPixelBufferGetBaseAddressOfPlane(pixelBufferCopy, 1);
        memset(uv, 0x80, plane_h2 * uv_stride);
        
        int y_bufferSize = w * h;
        int uv_bufferSize = w * h / 4;
        uint8_t *y_planeData = (uint8_t *) malloc(y_bufferSize * sizeof(uint8_t));
        uint8_t *u_planeData = (uint8_t *) malloc(uv_bufferSize * sizeof(uint8_t));
        uint8_t *v_planeData = (uint8_t *) malloc(uv_bufferSize * sizeof(uint8_t));
            
        int u_offset = 0;
        int v_offset = 0;
        uint8_t R, G, B;
        uint8_t Y, U, V;
        
        for (int i = 0; i < h; i ++) {
            for (int j = 0; j < w; j ++) {
                int offset = i * w + j;
                B = baseAddress[offset * 4];
                G = baseAddress[offset * 4 + 1];
                R = baseAddress[offset * 4 + 2];
                bt709_rgb2yuv8bit_TV(R, G, B, Y, U, V);
                y_planeData[offset] = Y;
                //隔行扫描 偶数行的偶数列取U 奇数行的偶数列取V
                if (j % 2 == 0) {
                    (i % 2 == 0) ? u_planeData[u_offset++] = U : v_planeData[v_offset++] = V;
                }
            }
        }
        
        for (int i = 0; i < plane_h1; i ++) {
            memcpy(y + i * y_stride, y_planeData + i * w, w);
            if (i < plane_h2) {
                for (int j = 0 ; j < w ; j+=2) {
                    //NV12 和 NV21 格式都属于 YUV420SP 类型。它也是先存储了 Y 分量，但接下来并不是再存储所有的 U 或者 V 分量，而是把 UV 分量交替连续存储。
                    //NV12 是 IOS 中有的模式，它的存储顺序是先存 Y 分量，再 UV 进行交替存储。
                    memcpy(uv + i * y_stride + j, u_planeData + i * w/2 + j/2, 1);
                    memcpy(uv + i * y_stride + j + 1, v_planeData + i * w/2 + j/2, 1);
                }
            }
        }
        free(y_planeData);
        free(u_planeData);
        free(v_planeData);
        
        CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
        CVPixelBufferUnlockBaseAddress(pixelBufferCopy, 0);
        return pixelBufferCopy;
    }

截图说明 #

• 正常BGRA格式pixelBuffer截图
• 转换后错乱YUV格式pixelBuffer截图

问题排查与修复方案 #

1. 忽略CVPixelBuffer行间距（stride）导致RGB数据读取错误 #

代码中注释了stride计算，直接用i*w +j推导像素偏移，默认BGRA行字节数为w*4，但实际CVPixelBuffer会做内存对齐（比如宽度884，884*4=3536，可能对齐到4096），导致读取的B/G/R数据偏移错误。

修复：

// 取消注释并正确计算stride
int stride = (int)CVPixelBufferGetBytesPerRow(pixelBuffer);
// 读取像素时使用stride
B = baseAddress[i * stride + j * 4];
G = baseAddress[i * stride + j * 4 + 1];
R = baseAddress[i * stride + j * 4 + 2];

2. UV平面写入错误使用Y平面的stride #

代码中写入UV平面时用了uv + i*y_stride +j，但UV平面有独立的行间距uv_stride，错误偏移会导致UV数据乱序。

修复：

// 替换y_stride为uv_stride
memcpy(uv + i * uv_stride + j, u_planeData + i * w/2 + j/2, 1);
memcpy(uv + i * uv_stride + j + 1, v_planeData + i * w/2 + j/2, 1);

3. UV分量采样逻辑错误 #

NV12格式的UV平面是每行交替存储U/V，对应Y平面中2x2像素块的U/V值。原代码将偶数行的U和奇数行的V分开存储，导致UV分量与Y像素的对应关系错位。

修复：
调整采样逻辑，每个2x2的Y像素块对应一组U/V：

// 重置采样逻辑
int uv_idx = 0;
for (int i = 0; i < h; i += 2) {
    for (int j = 0; j < w; j += 2) {
        // 取2x2块左上角像素的U/V（或对4个像素取平均提升精度）
        int offset = i * stride + j * 4;
        B = baseAddress[offset];
        G = baseAddress[offset + 1];
        R = baseAddress[offset + 2];
        bt709_rgb2yuv8bit_TV(R, G, B, Y, U, V);
        u_planeData[uv_idx] = U;
        v_planeData[uv_idx] = V;
        uv_idx++;
    }
}

4. 浮点计算无范围限制导致溢出 #

bt709_rgb2yuv8bit_TV函数直接将浮点结果赋值给uint8_t，未限制范围，计算结果超出0-255时会出现溢出错误。

修复：
添加范围限制：

static void bt709_rgb2yuv8bit_TV(uint8_t R, uint8_t G, uint8_t B, uint8_t &Y, uint8_t &U, uint8_t &V)
{
    float y_val = 0.183 * R + 0.614 * G + 0.062 * B + 16;
    float u_val = -0.101 * R - 0.339 * G + 0.439 * B + 128;
    float v_val = 0.439 * R - 0.399 * G - 0.040 * B + 128;
    
    // 限制结果在0-255范围内
    Y = static_cast<uint8_t>(std::max(0.0f, std::min(255.0f, y_val)));
    U = static_cast<uint8_t>(std::max(0.0f, std::min(255.0f, u_val)));
    V = static_cast<uint8_t>(std::max(0.0f, std::min(255.0f, v_val)));
}

内容的提问来源于stack exchange，提问作者blackox626