如何为YCbCr像素格式创建vImage_CGImageFormat？技术咨询

Great question—building a high-performance pixel format converter with vImage is a smart choice for real-time AVCapture video processing, since vImage is optimized for Apple’s hardware. Let’s walk through exactly how to set up the vImage_CGImageFormat structs for those YCbCr formats you need, plus tie it all together with a practical conversion workflow.

The key difference between BGRA and YCbCr formats lies in how color data is stored (planar vs bi-planar) and the color space/range. Here’s how to define each required format:

Bi-Planar YCbCr (Video Range & Full Range) #

These formats store luminance (Y) in one full-resolution plane, and chrominance (Cb/Cr) in a single half-resolution plane (interleaved CbCr pixels). The only difference between Video Range and Full Range is the data value range:

• Video Range: Y uses 16-235, CbCr uses 16-240 (broadcast standard)
• Full Range: All components use 0-255 (full pixel depth)

Video Range (kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) #

vImage_CGImageFormat CreateBiPlanarVideoRangeFormat() {
    CGColorSpaceRef ycbcrColorSpace = CGColorSpaceCreateITUR_709(); // Standard video color space
    vImage_CGImageFormat format = {
        .bitsPerComponent = 8,          // Each color component is 8 bits
        .bitsPerPixel = 16,             // Average 8 bits Y + 4 bits Cb + 4 bits Cr per pixel
        .colorSpace = ycbcrColorSpace,
        .bitmapInfo = kCGImageAlphaNone | kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
        .version = 0,
        .decode = NULL,
        .renderingIntent = kCGRenderingIntentDefault
    };
    return format;
}

Full Range (kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) #

This is nearly identical—just swap the pixel format in bitmapInfo:

vImage_CGImageFormat CreateBiPlanarFullRangeFormat() {
    CGColorSpaceRef ycbcrColorSpace = CGColorSpaceCreateITUR_709();
    vImage_CGImageFormat format = {
        .bitsPerComponent = 8,
        .bitsPerPixel = 16,
        .colorSpace = ycbcrColorSpace,
        .bitmapInfo = kCGImageAlphaNone | kCVPixelFormatType_420YpCbCr8BiPlanarFullRange,
        .version = 0,
        .decode = NULL,
        .renderingIntent = kCGRenderingIntentDefault
    };
    return format;
}

Planar YCbCr (kCVPixelFormatType_420YpCbCr8Planar) #

This format uses three separate planes: full-resolution Y, half-resolution Cb, and half-resolution Cr. Here’s the format definition:

vImage_CGImageFormat CreatePlanar420Format() {
    CGColorSpaceRef ycbcrColorSpace = CGColorSpaceCreateITUR_709();
    vImage_CGImageFormat format = {
        .bitsPerComponent = 8,          // 8 bits per component
        .bitsPerPixel = 24,             // 8+8+8 bits per logical pixel (with subsampling)
        .colorSpace = ycbcrColorSpace,
        .bitmapInfo = kCGImageAlphaNone | kCVPixelFormatType_420YpCbCr8Planar,
        .version = 0,
        .decode = NULL,
        .renderingIntent = kCGRenderingIntentDefault
    };
    return format;
}

Here’s how to tie these formats into a working converter using vImageConvert_AnyToAny, with a focus on performance:

- (BOOL)convertCapturePixelBuffer:(CVPixelBufferRef)sourceBuffer 
                        toBGRAData:(void *)destinationData 
                            width:(size_t)width 
                           height:(size_t)height 
                      sourceFormatType:(OSType)sourceFormatType {
    
    // Create source format based on input type
    vImage_CGImageFormat sourceFormat;
    if (sourceFormatType == kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) {
        sourceFormat = CreateBiPlanarVideoRangeFormat();
    } else if (sourceFormatType == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
        sourceFormat = CreateBiPlanarFullRangeFormat();
    } else if (sourceFormatType == kCVPixelFormatType_420YpCbCr8Planar) {
        sourceFormat = CreatePlanar420Format();
    } else {
        NSLog(@"Unsupported source format");
        return NO;
    }

    // Create BGRA destination format (your existing implementation)
    CGColorSpaceRef bgraColorSpace = CGColorSpaceCreateDeviceRGB();
    vImage_CGImageFormat destFormat = {
        .bitsPerComponent = 8,
        .bitsPerPixel = 32,
        .colorSpace = bgraColorSpace,
        .bitmapInfo = kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little,
        .version = 0,
        .decode = NULL,
        .renderingIntent = kCGRenderingIntentDefault
    };

    // Initialize source vImage buffer from CVPixelBuffer
    vImage_Buffer vSourceBuffer;
    vImage_Error error = vImageBuffer_InitWithCVPixelBuffer(&vSourceBuffer, &sourceFormat, sourceBuffer, kvImageNoFlags);
    if (error != kvImageNoError) {
        NSLog(@"Failed to init source buffer: %ld", error);
        goto cleanup;
    }

    // Initialize destination vImage buffer
    vImage_Buffer vDestBuffer = {
        .data = destinationData,
        .width = width,
        .height = height,
        .rowBytes = width * 4 // 4 bytes per BGRA pixel
    };

    // Reuse this converter across frames for massive performance gains!
    static vImageConverterRef converter = NULL;
    if (!converter) {
        error = vImageConverter_CreateWithCGImageFormat(&sourceFormat, &destFormat, NULL, kvImageNoFlags, &error);
        if (error != kvImageNoError) {
            NSLog(@"Failed to create converter: %ld", error);
            goto cleanup;
        }
    }

    // Perform conversion
    error = vImageConvert_AnyToAny(converter, &vSourceBuffer, &vDestBuffer, NULL, kvImageNoFlags);

cleanup:
    // Release Core Foundation objects (ARC doesn't manage these)
    CGColorSpaceRelease(sourceFormat.colorSpace);
    CGColorSpaceRelease(destFormat.colorSpace);
    return error == kvImageNoError;
}

• Reuse the vImageConverter: Creating a converter is expensive—initialize it once when your library starts (or when the source/dest format changes) and reuse it for every frame.
• Memory Alignment: Ensure your destination buffer is 16-byte aligned (use posix_memalign instead of malloc) to leverage Accelerate’s optimized vector operations.
• Cleanup: Always release CGColorSpaceRef objects, as they’re not managed by ARC.
• Debugging: Add the kvImagePrintDiagnosticsToConsole flag when creating the converter to get detailed format compatibility logs if you hit issues.

内容的提问来源于stack exchange，提问作者Alexander Ushakov