直接调用CoreML出现图像格式不支持错误，求解决方案

使用基于YOLO且自带NMS输出的检测器，直接调用CoreML时触发错误：

Image is not expected type 32BGRA or 32ARGB, instead is Unsupported (875704422)

即使不对图像做裁剪操作，同样会触发该错误。但用Apple官方的Vision框架处理相同输入却没有问题。由于需要自定义特定区域裁剪，且CoreML代码逻辑更易理解，希望保留CoreML调用方式，求解决办法。

CoreML调用出错代码：

do {
    // 基于CoreMLHelpers封装的自定义工具类
    // 不裁剪直接传图也会报错
    let inSize = CVPixelBuffer.Size(width: 1280, height: 640)
    let crop = CVPixelBuffer.Rect(x: 0, y: 0, width: inSize.width, height: inSize.height)
    if var buffer = self.preparedBuffer, buffer.size == inSize {
        imageBuffer.crop(rect: crop, outPixelBuffer: &buffer)
    } else {
        self.preparedBuffer = imageBuffer.crop(rect: crop, outSize: inSize)
    }
    
    let out = try model.prediction(image: self.preparedBuffer!, iouThreshold: 0.4, confidenceThreshold: 0.25)
    //..
} catch {
    NSLog("Detector error: \(error)")
}

Vision框架无错误调用代码：

let imageRequestHandler = VNImageRequestHandler(cmSampleBuffer: buffer, orientation: vOrientation)

核心原因 #

Vision框架会自动处理像素格式转换，但直接调用CoreML时需要严格匹配模型要求的像素格式（32BGRA或32ARGB），自定义裁剪工具输出的CVPixelBuffer格式不满足要求。

具体修复步骤 #

• 强制转换像素格式
  在将CVPixelBuffer传入CoreML模型前，把它转换成32BGRA格式。示例代码如下：

  // 将buffer转换为32BGRA格式的工具方法
  func convertTo32BGRA(from buffer: CVPixelBuffer) -> CVPixelBuffer? {
      let attributes: [CFString: Any] = [
          kCVPixelBufferCGImageCompatibilityKey: true,
          kCVPixelBufferCGBitmapContextCompatibilityKey: true,
          kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA
      ]
      
      var convertedBuffer: CVPixelBuffer?
      let status = CVPixelBufferCreate(
          nil,
          CVPixelBufferGetWidth(buffer),
          CVPixelBufferGetHeight(buffer),
          kCVPixelFormatType_32BGRA,
          attributes as CFDictionary,
          &convertedBuffer
      )
      
      guard status == kCVReturnSuccess, let destBuffer = convertedBuffer else {
          return nil
      }
      
      CVPixelBufferLockBaseAddress(buffer, .readOnly)
      CVPixelBufferLockBaseAddress(destBuffer, .readWrite)
      
      defer {
          CVPixelBufferUnlockBaseAddress(buffer, .readOnly)
          CVPixelBufferUnlockBaseAddress(destBuffer, .readWrite)
      }
      
      let sourceData = CVPixelBufferGetBaseAddress(buffer)
      let destData = CVPixelBufferGetBaseAddress(destBuffer)
      let bytesPerRow = CVPixelBufferGetBytesPerRow(buffer)
      let height = CVPixelBufferGetHeight(buffer)
      
      memcpy(destData, sourceData, bytesPerRow * height)
      return destBuffer
  }
  

  在调用model.prediction前加入转换逻辑：

  guard let convertedBuffer = convertTo32BGRA(from: self.preparedBuffer!) else {
      // 处理转换失败情况
      return
  }
  let out = try model.prediction(image: convertedBuffer, iouThreshold: 0.4, confidenceThreshold: 0.25)
  

• 修改裁剪工具的输出格式
  调整自定义crop方法的逻辑，让它直接输出32BGRA格式的CVPixelBuffer，避免额外的格式转换步骤。比如在创建裁剪后的buffer时，指定像素格式为kCVPixelFormatType_32BGRA。

• 验证模型输入要求
  用Xcode的模型检查器打开CoreML模型，确认输入图像的像素格式要求是否为32BGRA或32ARGB，避免格式匹配偏差。

内容的提问来源于stack exchange，提问作者Gralex