AVAssetWriter首帧失真：CVImageBuffer→CIImage→MLTexture流程排查

核心排查方向 #

• MLTexture属性校验
  首帧处理时，MLTexture的宽高、像素格式可能与原始CVImageBuffer不匹配。打印参数对比：

  print("Original buffer format: \(CVPixelBufferGetPixelFormatType(originalBuffer))")
  print("MLTexture format: \(mlTexture.pixelFormat), size: \(mlTexture.width)x\(mlTexture.height)")
  

  若首帧MLTexture属性异常，说明纹理创建时参数传递有误。

• Metal渲染同步问题
  Metal命令队列默认异步执行，首帧可能在CIImage渲染到MLTexture的命令未完成时，就开始转换为CVPixelBuffer，导致读取到未完全渲染的纹理数据。

• CVPixelBuffer参数匹配问题
  从MLTexture转CVPixelBuffer时，首帧的像素格式、颜色空间或字节对齐设置可能与AVAssetWriterInput的输出配置不一致。对比两种流程下首帧CVPixelBuffer的属性：

  // 检查像素格式
  let bufferFormat = CVPixelBufferGetPixelFormatType(pixelBuffer)
  // 检查颜色空间
  guard let colorSpace = CVPixelBufferGetColorSpace(pixelBuffer) else { return }
  print("Buffer color space: \(colorSpace)")
  

• Metal资源初始化时机
  若MTLDevice、MTLCommandQueue等资源在首帧处理时才初始化，会存在资源加载延迟，导致首帧纹理创建不完整。

针对性解决方法 #

1. 提前初始化Metal资源
   在视频编码启动前完成Metal设备和命令队列的初始化，避免首帧时的初始化开销：

   let device = MTLCreateSystemDefaultDevice()!
   let commandQueue = device.makeCommandQueue()!
   let ciContext = CIContext(mtlDevice: device)
   // 全局持有这些资源，不要在每帧处理时重复创建
   

2. 首帧强制同步Metal渲染
   首帧处理时，等待Metal命令缓冲区执行完成后再转换CVPixelBuffer：

   func processFirstFrame(ciImage: CIImage, mlTexture: MLTexture) -> CVPixelBuffer? {
       let commandBuffer = commandQueue.makeCommandBuffer()!
       ciContext.render(ciImage, to: mlTexture, commandBuffer: commandBuffer)
       commandBuffer.commit()
       commandBuffer.waitUntilCompleted() // 首帧添加同步等待，后续帧可移除提升性能
       
       // 后续MLTexture转CVPixelBuffer的逻辑
       return pixelBuffer
   }
   

3. 严格匹配CVPixelBuffer创建参数
   创建CVPixelBuffer时，明确指定与原始帧、AVAssetWriter完全一致的参数：

   let originalWidth = CVPixelBufferGetWidth(originalBuffer)
   let originalHeight = CVPixelBufferGetHeight(originalBuffer)
   let originalFormat = CVPixelBufferGetPixelFormatType(originalBuffer)
   
   let attributes: [String: Any] = [
       kCVPixelBufferPixelFormatTypeKey as String: originalFormat,
       kCVPixelBufferWidthKey as String: originalWidth,
       kCVPixelBufferHeightKey as String: originalHeight,
       kCVPixelBufferCGImageCompatibilityKey as String: true,
       kCVPixelBufferCGBitmapContextCompatibilityKey as String: true,
       kCVPixelBufferColorSpaceKey as String: CGColorSpaceCreateDeviceRGB()
   ]
   
   var pixelBuffer: CVPixelBuffer?
   let status = CVPixelBufferCreate(kCFAllocatorDefault, originalWidth, originalHeight, originalFormat, attributes as CFDictionary, &pixelBuffer)
   guard status == kCVReturnSuccess, let buffer = pixelBuffer else { return nil }
   
   // 将MLTexture的数据复制到buffer中
   

4. 单独验证首帧CIImage渲染结果
   跳过MLTexture步骤，直接将首帧的CIImage渲染为CVPixelBuffer，确认滤镜（文字叠加）的输出是否正常。若CIImage本身渲染异常，需排查滤镜逻辑的首帧初始化问题（比如文字布局未正确计算）。

内容的提问来源于stack exchange，提问作者user924