AVCaptureVideoOutput结合MultipeerConnectivity流摄像头预览问题排查与优化

云上老周

2026-8-2

问题

正在开发一款摄像头遥控应用，iPhone作为摄像头端、iPad作为遥控端。通过AVCaptureVideoOutput获取iPhone摄像头预览画面，借助MultipeerConnectivity框架的OutputStream传输流数据，iPad接收后设置UIView.layer.contents显示画面。

当前实现代码如下：

iPhone（摄像头端）`AVCaptureVideoDataOutputSampleBufferDelegate`实现

func captureOutput(_ output: AVCaptureOutput,
                   didOutput sampleBuffer: CMSampleBuffer,
                   from connection: AVCaptureConnection) {
    DispatchQueue.global(qos: .utility).async { [unowned self] in
        let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
        
        if let imageBuffer {
            CVPixelBufferLockBaseAddress(imageBuffer, [])
            let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
            let bytesPerRow: size_t? = CVPixelBufferGetBytesPerRow(imageBuffer)
            let width: size_t? = CVPixelBufferGetWidth(imageBuffer)
            let height: size_t? = CVPixelBufferGetHeight(imageBuffer)
            
            let colorSpace = CGColorSpaceCreateDeviceRGB()
            let newContext = CGContext(data: baseAddress,
                                       width: width ?? 0,
                                       height: height ?? 0,
                                       bitsPerComponent: 8,
                                       bytesPerRow: bytesPerRow ?? 0,
                                       space: colorSpace,
                                       bitmapInfo: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue)
            
            if let newImage = newContext?.makeImage() {
                let image = UIImage(cgImage: newImage,
                                    scale: 0.2,
                                    orientation: .up)
                
                CVPixelBufferUnlockBaseAddress(imageBuffer, [])
                
                if let data = image.jpegData(compressionQuality: 0.2) {
                    let bytesWritten = data.withUnsafeBytes({
                        viewFinderStream?
                            .write($0.bindMemory(to: UInt8.self).baseAddress!, maxLength: data.count)
                    })
                }
            }
        }
    }
}

iPad（遥控端）`StreamDelegate`实现

func stream(_ aStream: Stream, handle eventCode: Stream.Event) {
    let inputStream = aStream as! InputStream

    switch eventCode {
    case .hasBytesAvailable:
        DispatchQueue.global(qos: .userInteractive).async { [unowned self] in
            var buffer = [UInt8](repeating: 0, count: 1024)
            let numberBytes = inputStream.read(&buffer, maxLength: 1024)
            let data = Data(referencing: NSData(bytes: &buffer, length: numberBytes))
            if let imageData = UIImage(data: data) {
                DispatchQueue.main.async {
                    previewCameraView.layer.contents = imageData.cgImage
                }
            }
        }
    case .hasSpaceAvailable:
        break
    default:
        break
    }
}

遇到的问题：

iPad仅显示极小部分画面（仅视图左上角少量像素，其余灰色），控制台出现警告：

2023-02-02 20:24:44.487399+0700 MultipeerVideo-Assignment[31170:1065836] Warning! [0x15c023800] Decoding incomplete with error code -1. This is expected if the image has not been fully downloaded.

iPhone CPU占用率接近100%

解决方案

一、画面显示不全的原因及修复

核心问题

发送端直接将整张图片的JPEG数据写入输出流，但接收端每次仅读取1024字节就尝试解码。一张压缩后的JPEG图片远大于1024字节，导致每次读取的都是不完整的JPEG数据，无法解码出完整图片，进而出现仅显示部分像素的情况。

修复步骤

发送端：添加数据长度标识
在发送图片数据前，先将数据长度转换为固定4字节的UInt32（大端字节序）写入流，让接收端先知晓完整图片的字节数：

if let data = image.jpegData(compressionQuality: 0.2) {
    // 先写入数据长度
    let length = UInt32(data.count).bigEndian
    let lengthData = Data(bytes: &length, count: MemoryLayout<UInt32>.size)
    _ = lengthData.withUnsafeBytes {
        viewFinderStream?.write($0.bindMemory(to: UInt8.self).baseAddress!, maxLength: lengthData.count)
    }
    // 再写入图片数据
    let bytesWritten = data.withUnsafeBytes({
        viewFinderStream?
            .write($0.bindMemory(to: UInt8.self).baseAddress!, maxLength: data.count)
    })
}

接收端：缓存数据直到完整接收
维护一个缓冲区，先读取长度标识，再根据长度接收足够字节后再解码：

// 类级变量存储缓冲区和待接收长度
private var receiveBuffer = Data()
private var expectedLength: UInt32?

func stream(_ aStream: Stream, handle eventCode: Stream.Event) {
    let inputStream = aStream as! InputStream

    switch eventCode {
    case .hasBytesAvailable:
        DispatchQueue.global(qos: .userInteractive).async { [unowned self] in
            var buffer = [UInt8](repeating: 0, count: 1024)
            while inputStream.hasBytesAvailable {
                let bytesRead = inputStream.read(&buffer, maxLength: buffer.count)
                guard bytesRead > 0 else { break }
                receiveBuffer.append(buffer.prefix(bytesRead))
                
                // 解析长度标识
                if expectedLength == nil {
                    if receiveBuffer.count >= MemoryLayout<UInt32>.size {
                        let lengthData = receiveBuffer.prefix(MemoryLayout<UInt32>.size)
                        expectedLength = lengthData.withUnsafeBytes {
                            $0.load(as: UInt32.self).bigEndian
                        }
                        receiveBuffer.removeFirst(MemoryLayout<UInt32>.size)
                    }
                }
                
                // 检查是否接收完整数据
                if let expectedLength = expectedLength, receiveBuffer.count >= Int(expectedLength) {
                    let imageData = receiveBuffer.prefix(Int(expectedLength))
                    if let image = UIImage(data: imageData) {
                        DispatchQueue.main.async {
                            self.previewCameraView.layer.contents = image.cgImage
                        }
                    }
                    // 重置缓冲区和长度标识
                    receiveBuffer.removeFirst(Int(expectedLength))
                    self.expectedLength = nil
                }
            }
        }
    case .hasSpaceAvailable:
        break
    default:
        break
    }
}

二、降低iPhone CPU占用率的优化方法

1. 跳过不必要的像素格式转换

直接对CVPixelBuffer进行JPEG编码，避免转换为CGImage和UIImage的开销：

if let imageBuffer {
    CVPixelBufferLockBaseAddress(imageBuffer, [])
    // 直接编码CVPixelBuffer为JPEG
    let ciImage = CIImage(cvPixelBuffer: imageBuffer)
    let context = CIContext(options: [.useSoftwareRenderer: false]) // 启用硬件加速
    if let jpegData = context.jpegRepresentation(of: ciImage, colorSpace: CGColorSpaceCreateDeviceRGB(), options: [.quality: 0.2]) {
        // 写入长度和数据（同之前的发送逻辑）
        let length = UInt32(jpegData.count).bigEndian
        let lengthData = Data(bytes: &length, count: MemoryLayout<UInt32>.size)
        _ = lengthData.withUnsafeBytes {
            viewFinderStream?.write($0.bindMemory(to: UInt8.self).baseAddress!, maxLength: lengthData.count)
        }
        _ = jpegData.withUnsafeBytes {
            viewFinderStream?.write($0.bindMemory(to: UInt8.self).baseAddress!, maxLength: jpegData.count)
        }
    }
    CVPixelBufferUnlockBaseAddress(imageBuffer, [])
}

2. 降低摄像头采集分辨率

在配置AVCaptureSession时选择更低的预设分辨率，减少数据处理量：

let captureSession = AVCaptureSession()
captureSession.sessionPreset = .vga640x480 // 或更低的.qvga320x240

3. 限制采集帧率

通过AVCaptureVideoDataOutput限制帧率，减少每秒处理的画面数量：

let videoOutput = AVCaptureVideoDataOutput()
videoOutput.minFrameDuration = CMTimeMake(value: 1, timescale: 15) // 限制为15帧/秒

4. 确保后台线程处理所有耗时操作

保持编码、流写入等操作在后台队列执行，避免阻塞主线程，同时避免不必要的线程切换。

内容的提问来源于stack exchange，提问作者cleanrun

AVCaptureVideoOutput结合MultipeerConnectivity流摄像头预览问题排查与优化

iPhone（摄像头端）AVCaptureVideoDataOutputSampleBufferDelegate实现 #

iPad（遥控端）StreamDelegate实现 #

一、画面显示不全的原因及修复 #

核心问题 #

修复步骤 #

二、降低iPhone CPU占用率的优化方法 #

1. 跳过不必要的像素格式转换 #

2. 降低摄像头采集分辨率 #

3. 限制采集帧率 #

4. 确保后台线程处理所有耗时操作 #