iOS端从240fps FHD视频提取FHD帧失败求助

我正在开发一款iPhone视频分析应用，目标视频是用iPhone后置摄像头以240fps帧率拍摄的FHD（1920×1080）视频。但提取视频帧时，得到的始终是HD（1280×720）分辨率的图像，而我需要FHD分辨率的帧。

我尝试了两种方法均失败：

方法1：Swift原生AVAssetReader代码 #

func extract_frames(start_frame_num: Int, number_of_frames: Int) {
        guard let videoURL = selectedVideoURL else {
            print("No video URL found")
            return
        }
        

        guard let assetReader = try? AVAssetReader(asset: asset) else {
            print("Could not create AVAssetReader")
            return
        }

        let videoTrack = asset.tracks(withMediaType: .video).first!
        let readerOutputSettings: [String: Any] = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32ARGB)]
        
        let assetReaderOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: readerOutputSettings)

        
        if assetReader.canAdd(assetReaderOutput) {
            assetReader.add(assetReaderOutput)
        } else {
            print("Couldn't add reader output")
            return
        }

        let frameDuration = CMTimeMake(value: 1, timescale: Int32(videoTrack.nominalFrameRate))
        let startFrameTime = CMTimeMultiply(frameDuration, multiplier: Int32(start_frame_num))
        
        // Set the time range before starting to read
        assetReader.timeRange = CMTimeRangeMake(start: startFrameTime, duration: asset.duration)

        assetReader.startReading()

        var currentFrame = start_frame_num

        frames.removeAll()
        print("existing frames are all removed.")
        // Extract frames
        while currentFrame < (start_frame_num + number_of_frames) {
            guard let sampleBuffer = assetReaderOutput.copyNextSampleBuffer() else {
                break
            }
            
            if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
                // Convert to UIImage
                let ciImage = CIImage(cvImageBuffer: imageBuffer)
                let context = CIContext(options: nil)
                if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
                    let uiImage = UIImage(cgImage: cgImage)
                    frames.append(uiImage)
                    print("Extracted frame \(currentFrame)")
                }
            }

            currentFrame += 1
        }

        assetReader.cancelReading()
    }

方法2：OpenCV Swift代码（Mac成功，iOS失败） #

func extractFrames(startFrameNum: Int = 0, videoNum: Int = 1) {
        guard let videoURL = selectedVideoURL else {
            print("No video URL found")
            return
        }

        let videoCapture = VideoCapture(filename: videoURL.path)

        if !videoCapture.isOpened() {
            print("Error: Could not open video.")
            return
        }
        
        print("videoCapture width : \(videoCapture.get(propId: VideoCaptureProperties.CAP_PROP_FRAME_WIDTH.rawValue))")
        print("videoCapture height: \(videoCapture.get(propId: VideoCaptureProperties.CAP_PROP_FRAME_HEIGHT.rawValue))")

        var frameCount = 0
        let frameToStop = startFrameNum + videoNum

        while videoCapture.grab() {
            if frameCount >= startFrameNum && frameCount < frameToStop {
                let mat = Mat()
                if videoCapture.retrieve(image: mat, flag: 0) {
                    let uiImage = mat.toUIImage()
                    self.frames.append(uiImage)
                    print("Frame at position \(frameCount) was saved.")
                }
            }
            frameCount += 1
        }
        print("\(videoURL.path) extraction completed. \(videoNum) frames extracted.\(frameCount) frames existed in this video. ")
    }

已知Frame Grabber这款iPhone应用能成功提取该视频的FHD帧，求解决方法。

针对AVAssetReader的修复 #

iPhone拍摄的240fps视频通常包含多条不同分辨率的轨道（用于预览和原始录制），你当前的代码直接取第一条视频轨道，且未指定输出分辨率约束，导致AVAssetReader默认返回低分辨率帧。

修改步骤： #

1. 筛选最高分辨率的视频轨道：

let videoTracks = asset.tracks(withMediaType: .video)
guard let videoTrack = videoTracks.max(by: {
    $0.naturalSize.width * $0.naturalSize.height < $1.naturalSize.width * $1.naturalSize.height
}) else {
    print("No video track found")
    return
}

2. 添加分辨率约束到输出设置：
   根据视频轨道的原始尺寸（考虑旋转情况），强制指定输出宽高：

let originalSize = videoTrack.naturalSize
let orientation = videoTrack.preferredTransform
// 判断视频是否旋转（90/270度）
let isRotated = (orientation.a == 0 && orientation.d == 0) || (orientation.b == 0 && orientation.c == 0)
let outputWidth = isRotated ? originalSize.height : originalSize.width
let outputHeight = isRotated ? originalSize.width : originalSize.height

let readerOutputSettings: [String: Any] = [
    kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32ARGB),
    kCVPixelBufferWidthKey as String: outputWidth,
    kCVPixelBufferHeightKey as String: outputHeight,
    AVVideoCodecKey as String: AVVideoCodecTypeH264.rawValue // 若视频是HEVC编码，改为AVVideoCodecTypeHEVC.rawValue
]

针对OpenCV iOS端的修复 #

iOS上的OpenCV默认可能优先选择低分辨率流，且硬件加速可能导致分辨率降级，需强制指定读取原始分辨率：

let videoCapture = VideoCapture(filename: videoURL.path)

if !videoCapture.isOpened() {
    print("Error: Could not open video.")
    return
}

// 强制设置为FHD分辨率
videoCapture.set(propId: VideoCaptureProperties.CAP_PROP_FRAME_WIDTH.rawValue, value: 1920)
videoCapture.set(propId: VideoCaptureProperties.CAP_PROP_FRAME_HEIGHT.rawValue, value: 1080)
// 禁用硬件加速（部分设备需此设置）
videoCapture.set(propId: VideoCaptureProperties.CAP_PROP_HW_ACCELERATION.rawValue, value: VideoCaptureHwAcceleration.CAP_HW_ACCELERATION_NONE.rawValue)

同时确保传入的videoURL.path是沙箱内的真实文件路径，而非ph://类型的asset URL。

额外验证 #

提取帧后可打印尺寸确认：

print("Extracted frame size: \(uiImage.size.width) × \(uiImage.size.height)")

内容的提问来源于stack exchange，提问作者Jeehoon Park