Swift中不使用内置摄像头，将IPCamer流接入AVCaptureSession

Hey there! I get what you're trying to do—you've got a working IP camera stream via VLC (outputting as UIImage or UIView) and want to plug that into an AVCaptureSession setup, probably to leverage its built-in features like video recording, real-time processing, or just using the familiar AVCaptureVideoPreviewLayer. Let's break down the practical approaches here, since there's no one-size-fits-all example out there.

Core Context First #

AVCaptureSession is designed to work with native capture devices (like your iPhone's cameras) via AVCaptureDeviceInput, but since you already have pre-processed frames from VLC, we don't need to hook into a physical device. Instead, we'll focus on injecting VLC's frame data into the AVFoundation pipeline, or adapting VLC's output to work with AVCaptureVideoPreviewLayer.

Approach 1: Custom AVCaptureInput (For Full Session Integration) #

If you need full integration with AVCaptureSession (e.g., using AVCaptureMovieFileOutput for recording, or AVCaptureVideoDataOutput for frame processing), you'll need to create a custom AVCaptureInput subclass to feed VLC's frames into the session. Here's how to pull this off:

1. Convert VLC's UIImage to CVPixelBuffer
   AVFoundation works with low-level pixel buffers, so first convert your UIImage frames to CVPixelBuffer (the format AVFoundation expects). Here's a quick Swift implementation:

   func imageToPixelBuffer(_ image: UIImage) -> CVPixelBuffer? {
       let pixelFormat = kCVPixelFormatType_32BGRA
       let attrs = [
           kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue,
           kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue
       ] as CFDictionary
       
       var pixelBuffer: CVPixelBuffer?
       let status = CVPixelBufferCreate(
           kCFAllocatorDefault,
           Int(image.size.width),
           Int(image.size.height),
           pixelFormat,
           attrs,
           &pixelBuffer
       )
       
       guard status == kCVReturnSuccess, let buffer = pixelBuffer else { return nil }
       
       CVPixelBufferLockBaseAddress(buffer, .readOnly)
       defer { CVPixelBufferUnlockBaseAddress(buffer, .readOnly) }
       
       guard let context = CGContext(
           data: CVPixelBufferGetBaseAddress(buffer),
           width: Int(image.size.width),
           height: Int(image.size.height),
           bitsPerComponent: 8,
           bytesPerRow: CVPixelBufferGetBytesPerRow(buffer),
           space: CGColorSpaceCreateDeviceRGB(),
           bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue
       ) else { return nil }
       
       context.draw(image.cgImage!, in: CGRect(origin: .zero, size: image.size))
       return buffer
   }
   

2. Wrap CVPixelBuffer into CMSampleBuffer
   Next, package the CVPixelBuffer into a CMSampleBuffer (the container AVCaptureSession uses for media samples). You'll need to set correct timing info (use VLC's frame timestamps if available, or generate your own):

   func pixelBufferToSampleBuffer(_ pixelBuffer: CVPixelBuffer) -> CMSampleBuffer? {
       var sampleBuffer: CMSampleBuffer?
       let formatDescription = CMVideoFormatDescriptionCreateForImageBuffer(nil, pixelBuffer)!
       let timingInfo = CMSampleTimingInfo(
           duration: CMTimeMake(value: 1, timescale: 30), // Adjust to match your stream's FPS
           presentationTimeStamp: CMClockGetTime(CMClockGetHostTimeClock()),
           decodeTimeStamp: CMTime.invalid
       )
       
       CMSampleBufferCreateForImageBuffer(
           allocator: kCFAllocatorDefault,
           imageBuffer: pixelBuffer,
           dataReady: true,
           makeDataReadyCallback: nil,
           refcon: nil,
           formatDescription: formatDescription,
           sampleTiming: &timingInfo,
           sampleBufferOut: &sampleBuffer
       )
       
       return sampleBuffer
   }
   

3. Implement Custom AVCaptureInput
   You'll need to create a subclass of AVCaptureInput that exposes an input port and handles pushing CMSampleBuffer instances into the session. This is the most complex part, as you have to conform to AVCaptureInput's lifecycle methods:

   class VLCStreamCaptureInput: AVCaptureInput {
       private let port: VLCStreamCapturePort
       
       override var ports: [AVCaptureInputPort] { [port] }
       
       init() {
           port = VLCStreamCapturePort()
           super.init()
       }
       
       required init?(coder: NSCoder) { fatalError("Not implemented") }
       
       override func startRunning() throws {
           // Start your VLC stream capture here
       }
       
       override func stopRunning() throws {
           // Stop your VLC stream capture here
       }
       
       // Call this from your VLC frame callback to push samples into the session
       func pushSampleBuffer(_ sampleBuffer: CMSampleBuffer) {
           port.dispatchSampleBuffer(sampleBuffer)
       }
   }
   
   class VLCStreamCapturePort: AVCaptureInputPort {
       override var mediaType: AVMediaType { .video }
       
       private let sampleQueue = DispatchQueue(label: "com.yourapp.vlc.samplequeue")
       
       func dispatchSampleBuffer(_ sampleBuffer: CMSampleBuffer) {
           sampleQueue.async {
               // Deliver the sample buffer to the session's internal queue
               self.session?.sessionQueue.async {
                   // Note: Apple doesn't officially document this direct delivery, so test thoroughly
                   // Alternatively, use AVCaptureVideoDataOutput's delegate to process samples directly
               }
           }
       }
   }
   

   Note: Directly pushing samples into AVCaptureSession isn't fully documented by Apple, so you might hit edge cases. For most use cases, Approach 2 is more reliable.

Approach 2: Skip AVCaptureSession Input, Use Preview Layer Directly #

If your main goal is to display the stream in an AVCaptureVideoPreviewLayer-like setup (or use AVFoundation's processing tools), you don't need to feed the stream into AVCaptureSession at all. Instead:

• Render VLC frames to a custom CALayer: Create a CALayer subclass that updates its contents with each UIImage from VLC. This mimics the behavior of AVCaptureVideoPreviewLayer without needing a session.
• Leverage AVFoundation tools separately: If you need to record the stream, use AVAssetWriter to write your CMSampleBuffer instances (from Approach 1) directly to a video file. For real-time processing, use Core Image to filter the UIImage frames before rendering.

Example of a simple frame-rendering layer:

class VLCStreamPreviewLayer: CALayer {
    private let queue = DispatchQueue(label: "com.yourapp.vlc.previewqueue")
    
    func updateFrame(_ image: UIImage) {
        queue.async {
            self.contents = image.cgImage
        }
    }
}

Then add this layer to your view's layer hierarchy, and call updateFrame from your VLC frame callback.

Final Recommendations #

• If you need full AVCaptureSession integration: Go with Approach 1, but be prepared to handle edge cases (like session lifecycle, timing sync). Test thoroughly on target devices.
• If you just need preview + basic processing/recording: Approach 2 is simpler and more reliable, since it avoids the complexity of custom AVCaptureInput implementations.

内容的提问来源于stack exchange，提问作者Brian Hamilton