画中画（Picture-in-Picture, PiP）是一种多窗口模式，允许用户在屏幕一角的小窗口中观看视频，同时在主屏幕上继续与其他应用或内容进行交互。本文详细介绍如何基于 AVSampleBufferDisplayLayer 实现画中画功能，使画中画窗口和主播放窗口共享 TTVideoEngine 回调的 pixelBuffer 进行渲染，实现无缝切换体验。

前提条件

• 系统版本：iOS 15 及以上版本

实现步骤

步骤 1：系统设置及项目配置

1. 在 iOS 设备上，导航至设置 > 通用 > 画中画页面，确保自动开启画中画开关已启用。
   
2. 在 Xcode 项目中：
   1. 选择 App 的 Target。
   2. 单击 Signing & Capabilities 页签。
   3. 单击 + Capability 按钮添加 Background Modes 功能，勾选 Audio, AirPlay, and Picture in Picture。
      
3. 参考以下代码，在需要开启画中画的页面配置 AudioSession，以便系统正确处理画中画模式下的音频播放。

   AVAudioSession *audioSession = [AVAudioSession sharedInstance];
   NSError *categoryError = nil;
   [audioSession setCategory:AVAudioSessionCategoryPlayback
                      mode:AVAudioSessionModeMoviePlayback
                   options:AVAudioSessionCategoryOptionOverrideMutedMicrophoneInterruption
                     error:&categoryError];
   if (categoryError) {
       NSLog(@"volc--set audio session category error: %@", categoryError.localizedDescription);
   }
   NSError *activeError = nil;
   [audioSession setActive:YES error:&activeError];
   if (activeError) {
       NSLog(@"volc--set audio session active error: %@", activeError.localizedDescription);
   }
   

步骤 2：配置显示视图和画中画控制器

1. 参考以下代码创建并配置显示视图。

   - (void)__setupDisplayerView {
       self.displayView = [[VEVideoPlayerDisplayView alloc] init];
       self.displayView.userInteractionEnabled = NO;
       self.displayView.clipsToBounds = YES;
       self.displayLayer = (AVSampleBufferDisplayLayer *)self.displayView.layer;
       self.displayLayer.opaque = YES;
       self.displayLayer.videoGravity = AVLayerVideoGravityResizeAspect;
       [self.view insertSubview:self.displayView aboveSubview:self.posterImageView];
       [self.displayView mas_makeConstraints:^(MASConstraintMaker *make) {
           make.edges.equalTo(self.view);
       }];
   }
   

2. 参考以下代码创建并配置 pipController。AVPictureInPictureController是 iOS 系统管理画中画生命周期的核心控制器，通过它来启动和停止画中画模式。

   - (void)__setupPipController {
       [self __updateAudioSession];
       AVPictureInPictureControllerContentSource *contentSource = [[AVPictureInPictureControllerContentSource alloc] initWithSampleBufferDisplayLayer:self.displayLayer playbackDelegate:self];
       self.pipController = [[AVPictureInPictureController alloc] initWithContentSource:contentSource];
       self.pipController.canStartPictureInPictureAutomaticallyFromInline = YES;
       self.pipController.requiresLinearPlayback = YES;
       self.pipController.delegate = self;
   }
   

步骤 3：配置视频帧回调处理

需要设置视频帧回调，以便将视频帧同时渲染到主窗口和画中画窗口。

1. 配置 TTVideoEngine 的视频帧回调：

   static void process(void *context, CVPixelBufferRef frame, int64_t timestamp) {
       NSLog(@"volc--frame=%@, ts=%.f", frame, timestamp);
       id ocContext = (__bridge id)context;
       VEVideoPlayerController *controller = ocContext;
       [controller __dispatchPixelBuffer:frame];
   }
   
   static void release(void *context) {
       NSLog(@"volc--frame release");
   }
   
   - (void)__startObserveVideoFrame {
       EngineVideoWrapper *wrapper = malloc(sizeof(EngineAudioWrapper));
       wrapper->process = process;
       wrapper->release = release;
       wrapper->context = (__bridge void *)self;
       [self.videoEngine setVideoWrapper:wrapper];
   }
   

2. 实现视频帧处理和渲染方法：

   - (void)__dispatchPixelBuffer:(CVPixelBufferRef)pixelBuffer {
       if (!pixelBuffer) {
           return;
       }
       CMSampleTimingInfo timing = {kCMTimeInvalid, kCMTimeInvalid, kCMTimeInvalid};
       CMVideoFormatDescriptionRef videoInfo = NULL;
       OSStatus result = CMVideoFormatDescriptionCreateForImageBuffer(NULL, pixelBuffer, &videoInfo);
       NSParameterAssert(result == 0 && videoInfo != NULL);
       CMSampleBufferRef sampleBuffer = NULL;
       result = CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault,pixelBuffer, true, NULL, NULL, videoInfo, &timing, &sampleBuffer);
       NSParameterAssert(result == 0 && sampleBuffer != NULL);
       CFRelease(videoInfo);
       CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);
       CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
       CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
       [self enqueueSampleBuffer:sampleBuffer toLayer:self.displayLayer];
       CFRelease(sampleBuffer);
   }
   
   - (void)enqueueSampleBuffer:(CMSampleBufferRef)sampleBuffer toLayer:(AVSampleBufferDisplayLayer*)layer {
       if (!sampleBuffer || !layer.readyForMoreMediaData) {
           NSLog(@"volc--sampleBuffer invalid");
           return;
       }
       if (@available(iOS 16.0, *)) {
           if (layer.status == AVQueuedSampleBufferRenderingStatusFailed) {
               NSLog(@"volc--sampleBufferLayer error:%@",layer.error);
               [layer flush];
           }
       } else {
           [layer flush];
       }
       if (@available(iOS 15.0, *)) {
           [layer enqueueSampleBuffer:sampleBuffer];
       } else {
           VEPlayerContextRunOnMainThread(^{
               [layer enqueueSampleBuffer:sampleBuffer];
           });
       }
   }
   

步骤 4：开启或关闭画中画功能

在用户界面中添加画中画控制按钮，实现开启或关闭画中画功能。

1. 实现画中画切换方法：

   if (self.pipController.isPictureInPictureActive) {
       [self.pipController stopPictureInPicture];
   } else {
       [self.pipController startPictureInPicture];
   }
   

2. 将此方法绑定到用户界面上的画中画按钮。用户可以通过点击画中画按钮在全屏播放和画中画模式之间切换，实现视频内容的无缝观看体验。