Android Media3如何实现单帧带特效的预览效果？

我正在开发一款视频编辑器，选用了Media3库来实现视频特效，但遇到了一个问题：无法为每个滤镜渲染带特效的视频单帧预览（类似附图效果）。目前我能通过PixelCopy获取当前播放视频的缩略图，但只能在playerView可见时才能获取，无法批量生成不同特效的单帧预览。

现有ExoPlayer应用特效代码 #

val path = "android.resource://" + packageName + "/" + R.raw.sample

val mediaItem = MediaItem.Builder().setUri(Uri.parse(path)).build()
player = ExoPlayer.Builder(this).build()
    .also { exoPlayer ->
        playerView?.player = exoPlayer
        exoPlayer.setMediaItem(mediaItem)
        exoPlayer.setVideoEffects(listOf(
            HslAdjustment.Builder().adjustHue(0f).adjustSaturation(50f).build(),
            RgbFilter.createInvertedFilter()
        ))
        exoPlayer.prepare()
    }

现有PixelCopy获取缩略图代码 #

private fun setVideoThumbnail() {
    val textureView = playerView?.videoSurfaceView as SurfaceView
    val bitmap = Bitmap.createBitmap(500, 500, Bitmap.Config.ARGB_8888)
    val listener =
        OnPixelCopyFinishedListener { copyResult: Int ->
            if (copyResult == PixelCopy.SUCCESS) {
                bitmap.recycle()
            }
        }
    PixelCopy.request(textureView, bitmap, listener, thumbnailImage!!.handler)

    thumbnailImage?.setImageBitmap(bitmap)
    thumbnailImage?.visibility = View.VISIBLE
}

解决方案：脱离PlayerView渲染单帧特效预览 #

方法1：使用Media3解码器离线渲染单帧 #

不需要依赖可见的PlayerView，直接通过Media3的解码能力提取帧并应用特效：

• 用MediaExtractor定位到目标时间戳的视频帧，通过解码器获取原始帧数据
• 利用Media3的VideoEffect底层逻辑，将特效应用到解码后的帧
• 转换为Bitmap后作为预览图

示例代码片段：

// 提取视频帧
val extractor = MediaExtractor()
extractor.setDataSource(context, Uri.parse(videoPath), null)
val videoTrackIndex = extractor.trackIndices.first { extractor.getTrackFormat(it).sampleMimeType?.startsWith("video/") == true }
extractor.selectTrack(videoTrackIndex)
extractor.seekTo(targetTimestampUs, MediaExtractor.SEEK_TO_CLOSEST_SYNC)

// 初始化解码器
val decoderFactory = DefaultDecoderFactory(context)
val decoder = decoderFactory.createVideoDecoder(extractor.getTrackFormat(videoTrackIndex))
val surfaceHelper = TextureSurfaceHelper.create(context, null)
decoder.setOutputSurface(surfaceHelper.surface)

// 解码帧
val inputBuffer = decoder.dequeueInputBuffer(0)
if (inputBuffer != null) {
    val sampleSize = extractor.readSampleData(inputBuffer.data, 0)
    if (sampleSize >= 0) {
        inputBuffer.timeUs = extractor.sampleTime
        decoder.queueInputBuffer(inputBuffer)
    }
}

// 获取解码帧并应用特效
val outputBuffer = decoder.dequeueOutputBuffer()
if (outputBuffer != null && outputBuffer.isFrame) {
    // 将帧转换为Bitmap
    val bitmap = Bitmap.createBitmap(outputBuffer.width, outputBuffer.height, Bitmap.Config.ARGB_8888)
    val canvas = Canvas(bitmap)
    surfaceHelper.surface?.let { canvas.drawBitmap(bitmap, 0f, 0f, null) }
    
    // 应用特效（可通过OpenGL或RenderScript实现Media3滤镜逻辑）
    applyFilterEffects(bitmap, listOf(HslAdjustment.Builder().adjustSaturation(50f).build()))
    
    // 保存或展示预览图
}

方法2：用不可见SurfaceView批量渲染 #

创建一个不可见的SurfaceView（仅设置INVISIBLE而非GONE，保证Surface正常创建），将ExoPlayer输出定向到该Surface，循环切换特效生成预览：

• 初始化后台SurfaceView，添加到父布局中
• 依次为ExoPlayer设置不同特效，seek到目标帧，监听帧渲染完成事件
• 用PixelCopy获取每个特效对应的Bitmap

优化后的代码：

// 创建后台不可见SurfaceView
val hiddenSurfaceView = SurfaceView(context).apply {
    layoutParams = ViewGroup.LayoutParams(500, 500)
    visibility = View.INVISIBLE
    rootView.addView(this)
}

// 批量生成多特效预览
fun generateFilterPreviews(targetTimestampUs: Long, effectsList: List<List<VideoEffect>>) {
    val player = ExoPlayer.Builder(context).build().apply {
        setMediaItem(mediaItem)
        prepare()
        seekTo(targetTimestampUs)
        setVideoSurfaceView(hiddenSurfaceView)
    }

    effectsList.forEachIndexed { index, effects ->
        player.setVideoEffects(effects)
        player.addListener(object : Player.Listener {
            override fun onRenderedFirstFrame() {
                copySurfaceToBitmap(hiddenSurfaceView) { bitmap ->
                    // 保存当前特效的预览图
                    previewList.add(bitmap)
                    player.removeListener(this)
                    // 处理下一个特效
                    if (index < effectsList.size - 1) {
                        player.seekTo(targetTimestampUs)
                    } else {
                        player.release()
                    }
                }
            }
        })
    }
}

private fun copySurfaceToBitmap(surfaceView: SurfaceView, callback: (Bitmap) -> Unit) {
    val bitmap = Bitmap.createBitmap(500, 500, Bitmap.Config.ARGB_8888)
    PixelCopy.request(surfaceView, bitmap, { result ->
        if (result == PixelCopy.SUCCESS) {
            callback(bitmap)
        } else {
            bitmap.recycle()
        }
    }, Handler(Looper.getMainLooper()))
}

方法3：用RenderScript直接处理Bitmap #

如果只需要对单帧原始图应用特效，可跳过ExoPlayer，直接用RenderScript实现滤镜逻辑：

• 用MediaMetadataRetriever获取视频原始帧Bitmap
• 编写RenderScript内核实现Media3的HSL、RGB等滤镜效果
• 处理后的Bitmap直接作为预览图

示例（反转滤镜）：

// 获取原始帧
val retriever = MediaMetadataRetriever()
retriever.setDataSource(videoPath)
val originalBitmap = retriever.getFrameAtTime(targetTimestampUs, MediaMetadataRetriever.OPTION_CLOSEST_SYNC) ?: return

// RenderScript实现反转滤镜
val rs = RenderScript.create(context)
val input = Allocation.createFromBitmap(rs, originalBitmap)
val output = Allocation.createTyped(rs, input.type)
val script = ScriptC_invert(rs) // 自定义RenderScript脚本
script.forEach_invert(input, output)
output.copyTo(originalBitmap)
rs.destroy()

// originalBitmap即为带反转特效的预览图

内容的提问来源于stack exchange，提问作者thaovd1712