MediaMuxer合并图片与视频报错：Stop() called but track is not started or stopped

• 执行代码时触发错误提示：Stop() called but track is not started or stopped
• 输出视频损坏，未包含图片合并内容

问题根源分析 #

1. 音频轨道未写入数据：代码仅处理了视频帧的写入，完全忽略原视频的音频数据，导致MediaMuxer的音频轨道处于未激活状态，调用stop()时触发错误。
2. 视频帧处理逻辑错误：直接将MediaMetadataRetriever提取的Bitmap像素数据写入Muxer是错误的——Muxer需要的是编码后的视频流数据，而非原始像素，这会导致视频文件损坏。
3. 资源释放时机不合理：协程完成后直接释放资源，未确保所有音视频数据写入完成，且MediaExtractor等资源未被正确释放。
4. 时间戳计算不可靠：依赖METADATA_KEY_VIDEO_FRAME_COUNT和METADATA_KEY_CAPTURE_FRAMERATE计算帧时间戳，这两个元数据并非所有视频都提供，容易导致时间轴混乱。

修复后的完整代码 #

核心合并函数 #

private fun mergeImageWithVideo(videoPath: String, imagePath: String) {
    val outputFile = File(getExternalFilesDir(null), "merged_video.mp4")
    if (outputFile.exists()) outputFile.delete()

    val mediaMetadataRetriever = MediaMetadataRetriever()
    mediaMetadataRetriever.setDataSource(videoPath)
    val overlayBitmap = BitmapFactory.decodeFile(imagePath) ?: run {
        showELog("Failed to load overlay image")
        return
    }

    val videoExtractor = MediaExtractor()
    videoExtractor.setDataSource(videoPath)

    // 获取音视频轨道索引
    val audioTrackIndex = getAudioTrackIndex(videoExtractor)
    val videoTrackIndex = getVideoTrackIndex(videoExtractor)

    // 获取音视频格式
    val audioFormat = videoExtractor.getTrackFormat(audioTrackIndex)
    val videoFormat = videoExtractor.getTrackFormat(videoTrackIndex)

    // 初始化MediaMuxer
    val mediaMuxer = MediaMuxer(outputFile.absolutePath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)
    val muxerAudioTrackIndex = mediaMuxer.addTrack(audioFormat)
    val muxerVideoTrackIndex = mediaMuxer.addTrack(videoFormat)

    // 准备视频编码：将Bitmap叠加后编码为标准视频流
    val videoWidth = videoFormat.getInteger(MediaFormat.KEY_WIDTH)
    val videoHeight = videoFormat.getInteger(MediaFormat.KEY_HEIGHT)
    val frameRate = videoFormat.getInteger(MediaFormat.KEY_FRAME_RATE, 30)
    val bitRate = videoFormat.getInteger(MediaFormat.KEY_BIT_RATE, 2 * 1024 * 1024)

    val codecFormat = MediaFormat.createVideoFormat(videoFormat.getString(MediaFormat.KEY_MIME)!!, videoWidth, videoHeight).apply {
        setInteger(MediaFormat.KEY_FRAME_RATE, frameRate)
        setInteger(MediaFormat.KEY_BIT_RATE, bitRate)
        setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 10)
        setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface)
    }

    val videoCodec = MediaCodec.createEncoderByType(codecFormat.getString(MediaFormat.KEY_MIME)!!)
    videoCodec.configure(codecFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
    val inputSurface = videoCodec.createInputSurface()
    videoCodec.start()

    mediaMuxer.start()

    lifecycleScope.launch(Dispatchers.IO) {
        try {
            // 处理音频：直接复制原视频音频数据
            processAudioTrack(videoExtractor, audioTrackIndex, mediaMuxer, muxerAudioTrackIndex)
            // 处理视频：叠加图片后编码写入
            processVideoTrack(
                mediaMetadataRetriever,
                videoExtractor,
                videoTrackIndex,
                videoCodec,
                inputSurface,
                overlayBitmap,
                mediaMuxer,
                muxerVideoTrackIndex
            )
        } catch (e: Exception) {
            showELog("Merge failed: ${e.message}")
        } finally {
            // 释放所有资源
            videoCodec.stop()
            videoCodec.release()
            inputSurface.release()
            mediaMuxer.stop()
            mediaMuxer.release()
            videoExtractor.release()
            mediaMetadataRetriever.release()
            overlayBitmap.recycle()
            showELog("Merge completed: ${outputFile.absolutePath}")
        }
    }
}

音频轨道处理函数 #

private fun processAudioTrack(
    extractor: MediaExtractor,
    trackIndex: Int,
    muxer: MediaMuxer,
    muxerTrackIndex: Int
) {
    extractor.selectTrack(trackIndex)
    val buffer = ByteBuffer.allocate(1024 * 1024)
    val bufferInfo = MediaCodec.BufferInfo()

    while (true) {
        val sampleSize = extractor.readSampleData(buffer, 0)
        if (sampleSize < 0) break

        bufferInfo.apply {
            offset = 0
            size = sampleSize
            presentationTimeUs = extractor.sampleTime
            flags = extractor.sampleFlags
        }

        muxer.writeSampleData(muxerTrackIndex, buffer, bufferInfo)
        extractor.advance()
    }

    extractor.unselectTrack(trackIndex)
}

视频轨道处理函数 #

private fun processVideoTrack(
    retriever: MediaMetadataRetriever,
    extractor: MediaExtractor,
    trackIndex: Int,
    codec: MediaCodec,
    inputSurface: Surface,
    overlayBitmap: Bitmap,
    muxer: MediaMuxer,
    muxerTrackIndex: Int
) {
    extractor.selectTrack(trackIndex)
    val bufferInfo = MediaCodec.BufferInfo()
    val canvas = inputSurface.lockCanvas(null)

    while (true) {
        val sampleTime = extractor.sampleTime
        if (sampleTime < 0) break

        // 获取当前帧并叠加图片
        val videoFrame = retriever.getFrameAtTime(sampleTime, MediaMetadataRetriever.OPTION_CLOSEST_SYNC) ?: break
        val overlayedFrame = overlayImageOnFrame(videoFrame, overlayBitmap)

        // 将叠加后的帧绘制到Surface供编码
        canvas.drawBitmap(overlayedFrame, 0f, 0f, null)
        inputSurface.unlockCanvasAndPost(canvas)

        // 获取编码后的输出数据并写入Muxer
        var outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, 10000)
        while (outputBufferIndex >= 0) {
            val outputBuffer = codec.getOutputBuffer(outputBufferIndex)
            outputBuffer?.let {
                it.position(bufferInfo.offset)
                it.limit(bufferInfo.offset + bufferInfo.size)
                muxer.writeSampleData(muxerTrackIndex, it, bufferInfo)
            }
            codec.releaseOutputBuffer(outputBufferIndex, false)
            outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, 0)
        }

        videoFrame.recycle()
        overlayedFrame.recycle()
        extractor.advance()
    }

    // 发送编码结束信号，处理剩余数据
    codec.signalEndOfInputStream()
    var outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, 10000)
    while (outputBufferIndex >= 0) {
        val outputBuffer = codec.getOutputBuffer(outputBufferIndex)
        outputBuffer?.let {
            it.position(bufferInfo.offset)
            it.limit(bufferInfo.offset + bufferInfo.size)
            muxer.writeSampleData(muxerTrackIndex, it, bufferInfo)
        }
        codec.releaseOutputBuffer(outputBufferIndex, false)
        outputBufferIndex = codec.dequeueOutputBuffer(bufferInfo, 0)
    }

    extractor.unselectTrack(trackIndex)
}

图片叠加函数（优化资源回收） #

private fun overlayImageOnFrame(videoFrame: Bitmap, overlayBitmap: Bitmap): Bitmap {
    val resultBitmap = Bitmap.createBitmap(videoFrame.width, videoFrame.height, videoFrame.config)
    val canvas = Canvas(resultBitmap)
    canvas.drawBitmap(videoFrame, 0f, 0f, null)

    val scaledOverlay = if (videoFrame.width != overlayBitmap.width || videoFrame.height != overlayBitmap.height) {
        Bitmap.createScaledBitmap(overlayBitmap, videoFrame.width, videoFrame.height, false)
    } else {
        overlayBitmap
    }
    canvas.drawBitmap(scaledOverlay, 0f, 0f, null)

    if (scaledOverlay !== overlayBitmap) {
        scaledOverlay.recycle()
    }
    return resultBitmap
}

轨道索引获取函数（保留原逻辑） #

private fun getVideoTrackIndex(extractor: MediaExtractor): Int {
    for (i in 0 until extractor.trackCount) {
        val format = extractor.getTrackFormat(i)
        val mime = format.getString(MediaFormat.KEY_MIME)
        if (mime?.startsWith("video/") == true) {
            return i
        }
    }
    throw RuntimeException("Video track not found")
}

private fun getAudioTrackIndex(extractor: MediaExtractor): Int {
    for (i in 0 until extractor.trackCount) {
        val format = extractor.getTrackFormat(i)
        val mime = format.getString(MediaFormat.KEY_MIME)
        if (mime?.startsWith("audio/") == true) {
            return i
        }
    }
    throw RuntimeException("Audio track not found")
}

关键修复说明 #

1. 新增音频处理逻辑：直接复制原视频的音频数据到输出文件，确保MediaMuxer的音频轨道有数据写入，解决stop()报错问题。
2. 标准视频编码流程：使用MediaCodec将叠加后的Bitmap编码为符合规范的视频流，再写入Muxer，保证输出视频可正常播放。
3. 资源优化：及时回收Bitmap、释放Codec和Surface资源，避免内存泄漏。
4. 时间戳同步：使用原视频帧的实际时间戳，确保音视频同步。

内容的提问来源于stack exchange，提问作者ANANDA SANTRA