结合BufferedImage与WAV音频生成视频时FFmpegFrameRecorder报错无音频输出流

尝试用JavaCV将BufferedImage数组与WAV音频结合生成MP4视频，编写代码如下：

public static void compileVideo(BufferedImage[] bufferedImages, String audioPath , String savePath) throws FFmpegFrameRecorder.Exception, FrameGrabber.Exception {
        FrameGrabber grabber = new FFmpegFrameGrabber(audioPath); // need this for audio
        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(savePath, 316, 100, grabber.getAudioChannels());
        FFmpegLogCallback.set();

        
        //grabber.setAudioChannels(grabber.getAudioChannels());
        System.out.println(grabber.getAudioChannels());
        // set up the recorder video info
        recorder.setFrameRate(60);
        recorder.setFormat("mp4");
        recorder.setVideoQuality(0); // highest quality

        // set recorder audio info
        recorder.setSampleFormat(grabber.getSampleFormat());
        recorder.setSampleRate(grabber.getSampleRate());
        recorder.setAudioQuality(0); // highest quality
        
        grabber.start();
        recorder.start();

        for (BufferedImage image: bufferedImages) {
            recorder.record(Java2DFrameUtils.toFrame(image)); // convert buffered image to frame and record it
            recorder.record(grabber.grabFrame()); //TODO: Grab multiple audio frames because there are more audio frames than video frames (BufferedImages)
        }
        recorder.stop();
        grabber.stop();
    }

运行时抛出错误：

Exception in thread "main" org.bytedeco.javacv.FFmpegFrameRecorder$Exception: No audio output stream (Is audioChannels > 0 and has start() been called?) (For more details, make sure FFmpegLogCallback.set() has been called.)
    at org.bytedeco.javacv.FFmpegFrameRecorder.recordSamples(FFmpegFrameRecorder.java:1154)
    at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:1021)
    at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:1010)
    at com.Lazzza.imageToVideo.compileVideo(imageToVideo.java:31)
    at com.Lazzza.Main.main(Main.java:22)

已知grabber.getAudioChannels()返回值为2，音频文件为WAV格式。

1. 调整初始化顺序：先启动音频抓取器再创建录制器 #

当前代码在grabber.start()之前就调用grabber.getAudioChannels()等参数，虽然返回了2，但部分音频内部参数可能未正确加载。必须先启动抓取器，再用其参数初始化录制器，确保所有音频元数据加载完成。

2. 显式设置音频编码器 #

MP4容器默认需要AAC编码的音频，而WAV是无压缩格式，需显式指定音频编码器，避免音频流初始化失败：

recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);

3. 修复音视频帧同步逻辑 #

原代码每帧视频对应一帧音频，会导致音视频时长不匹配。需根据视频总时长计算需要读取的音频帧数量，保证两者时长一致。

修改后的完整代码 #

import org.bytedeco.ffmpeg.avcodec;
import org.bytedeco.javacv.*;

import java.awt.image.BufferedImage;

public class ImageToVideo {
    public static void compileVideo(BufferedImage[] bufferedImages, String audioPath, String savePath) throws FFmpegFrameRecorder.Exception, FrameGrabber.Exception {
        FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(audioPath);
        FFmpegLogCallback.set();

        // 先启动音频抓取器，确保所有音频参数加载完成
        grabber.start();

        // 用已加载的音频参数初始化录制器
        int videoWidth = 316;
        int videoHeight = 100;
        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(savePath, videoWidth, videoHeight, grabber.getAudioChannels());

        // 视频配置
        recorder.setFrameRate(60);
        recorder.setFormat("mp4");
        recorder.setVideoQuality(0);

        // 音频配置：显式设置编码器，MP4需AAC编码
        recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
        recorder.setSampleFormat(grabber.getSampleFormat());
        recorder.setSampleRate(grabber.getSampleRate());
        recorder.setAudioQuality(0);

        recorder.start();

        // 计算视频总时长（秒）
        double videoTotalDuration = (double) bufferedImages.length / recorder.getFrameRate();
        // 计算需要读取的音频样本总数
        long totalAudioSamples = (long) (videoTotalDuration * grabber.getSampleRate() * grabber.getAudioChannels());
        long recordedSamples = 0;

        // 先写入所有视频帧
        for (BufferedImage image : bufferedImages) {
            recorder.record(Java2DFrameUtils.toFrame(image));
        }

        // 读取并写入对应时长的音频帧
        Frame audioFrame;
        while (recordedSamples < totalAudioSamples && (audioFrame = grabber.grabFrame(false, true, false, false)) != null) {
            recorder.record(audioFrame);
            // 累加已录制的样本数
            recordedSamples += audioFrame.samples.length * grabber.getAudioChannels();
        }

        recorder.stop();
        grabber.stop();
        recorder.release();
        grabber.release();
    }
}

内容的提问来源于stack exchange，提问作者Lazzza