使用Webcodec API解码x264 NALU时遇比特流缓冲区大小错误

核心问题分析 #

该错误源于解码器无法正确解析输入的NALU数据或配置信息，结合你的代码，主要存在以下几个关键问题：

1. AVC配置描述符（avcC）构造不符合标准 #

你的convertToAvcC方法中，avcC头部结构错误：

• 第5个字节错误设置为0xff，正确值应为0xe0（6位保留位 + 0个额外SPS，因为仅传入1个SPS）；
• PPS前的num_of_pps_minus_1字段错误设置为1，正确值应为0（仅传入1个PPS）。

2. NALU起始码处理逻辑错误 #

x264的NALU起始码为0x000001或0x00000001，你当前的处理会错误截断NALU头部，导致解码器无法识别NALU类型。

3. 重复发送SPS/PPS给解码器 #

配置解码器时已通过description传入SPS/PPS，但解码流程中又将这些配置帧作为普通帧发送，干扰了解码逻辑。

4. 时间戳（timestamp）不规范 #

使用Date.now() * 1000作为时间戳会导致同一帧的NALU时间戳不一致，解码器无法正确关联帧数据。

修复后的完整代码 #

class Decoder {
    private decoder!: VideoDecoder;
    private sps!: Uint8Array;
    private pps!: Uint8Array;
    private isConfigured = false;
    private nalBuffer: Uint8Array[] = [];
    private canvas: OffscreenCanvas | null = null;
    private lastTimestamp = 0n;

    set Canvas(canvas: OffscreenCanvas) {
        this.canvas = canvas;
    }

    constructor() {
        this.decoder = new VideoDecoder({
            output: (frame) => this.displayFrame(frame),
            error: (err) => {
                console.error('Decoder error:', err);
                this.decoder.close();
                this.isConfigured = false;
                this.nalBuffer = [];
            }
        });
    }

    async decodeNAL(nal: Uint8Array) {
        if (this.decoder.state === "closed") return;

        // 正确识别并移除NALU起始码
        let startCodeIndex = 0;
        if (nal[0] === 0x00 && nal[1] === 0x00) {
            startCodeIndex = nal[2] === 0x01 ? 3 : (nal[2] === 0x00 && nal[3] === 0x01 ? 4 : 0);
        }
        if (startCodeIndex > 0) {
            nal = nal.slice(startCodeIndex);
        }

        const nalType = nal[0] & 0x1F;

        switch (nalType) {
            case 7: // SPS
                if (!this.isConfigured) this.sps = nal;
                break;
            case 8: // PPS
                if (!this.isConfigured) this.pps = nal;
                break;
            case 5: // I-Frame
            case 1: // P-Frame
                if (!this.sps || !this.pps) {
                    console.warn("Skipping frame, waiting for configuration frames");
                    return;
                }
                if (!this.isConfigured) await this.configureDecoder();
                this.processNal(nal);
                break;
        }
    }

    private processNal(nal: Uint8Array) {
        const acceptedNalTypes = [1, 5];
        if (!acceptedNalTypes.includes(nal[0] & 0x1F)) return;

        this.nalBuffer.push(nal);
        this.decodeFrame();
    }

    private async configureDecoder() {
        try {
            const codecString = this.getAvcCodecString(this.sps);
            const description = this.convertToAvcC(this.sps, this.pps);

            const config: VideoDecoderConfig = {
                codec: codecString,
                description: description,
                hardwareAcceleration: "no-preference",
            };

            const isSupported = await VideoDecoder.isConfigSupported(config);
            if (!isSupported.supported) {
                console.error('Unsupported decoder config:', isSupported.config);
                throw new Error('No supported codec configuration found');
            }

            this.decoder.configure(config);
            this.isConfigured = true;
        } catch (error) {
            console.error('Decoder configuration failed:', error);
        }
    }

    private decodeFrame() {
        if (!this.isConfigured) {
            console.warn('Decoder not configured');
            return;
        }

        // 使用固定步长递增时间戳（适配30fps）
        this.lastTimestamp += 33333n;

        for (const nal of this.nalBuffer) {
            try {
                const chunkType = (nal[0] & 0x1F) === 5 ? 'key' : 'delta';
                const chunk = new EncodedVideoChunk({
                    type: chunkType,
                    data: nal,
                    timestamp: Number(this.lastTimestamp),
                    duration: 33333
                });

                this.decoder.decode(chunk);
            } catch (error) {
                console.error('Decoding error:', error);
            }
        }

        this.nalBuffer = [];
    }

    private getAvcCodecString(sps: Uint8Array): string {
        if (sps.length < 4) return 'avc1.42E01E';

        const profile_idc = sps[1].toString(16).padStart(2, '0').toUpperCase();
        const constraint_set = sps[2].toString(16).padStart(2, '0').toUpperCase();
        const level_idc = sps[3].toString(16).padStart(2, '0').toUpperCase();

        return `avc1.${profile_idc}${constraint_set}${level_idc}`;
    }

    private convertToAvcC(sps: Uint8Array, pps: Uint8Array): Uint8Array {
        // 按H.264标准构造avcC描述符
        const avcCLength = 8 + sps.length + 4 + pps.length;
        const avcC = new Uint8Array(avcCLength);

        // 基础头部：版本+profile+兼容位+level+SPS数量标识
        avcC.set([0x01, sps[1], sps[2], sps[3], 0xe0], 0);
        // SPS长度+数据
        avcC.set([(sps.length >> 8) & 0xff, sps.length & 0xff], 5);
        avcC.set(sps, 7);
        // PPS数量标识+长度+数据
        const ppsOffset = 7 + sps.length;
        avcC.set([0x00, (pps.length >> 8) & 0xff, pps.length & 0xff], ppsOffset);
        avcC.set(pps, ppsOffset + 3);

        return avcC;
    }

    private displayFrame(frame: VideoFrame) {
        if (!this.canvas) throw new Error('Canvas not configured');
        const ctx = this.canvas.getContext("2d");
        if (ctx) {
            ctx.drawImage(frame, 0, 0, this.canvas.width, this.canvas.height);
        }
        frame.close();
    }
}

let decoder: Decoder = new Decoder();

self.addEventListener("message", async (e) => {
    switch (e.data.type) {
        case "buffer":
            await decoder.decodeNAL(new Uint8Array(e.data.data));
            break;
        case "canvas":
            decoder.Canvas = e.data.data as OffscreenCanvas;
    }
});

关键修复说明 #

1. 规范avcC构造：严格按照H.264标准生成配置描述符，确保解码器能正确解析视频参数；
2. 修复NALU起始码处理：准确识别并移除两种格式的起始码，保留完整的NALU数据；
3. 移除重复配置帧发送：仅在解码器配置阶段传入SPS/PPS，解码流程中不再重复发送；
4. 统一时间戳：使用固定步长的递增时间戳，保证帧数据的时序一致性。

内容的提问来源于stack exchange，提问作者Casah__