Zlib Node.js报错：invalid stored block lengths 问题求助

Hey there! Let's walk through what's causing this issue and how to fix it properly, instead of relying on a temporary byte-trimming workaround.

What's the Root Problem? #

Your C code is generating zlib-formatted compressed data, while Node.js's inflateRaw expects raw deflate data (no zlib headers or trailing checksums).

• The first two bytes 78 da you spotted are the standard zlib header: 78 is the CMF (Compression Method and Flags) byte, and da is the FLG (Flags) byte indicating maximum compression level.
• The tail differences you noticed happen because your C code allocates a buffer of the original uncompressed size (malloc(size)) but only fills out_size bytes of it—so the remaining bytes are uninitialized garbage. Node.js's deflateRawSync generates exactly the right length of compressed data, hence the full buffer mismatch.

How to Fix the C Compression Code #

To make your C code output raw deflate data (matching what Node.js's inflateRaw expects), you need to use deflateInit2 instead of deflateInit. This function lets you disable the zlib header and trailing ADLER32 checksum.

Here's the modified version of your compress_image function:

#include <zlib.h>
#include <assert.h>
#include <stdlib.h>

typedef unsigned char* compressed_image;

int compress_image(compressed_image raw, size_t size, compressed_image *out) {
    z_stream strm = {0};
    int ret;
    size_t out_size = 0;

    // Allocate initial buffer (starting with original size is safe)
    *out = (compressed_image)malloc(size);
    if (*out == NULL) return Z_MEM_ERROR;

    // Initialize deflate for RAW format (no zlib header/trailer)
    ret = deflateInit2(
        &strm,
        Z_BEST_COMPRESSION,    // Max compression level
        Z_DEFLATED,            // Only valid compression method for deflate
        -MAX_WBITS,            // Negative value = raw deflate (skip header)
        8,                     // Memory level (1-9, 8 is default)
        Z_DEFAULT_STRATEGY     // Default strategy works for most data types
    );
    if (ret != Z_OK) {
        free(*out);
        return ret;
    }

    strm.avail_in = size;
    strm.next_in = (unsigned char *)raw;

    do {
        strm.avail_out = size - out_size;
        strm.next_out = *out + out_size;
        ret = deflate(&strm, Z_FINISH);
        assert(ret != Z_STREAM_ERROR);
        out_size += (size - out_size) - strm.avail_out;
    } while (strm.avail_out == 0);

    // Clean up deflate stream
    (void)deflateEnd(&strm);

    // Optional: Shrink buffer to exact compressed size to avoid sending garbage tail bytes
    compressed_image temp = realloc(*out, out_size);
    if (temp != NULL) {
        *out = temp;
    }

    return out_size;
}

Key Changes Explained: #

• deflateInit2 with -MAX_WBITS: This tells zlib to produce raw deflate data without the zlib header and trailing ADLER32 checksum, which aligns perfectly with Node.js's inflateRaw.
• Adjusted buffer tracking: We now write to the buffer starting from the end of already filled data, preventing overwrites and ensuring accurate size calculation.
• Optional realloc: Shrinks the buffer to the exact compressed size, eliminating uninitialized tail bytes and fixing the buffer mismatch you observed.

Node.js Side Adjustment #

With the modified C code, you no longer need to trim the first two bytes! Your Node.js code can directly use inflateRaw on the received buffer:

const zlib = require('zlib');
const formidable = require('formidable');
const fs = require('fs');
const dump = require('buffer-hexdump');

router.post('/test', function(req, res, next) {
    let form = new formidable.IncomingForm();
    form.uploadDir = "./uploads";
    form.parse(req, function(err, fields, files) {
        if (err) {
            console.error(err);
            return res.status(500).send("Error parsing form");
        }

        const imageBuff = fs.readFileSync(files.image.path);
        
        zlib.inflateRaw(imageBuff, (err, buffer) => {
            if (err) {
                console.error(err);
                res.status(500).send("Decompression failed");
            } else {
                console.log("Decompressed data:", buffer.toString());
                res.status(200).send("POST received and decompressed successfully!");
            }
        });
    });
});

Why Your Temporary Workaround Worked (But Isn't Ideal) #

Trimming the first two bytes removed the zlib header, making the rest of the data look like raw deflate. The tail garbage bytes were harmless because zlib's decompressor stops when it reaches the end of the compressed data stream. However, sending extra uninitialized bytes is inefficient and could cause unexpected edge-case issues.

内容的提问来源于stack exchange，提问作者santiaguzz