如何读取任意格式文件为字符串用于霍夫曼压缩及解决解压异常问题

Hey there, let's break down your problem clearly—first, we need to clarify what you're actually trying to do: are you looking to compress the entire binary data of the file (so you can decompress it back to the original doc/pdf/jpeg later), or do you want to extract readable text from these files (like the actual words in a doc, text in a pdf, or OCR from a jpeg) before running Huffman compression? These are two totally different paths, so let's cover both:

Scenario 1: Compress the entire file (restore original format later) #

Your current getDataFromFileToString function actually works fine for reading any file format—because you're opening it in std::ios::binary mode, which reads every single byte (including non-text bytes and null values) into the std::string. Huffman compression is a byte-based algorithm, so it doesn't care what those bytes represent; it just works with their frequency.

The likely reason your decompression is failing is that you're writing the decompressed data back to a file using text mode instead of binary mode. On systems like Windows, text mode will modify binary data (e.g., converting \n to \r\n, or truncating at null bytes), which breaks structured formats like doc/pdf/jpeg.

Fix for decompression: #

Make sure your file-writing function uses binary mode too:

void writeDataToFile(std::string& data, const std::string& fName) {
    std::ofstream fw(fName, std::ios_base::out | std::ios::binary);
    if (!fw.is_open()) {
        std::cerr << "Couldn't create output file \"" << fName << "\"\n";
        exit(EXIT_FAILURE);
    }
    // Write the raw bytes directly—don't treat the string as text
    fw.write(data.data(), data.size());
    fw.close();
}

Also, double-check your Huffman logic doesn't mess with the raw bytes: don't do any encoding conversions, don't filter out nulls, just treat the std::string as a container of bytes (which it is, despite the name).

Scenario 2: Extract readable text from files first #

If your goal is to compress only the actual text content (not the entire file), then reading raw binary bytes won't help—formats like doc, pdf, and jpeg are structured, so you need specialized libraries to parse them and pull out the text.

Recommended libraries for each format: #

• DOC/DOCX: Use libdocx (lightweight for docx) or poppler (also handles pdf). On Windows, you could also use Microsoft's COM interfaces to interact with Word, but that's more tied to the OS.
• PDF: poppler-cpp is a solid C++ choice for extracting text from PDFs.
• JPEG/Images: Use Tesseract OCR (paired with Leptonica for image processing) to extract text from images via optical character recognition.

Quick example: Extracting text from a PDF with poppler #

#include <poppler/cpp/poppler-document.h>
#include <poppler/cpp/poppler-page.h>
#include <string>

std::string getTextFromPDF(const std::string& fName) {
    std::string extractedText;
    auto doc = poppler::document::load_from_file(fName);
    if (!doc) {
        std::cerr << "Failed to open PDF file: " << fName << "\n";
        exit(EXIT_FAILURE);
    }

    for (int pageNum = 0; pageNum < doc->pages(); ++pageNum) {
        auto page = doc->create_page(pageNum);
        if (page) {
            extractedText += page->text().to_std_string();
            delete page;
        }
    }

    delete doc;
    return extractedText;
}

Heads up: #

• These libraries require installation and linking. For example, on Linux you'd install libpoppler-cpp-dev via your package manager, and link against poppler-cpp when compiling.
• OCR accuracy depends on image quality (resolution, clarity) and Tesseract's language packs (make sure you install the right one for your text language).

Wrap-up #

If you just need a general file compression tool (works on any format), fix your decompression write function to use binary mode—your existing read function is already good. If you need to compress only the text content, you'll need to add the appropriate parsing/OCR libraries to extract that text first.

内容的提问来源于stack exchange，提问作者John Born