Qt C++应用下载大文件时遭遇std::bad_alloc错误求助

问题概述 #

开发Qt C++应用时，使用QNetworkAccessManager从远程服务器下载文件，仅在处理一个约363MB的特定文件时触发std::bad_alloc内存分配异常，较小文件可正常下载，系统内存充足但问题依旧。

关键细节 #

• 基于Qt的QNetworkAccessManager实现下载逻辑
• 仅363MB文件触发崩溃，小文件无异常
• 异常始终发生在文件内存分配阶段

已尝试的优化方案 #

• 使用QByteArray+QIODevice分块读写，避免调用readAll()一次性加载整个文件
• 调用deleteLater()进行资源清理
• 确认系统剩余内存充足

现有代码 #

下载请求触发函数 #

void NetworkManager::downloadDataFromServer(JsonManager *jdoc){
//    QQueue<QNetworkRequest> downloadQueue;
    QString base = jdoc->baseUrl;
//    QNetworkRequest request;
    m_totalFileCount = jdoc->files.size();

    for (int i = 0; i < m_totalFileCount; i++) {
        QJsonObject jsonObject = jdoc->files.at(i).toObject();
        QUrl reqUrl(base + jsonObject.value("url").toString());
        QNetworkRequest areq;
        areq.setUrl(reqUrl);

        QNetworkReply *reply = assetManager->get(areq);
        qDebug()<<"requesting:"<<jsonObject.value("url").toString();
        // Connect each reply's finished signal to a slot for handling it
        connect(reply, &QNetworkReply::finished, this, &NetworkManager::onAssets);
    }

}

下载完成处理槽函数 #

void NetworkManager::onAssets()
{
    QNetworkReply *reply = qobject_cast<QNetworkReply*>(sender());
    if(reply->url().toString().contains("91c11a970454232586c0bc9903af9505"))
    {
        qDebug()<<"hereerereeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee";
    }

    if (!reply) {
        // Handle the case where sender() did not return a valid reply
        return;
    }

    // If an error occurs in the process of obtaining data
    if(reply->error()){
        // We inform about it and show the error information
        qDebug() << "ERROR";
        qDebug() << reply->errorString();
        emit error(reply->errorString());
    } else {
        QString filePath = QCoreApplication::applicationDirPath() + "/app/"+reply->url().toString().replace(baseUrl,"");
        QFileInfo fileInfo(filePath);
        QDir dir(fileInfo.absolutePath());

        if (!dir.exists()) {
            if (dir.mkpath(fileInfo.absolutePath())) {
//                qDebug() << "Directory structure created: " << fileInfo.absolutePath();
            } else {
                qDebug() << "Failed to create directory structure: " << fileInfo.absolutePath();
            }
        }
        QFile file(filePath);
        if (file.open(QIODevice::WriteOnly))
        {
            QIODevice* source = reply; // No readAll() here
            while (!source->atEnd()) {
                QByteArray buffer = source->read(8192); // Read a chunk
                file.write(buffer); // Write the chunk to the file
            }
            file.close();
            m_downloadedFileCount++;
            float percent =(float) m_downloadedFileCount/(float)m_totalFileCount;
            emit downloadedFileCountChanged(percent, filePath);
        }
        else
        {
            qDebug() << "Failed to create or save asset file.";
        }
    }
    reply->deleteLater();
}

核心问题分析 #

现有代码使用finished信号处理下载，此时QNetworkReply已经将整个文件内容缓存到内存，363MB的数据直接占用大量内存，触发std::bad_alloc。即使在槽函数中分块读取，也无法改变reply已经预加载全部数据到内存的事实。

解决方案 #

1. 限制QNetworkReply的内存缓存大小 #

在创建reply后立即设置读取缓冲区大小，避免reply缓存整个文件：

QNetworkReply *reply = assetManager->get(areq);
reply->setReadBufferSize(1024 * 1024); // 设置1MB缓存上限

2. 改用readyRead信号实时读写数据 #

放弃finished信号一次性处理，改用readyRead信号在数据到达时立即写入文件，从根源避免内存堆积：

修改下载请求的信号连接

// 替换原有的finished信号连接
connect(reply, &QNetworkReply::readyRead, this, [this, reply]() {
    this->onReadyRead(reply);
});
connect(reply, &QNetworkReply::finished, this, [this, reply]() {
    this->onReplyFinished(reply);
});

新增实时读写槽函数

void NetworkManager::onReadyRead(QNetworkReply* reply) {
    if (!reply) return;

    QString filePath = QCoreApplication::applicationDirPath() + "/app/" + reply->url().toString().replace(baseUrl, "");
    QFileInfo fileInfo(filePath);
    QDir dir(fileInfo.absolutePath());
    if (!dir.exists()) {
        dir.mkpath(fileInfo.absolutePath());
    }

    QFile file(filePath);
    if (file.open(QIODevice::Append)) {
        // 此处readAll仅读取当前缓存的部分数据，而非整个文件
        file.write(reply->readAll());
        file.close();
    }
}

新增下载完成收尾槽函数

void NetworkManager::onReplyFinished(QNetworkReply* reply) {
    if (!reply) return;

    if (reply->error()) {
        qDebug() << "ERROR:" << reply->errorString();
        emit error(reply->errorString());
    } else {
        m_downloadedFileCount++;
        float percent = static_cast<float>(m_downloadedFileCount) / m_totalFileCount;
        QString filePath = QCoreApplication::applicationDirPath() + "/app/" + reply->url().toString().replace(baseUrl, "");
        emit downloadedFileCountChanged(percent, filePath);
    }

    reply->deleteLater();
}

3. 优化文件IO逻辑（可选） #

为避免每次readyRead都重复打开/关闭文件，可在发起请求时预先创建并打开文件，通过哈希表关联reply与文件对象：

类成员添加哈希表

class NetworkManager {
    // ... 其他成员
private:
    QHash<QNetworkReply*, QFile*> m_replyFileMap;
};

发起请求时初始化文件

// 在downloadDataFromServer的循环内，创建reply后添加：
QString filePath = QCoreApplication::applicationDirPath() + "/app/" + reqUrl.toString().replace(base, "");
QFileInfo fileInfo(filePath);
QDir dir(fileInfo.absolutePath());
if (!dir.exists()) {
    dir.mkpath(fileInfo.absolutePath());
}

QFile* file = new QFile(filePath);
if (file->open(QIODevice::WriteOnly)) {
    m_replyFileMap.insert(reply, file);
} else {
    qDebug() << "Failed to create file:" << filePath;
    reply->abort();
    delete file;
}

修改onReadyRead和onReplyFinished

void NetworkManager::onReadyRead(QNetworkReply* reply) {
    if (!reply) return;

    QFile* file = m_replyFileMap.value(reply);
    if (file) {
        file->write(reply->readAll());
    }
}

void NetworkManager::onReplyFinished(QNetworkReply* reply) {
    if (!reply) return;

    QFile* file = m_replyFileMap.take(reply);
    if (file) {
        file->close();
        delete file;
    }

    // ... 原有错误处理与进度更新逻辑
}

4. 限制并发下载数量（可选） #

如果同时下载多个大文件，即使单个文件缓存受限，叠加后仍可能触发内存问题。可实现下载队列，控制同时下载的文件数量（比如最多3个），完成一个再启动下一个。

内容的提问来源于stack exchange，提问作者Kitty Kagaya