使用simdjson解析JSON触发TAPE_ERROR但JSON结构无误的问题排查

程序抛出simdjson::simdjson_error异常终止：
what(): TAPE_ERROR: The JSON document has an improper structure: missing or superfluous commas, braces, missing keys, etc.

已确认JSON结构正确、文件路径无误，调试时打印download.client会输出整个内容，以下是相关代码与JSON内容，及对应的解决方法：

相关代码与JSON内容 #

Client解析代码 #

Client::Download Client::Download::parse(simdjson::ondemand::object &obj) {
  auto sha1 = obj["sha1"].get_string().value();
  auto size = obj["size"].get_int64().value();
  auto url = obj["url"].get_string().value();

  return {
    .sha1 = std::string(sha1),
    .size = size,
    .url = std::string(url)
  };
}

Client::ClientDownloads Client::ClientDownloads::parse(simdjson::ondemand::object &obj) {
  Client::ClientDownloads download;

  auto client_obj = obj["client"].get_object().value(); 
  auto client_mappings_obj = obj["client_mappings"].get_object().value();
  auto server_obj = obj["server"].get_object().value();
  auto server_mappings_obj = obj["server_mappings"].get_object().value();
  
  auto client = Client::Download::parse(client_obj);
  auto client_mappings = Client::Download::parse(client_mappings_obj);
  auto server = Client::Download::parse(server_obj);
  auto server_mappings = Client::Download::parse(server_mappings_obj);

  download.client = client;
  download.client_mappings = client_mappings;
  download.server = server;
  download.server_mappings = server_mappings;

  return download;
}

JSON文件内容 #

{
  "downloads": {
    "client": {
      "sha1": "9acca901e3564a91250b941cd2c55a55d0b71bca",
      "size": 28534222,
      "url": "https://piston-data.mojang.com/v1/objects/9acca901e3564a91250b941cd2c55a55d0b71bca/client.jar"
    },
    "client_mappings": {
      "sha1": "94b753018a4683ec7c25a33c9048d46fbf9a5db0",
      "size": 10413606,
      "url": "https://piston-data.mojang.com/v1/objects/94b753018a4683ec7c25a33c9048d46fbf9a5db0/client.txt"
    },
    "server": {
      "sha1": "e003d151668a0eff64c1191972707655e341f8f5",
      "size": 57017689,
      "url": "https://piston-data.mojang.com/v1/objects/e003d151668a0eff64c1191972707655e341f8f5/server.jar"
    },
    "server_mappings": {
      "sha1": "ad7bb6cf9bdb85fd561981e2c4634a9d3292592d",
      "size": 7824495,
      "url": "https://piston-data.mojang.com/v1/objects/ad7bb6cf9bdb85fd561981e2c4634a9d3292592d/server.txt"
    }
  }
}

main函数代码 #

int main() { 
    simdjson::ondemand::parser parser; 
    simdjson::padded_string json = simdjson::padded_string::load("E:/OneDrive/Desktop/simdjson_project/assets/client_download.json");
    simdjson::ondemand::document doc = parser.iterate(json);

    simdjson::ondemand::object obj = doc["downloads"].get_object().value();

    Client::ClientDownloads downloads = Client::ClientDownloads::parse(obj);

    std::cout << downloads.client.sha1 << std::endl;
    std::cout << downloads.server.sha1 << std::endl;

    return 0; 
}

问题原因与解决方法 #

核心问题：ondemand模式的生命周期限制 #

simdjson的ondemand模式是流式解析，所有从document或object获取的临时引用（如get_string().value()返回的字符串视图）都依赖于原始padded_string和parser的生命周期。你的代码中，parse函数接收的object引用在函数执行后可能失效，导致后续访问字符串时触发结构错误。此外，直接调用.value()会在获取失败时抛出异常，未做前置检查。

修正步骤 #

1. 改用dom模式解析（最稳妥）

dom模式会将整个JSON加载到内存中，不存在生命周期依赖问题，适配代码如下：

main函数修改：

int main() { 
    simdjson::dom::parser parser; 
    simdjson::dom::element doc = parser.load("E:/OneDrive/Desktop/simdjson_project/assets/client_download.json");

    simdjson::dom::object obj = doc["downloads"];

    Client::ClientDownloads downloads = Client::ClientDownloads::parse(obj);

    std::cout << downloads.client.sha1 << std::endl;
    std::cout << downloads.server.sha1 << std::endl;

    return 0; 
}

解析函数修改：

Client::Download Client::Download::parse(simdjson::dom::object &obj) {
    std::string sha1, url;
    int64_t size;
    obj["sha1"].get(sha1);
    obj["size"].get(size);
    obj["url"].get(url);

    return {
        .sha1 = sha1,
        .size = size,
        .url = url
    };
}

Client::ClientDownloads Client::ClientDownloads::parse(simdjson::dom::object &obj) {
    Client::ClientDownloads download;

    simdjson::dom::object client_obj = obj["client"]; 
    simdjson::dom::object client_mappings_obj = obj["client_mappings"];
    simdjson::dom::object server_obj = obj["server"];
    simdjson::dom::object server_mappings_obj = obj["server_mappings"];
    
    download.client = Client::Download::parse(client_obj);
    download.client_mappings = Client::Download::parse(client_mappings_obj);
    download.server = Client::Download::parse(server_obj);
    download.server_mappings = Client::Download::parse(server_mappings_obj);

    return download;
}

2. 优化ondemand模式的使用（如需保留流式解析）

• 先检查键是否存在再取值，避免无意义的异常：

  // 在Download::parse中添加检查
  if (!obj["sha1"].is_string() || !obj["size"].is_int64() || !obj["url"].is_string()) {
      throw std::invalid_argument("Invalid Download object structure");
  }
  

• 确保所有ondemand对象的生命周期覆盖到数据使用完毕，不要在parse函数中过早释放引用。

3. 检查文件格式

将JSON文件保存为UTF-8无BOM格式，使用LF换行符，避免Windows下的编码或换行符导致解析异常。

内容的提问来源于stack exchange，提问作者Somebody That Likes Math