如何将istream反序列化为thrust::host_vector<DataStruct>？

Hey there! Since you're already comfortable with malloc + memcpy for POD types, adapting this to thrust::host_vector is straightforward—because host_vector uses contiguous memory (just like std::vector), which works perfectly with direct byte-level operations from a stream. Here's how to do it:

Step 1: Prepare the host_vector with sufficient space #

First, resize your deserializedVector to match the number of DataStruct elements you expect. This allocates a contiguous block of memory large enough to hold all your data:

deserializedVector.resize(numberOfDataStructs);

Note: For POD types, resize will value-initialize new elements (e.g., setting int members to 0), but we'll overwrite this memory immediately with the stream data, so this doesn't affect the final result.

Step 2: Access the underlying memory pointer #

Use thrust::host_vector::data() to get a raw pointer to the start of the contiguous memory block. This is equivalent to the pointer you'd get from malloc:

DataStruct* vec_memory = deserializedVector.data();

Step 3: Read directly from the stream into the vector's memory #

Use std::istream::read() to copy bytes directly from your stream into the vector's memory. Since DataStruct is POD, this byte-level copy is safe (just like memcpy):

// Cast to char* since read expects a byte pointer
bufferToDeserialize->read(reinterpret_cast<char*>(vec_memory), 
                          numberOfDataStructs * sizeof(DataStruct));

Step 4: Verify the read operation (optional but recommended) #

Always check if the stream read was successful to catch issues like truncated data:

if (!bufferToDeserialize->good()) {
    throw std::runtime_error("Failed to read all DataStruct elements from stream");
}

Full Example Code #

Here's the complete snippet assuming DataStruct is a valid POD type:

#include <thrust/host_vector.h>
#include <istream>
#include <stdexcept>

// Example POD struct
struct DataStruct {
    int id;
    float value;
    double timestamp;
};

int main() {
    int numberOfDataStructs = 100;
    std::istream* bufferToDeserialize = /* Your input stream pointer */;
    thrust::host_vector<DataStruct> deserializedVector;

    // Perform deserialization
    deserializedVector.resize(numberOfDataStructs);
    DataStruct* vec_ptr = deserializedVector.data();
    
    bufferToDeserialize->read(reinterpret_cast<char*>(vec_ptr), 
                              numberOfDataStructs * sizeof(DataStruct));

    if (!bufferToDeserialize->good()) {
        throw std::runtime_error("Deserialization failed: incomplete data");
    }

    // Use deserializedVector as needed...
    return 0;
}

Why this works (and how it matches your malloc+memcpy approach) #

• Your original malloc call allocates contiguous memory; deserializedVector.resize() does the same, but with automatic memory management (no need to free later).
• memcpy copies bytes from a source to the allocated memory; istream::read() does the exact same thing, but pulls bytes directly from the stream instead of another memory buffer.
• Since thrust::host_vector guarantees contiguous storage for POD types, this approach is safe and efficient—no extra overhead compared to manual memory management.

Key Notes #

• Ensure DataStruct is truly POD: No user-defined constructors/destructors, no virtual functions, no non-POD members. If it's not POD, direct byte copying will cause undefined behavior.
• The stream must contain bytes that exactly match the memory layout of DataStruct (e.g., written with ostream::write() using the same struct definition, same compiler, and same endianness).

内容的提问来源于stack exchange，提问作者JMan Mousey