Eigen：可变大小数组累加的实现方案咨询与优化

Great question! Your current implementation works, but it’s far from optimal—let’s break down why and walk through better approaches tailored to Eigen’s strengths.

The Problem With Your Current Code #

Every time you call add(), you’re:

1. Creating a brand-new DataArray with the combined size of old + new data
2. Copying all existing data into this new array
3. Copying the new data into the array
4. Assigning this temporary array back to accumulated_data_ (which likely involves another full copy of all data)

This gets really expensive fast, especially if you’re calling add() frequently or working with large datasets—you’re paying for repeated memory allocations and redundant data copies that aren’t necessary.

Better Approaches Using Eigen’s Built-In Tools #

1. Preallocate Memory + In-Place Resizing (Best General Case)

Eigen’s dynamic arrays support reserve() (to set aside memory without changing the visible size) and conservativeResize() (to expand the array while preserving existing data). Combine these with bottomRows() to directly append new data without temporary arrays:

typedef Eigen::Array<double, Eigen::Dynamic, 3> DataArray;

class Accumulator {
private:
    DataArray accumulated_data_;

public:
    Accumulator() {
        // Optional: Start with a small reserved capacity to avoid immediate allocations
        accumulated_data_.reserve(100);
    }

    void add(const DataArray& new_data) {
        const int new_row_count = new_data.rows();
        const int total_rows_needed = accumulated_data_.rows() + new_row_count;

        // Expand reserved memory if needed (use 2x growth to minimize allocations)
        if (accumulated_data_.capacity() < total_rows_needed) {
            accumulated_data_.reserve(std::max(total_rows_needed, accumulated_data_.capacity() * 2));
        }

        // Resize the array in-place while keeping existing data
        accumulated_data_.conservativeResize(total_rows_needed, 3);

        // Copy new data directly into the bottom rows
        accumulated_data_.bottomRows(new_row_count) = new_data;
    }
};

Why This Works:

• reserve() ensures we only allocate memory when absolutely necessary, and using a 2x growth factor reduces the number of expensive memory allocation operations (a common optimization for dynamic containers).
• conservativeResize() preserves existing data—no need to copy old data into a temporary array.
• bottomRows() lets us write directly to the newly available space, so we only copy the new data once.

2. Preallocate to Known Total Size (If Possible)

If you can estimate the total number of rows you’ll need upfront, reserve that capacity immediately in the constructor. This eliminates all resize operations entirely:

Accumulator(int estimated_total_rows) {
    accumulated_data_.reserve(estimated_total_rows);
}

3. Move Semantics for Temporary New Data

If you’re passing temporary DataArray objects to add(), you can optimize further by overloading the method to take an rvalue reference and move the data instead of copying it:

void add(DataArray&& new_data) {
    const int new_row_count = new_data.rows();
    const int total_rows_needed = accumulated_data_.rows() + new_row_count;

    if (accumulated_data_.capacity() < total_rows_needed) {
        accumulated_data_.reserve(std::max(total_rows_needed, accumulated_data_.capacity() * 2));
    }

    accumulated_data_.conservativeResize(total_rows_needed, 3);
    // Move the temporary data instead of copying (avoids duplicating memory)
    accumulated_data_.bottomRows(new_row_count) = std::move(new_data);
}

Key Takeaways #

• Avoid creating temporary arrays for accumulation—use Eigen’s in-place resizing tools to cut down on redundant copies.
• Preallocate memory with reserve() to minimize expensive memory allocations, which are one of the biggest performance hits here.
• Use const references for input parameters (unless you need to modify the input) to make your code safer and more flexible.

内容的提问来源于stack exchange，提问作者Johannes