Rust Polars WebAssembly解析CSV时ptr::read对齐错误求助

错误信息 #

Grid.js:498 panicked at 'unsafe precondition(s) violated: ptr::read requires that the pointer argument is aligned and non-null', C:\Users\61414\.rustup\toolchains\nightly-2022-11-20-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\panicking.rs:89:58

相关代码 #

Rust代码 #

pub fn load_csv(&mut self, buff: &[u8]) -> String {
    let cursor = Cursor::new(buff);
    let lf = CsvReader::new(cursor).with_ignore_parser_errors(true).finish().unwrap().lazy();
    return lf.describe_plan();
}

TypeScript代码（Svelte前端） #

function load_csv_file(event){
    const file = event.target.files[0]
    let reader = new FileReader();
    reader.readAsArrayBuffer(file);
    reader.onload = data1 => {
        console.log(data1.target.result)
        console.log(data.load_csv(new Uint8Array(data1.target.result)))
    };
}

WASM调用栈 #

Error
    at http://localhost:5173/rustFunctions/grid/pkg/Grid.js:504:21
    at logError (http://localhost:5173/rustFunctions/grid/pkg/Grid.js:134:18)
    at imports.wbg.__wbg_new_abda76e883ba8a5f (http://localhost:5173/rustFunctions/grid/pkg/Grid.js:503:66)
    at console_error_panic_hook::Error::new::h5d7996250e9efb8e (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[154789]:0x30fc8e8)
    at console_error_panic_hook::hook_impl::h667dd0ae102fb048 (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[29585]:0x1c5c717)
    at console_error_panic_hook::hook::hc3def586df00a6f2 (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[169823]:0x31c84b3)
    at core::ops::function::Fn::call::h397369bf956b8712 (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[149254]:0x30a5529)
    at <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hfcda8f4a283bd42a (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[125833]:0x2ee5fac)
    at std::panicking::rust_panic_with_hook::h7e6939e50e26b51d (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[914]:0x4641c8)
    at std::panicking::begin_panic_handler::{{closure}}::hfff77ccb8fcf1114 (http://localhost:5173/rustFunctions/grid/pkg/grid_bg.wasm:wasm-function[34760]:0x1e31219)

已排查情况 #

• 确认数组缓冲区非空，传递的[u8]数据与本地非WASM环境下可正常解析的CSV数据完全一致
• 相同Rust代码在本地非WASM环境中可正常加载并解析文件

解决方案建议 #

1. 确保内存对齐：复制数据到WASM堆 #

WebAssembly对内存对齐有严格要求，浏览器内存中的Uint8Array可能不满足Polars内部操作的对齐标准。修改前端代码，将数据复制到WASM管理的内存中：

function load_csv_file(event){
    const file = event.target.files[0]
    let reader = new FileReader();
    reader.readAsArrayBuffer(file);
    reader.onload = data1 => {
        const uint8 = new Uint8Array(data1.target.result);
        // 使用wasm-bindgen提供的方法分配WASM内存并复制数据
        const ptr = data.__wbindgen_malloc(uint8.length);
        const heap = new Uint8Array(data.memory.buffer);
        heap.set(uint8, ptr);
        try {
            // 传递指针和长度给修改后的Rust函数
            console.log(data.load_csv(ptr, uint8.length));
        } finally {
            // 释放WASM内存
            data.__wbindgen_free(ptr);
        }
    };
}

对应的Rust函数修改为：

#[wasm_bindgen]
pub fn load_csv(ptr: *const u8, len: usize) -> String {
    // 从指针和长度创建安全切片
    let buff = unsafe { std::slice::from_raw_parts(ptr, len) };
    let lf = CsvReader::new(buff)
        .with_ignore_parser_errors(true)
        .finish()
        .unwrap()
        .lazy();
    lf.describe_plan()
}

2. 调整Polars版本与编译配置 #

使用的2022-11-20 nightly Rust版本可能存在WASM兼容性问题，尝试以下操作：

• 升级Polars到最新稳定版，修复旧版本中的对齐相关bug
• 改用稳定版Rust编译WASM模块，避免nightly版本的不稳定特性
• 在Cargo.toml中添加WASM优化配置：

[profile.release]
opt-level = "s"
lto = true
codegen-units = 1
panic = "abort"

3. 简化数据读取流程 #

&[u8]本身已实现Read trait，无需额外包装成Cursor，尝试直接传递给CsvReader：

pub fn load_csv(&mut self, buff: &[u8]) -> String {
    let lf = CsvReader::new(buff)
        .with_ignore_parser_errors(true)
        .finish()
        .unwrap()
        .lazy();
    lf.describe_plan()
}

内容的提问来源于stack exchange，提问作者lallma