Rust多线程代码报错:借用值生命周期不足(10亿行挑战场景)
处理10亿行挑战(1brc)中的Rust生命周期错误
我正在用Rust完成10亿行挑战(1brc),已经受够了折腾编译器要求。我的代码会创建多个线程,后续会join所有线程,但编译器提示main函数作用域中声明的某个变量存在生命周期超出定义作用域的错误。我推测编译器认为线程可能存活超过主线程,导致借用值在仍被引用时失效,但实际上我已经join了所有线程,后续只有主线程运行并引用该数据,不会出现线程超出变量作用域的情况,特此寻求帮助。
原代码
use std::collections::HashMap; use std::fs; use std::fs::File; use std::io::{BufWriter, Write}; use std::sync::{Arc, Mutex}; use std::thread::{self, available_parallelism}; fn main() { let n_threads = available_parallelism().unwrap().get(); let datalines = fs::read_to_string("data/measurements.txt").unwrap(); let lines: Vec<&str> = datalines.lines().collect(); let n_lines = lines.len(); let chunk_size = (n_lines + 1) / n_threads; let mut handles = Vec::with_capacity(n_threads); let data_map: Arc<Mutex<HashMap<&str, [f32; 4]>>> = Arc::new(Mutex::new(HashMap::new())); for chunk in lines.chunks(chunk_size) { let chunk = chunk.to_vec(); let data_map = Arc::clone(&data_map); let handle = thread::spawn(move || { for line in chunk { let data = &mut data_map.lock().unwrap(); if let Some((state, temperature)) = line.split_once(';') { let temperature = temperature.parse().unwrap(); let entry = data.entry(state).or_insert([-100.0, 0.0, 100.0, 0.0]); entry[0] = entry[0].min(temperature); entry[1] += temperature; entry[2] = entry[2].max(temperature); entry[3] += 1.0; } } }); handles.push(handle); } for handle in handles { handle.join().unwrap(); } let outfile = File::create("output.txt").expect("Panicked while opening/creating outfile"); let mut writer = BufWriter::new(outfile); let data = data_map.lock().unwrap(); for (state, stats) in &*data { let outline = format!( "{}={}/{:.1}/{}, ", state, stats[0], stats[1] / stats[3], stats[2] ); writer .write_all(outline.as_bytes()) .expect("Panicked while writing outline"); } writer.flush().expect("Panicked while flushing the output"); }
更新后代码运行输出
time cargo run --release in bash at 20:10:47 编译中 billion-lines-challenge v0.1.0 (/home/abbas-taqvi/rust-projects/1brc-rust-lite) 已完成`release`编译模式 [优化编译] 目标文件生成耗时0.17秒 运行中 `target/release/billion-lines-challenge` 初始化与读取文件耗时23 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程已启动 线程已结束 线程计算耗时42 写入输出文件耗时0 实际耗时 1m6.617s 用户态耗时 0m56.713s 内核态耗时 0m10.093s
内容的提问来源于stack exchange,提问作者Syed M Abbas Haider Taqvi
相关产品推荐
相关产品推荐

