MATLAB SQLite接口是否支持并发?并行写入失败无报错如何解决?
SQLite数据库本身依版本和设置支持不同程度的并发,因此MATLAB Database Toolbox的SQLite接口理论上也应具备一定并发能力,且数据库访问失败时至少应给出错误提示。但运行以下测试代码后,出现部分写入操作未执行且无错误报告的异常情况:
测试代码
conn=sqlite("sqlite_concurr_test.db","create"); conn.close(); ppool=parpool(4); ff=parallel.Future.empty(); disp("Write 500 numbers") for ii=1:500 ff(ii)=parfeval(ppool,@writeOne,0,ii); end for ii=1:500 ff(ii).wait() end delete(ppool); conn=sqlite("sqlite_concurr_test.db"); readback=conn.sqlread("test"); disp("Readback "+num2str(size(readback,1))+" numbers"); function writeOne(ii) conn=sqlite("sqlite_concurr_test.db"); conn.sqlwrite("test",array2table(ii)); conn.close(); end
运行结果
Starting parallel pool (parpool) using the 'Processes' profile ...
Connected to the parallel pool (number of workers: 4).
Write 500 numbers
Parallel pool using the 'Processes' profile is shutting down.
Readback 74 numbers
需要解决的核心问题:如何调整代码确保并行写入有效,或至少能捕获到所有访问错误?
1. 捕获并返回错误信息
当前parfeval设置为返回0个输出参数,导致worker进程的错误无法传递回主进程。修改代码添加错误捕获逻辑,让每个任务的错误状态能被主进程感知:
conn=sqlite("sqlite_concurr_test.db","create"); conn.close(); ppool=parpool(4); ff=parallel.Future.empty(); disp("Write 500 numbers") for ii=1:500 ff(ii)=parfeval(ppool,@writeOne,1,ii); % 修改为返回1个错误信息参数 end % 遍历任务结果,统计错误 errorCount = 0; for ii=1:500 [errMsg] = ff(ii).fetchOutputs(); if ~isempty(errMsg) errorCount = errorCount + 1; fprintf("任务 %d 执行失败: %s\n", ii, errMsg); end end delete(ppool); conn=sqlite("sqlite_concurr_test.db"); readback=conn.sqlread("test"); disp("Readback "+num2str(size(readback,1))+" numbers"); disp("总失败任务数: "+num2str(errorCount)); function errMsg = writeOne(ii) errMsg = ""; try conn=sqlite("sqlite_concurr_test.db"); conn.sqlwrite("test",array2table(ii)); conn.close(); catch ME errMsg = ME.message; % 确保异常时连接被关闭 if isvalid(conn) conn.close(); end end end
2. 开启SQLite WAL并发模式
SQLite默认的日志模式并发能力有限,开启**WAL(Write-Ahead Logging)**模式可大幅提升并发读写支持,允许多个读操作与单个写操作同时进行。在初始化数据库时执行以下配置:
conn=sqlite("sqlite_concurr_test.db","create"); conn.execute("PRAGMA journal_mode=WAL;"); % 启用WAL模式 conn.close();
3. 添加重试机制处理临时锁冲突
并发写入时可能遇到数据库临时锁,这类错误可以通过重试解决。修改writeOne函数添加重试逻辑:
function errMsg = writeOne(ii) errMsg = ""; maxRetries = 3; retryCount = 0; success = false; while ~success && retryCount < maxRetries try conn=sqlite("sqlite_concurr_test.db"); conn.sqlwrite("test",array2table(ii)); conn.close(); success = true; catch ME retryCount = retryCount + 1; if retryCount >= maxRetries errMsg = ME.message; end if isvalid(conn) conn.close(); end pause(0.1); % 短暂等待后重试 end end end
4. 确保连接正确释放
在所有分支(包括异常分支)中执行conn.close(),避免worker进程持有的数据库连接未释放,导致后续操作锁死。
内容的提问来源于stack exchange,提问作者X Zhang

