VS Code中Jupyter Notebook训练大数据集时内核崩溃求助
训练超大数据集时Sklearn内核崩溃问题
使用Sklearn训练超大数据集时出现内核崩溃问题:小数据集可正常运行,但数据集规模超过阈值后,内核直接崩溃。错误日志如下:
info 16:24:11.630: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -m pip list > ~/miniconda3/envs/auto-sklearn/bin/python -m pip list info 16:24:11.712: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)" > ~/miniconda3/envs/auto-sklearn/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)" info 16:24:11.910: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -m ipykernel_launcher --ip=127.0.0.1 --stdin=9011 --control=9009 --hb=9008 --Session.signature_scheme="hmac-sha256" --Session.key=b"39e77f25-eae0-4712-8a1a-628305c2ff03" --shell=9010 --transport="tcp" --iopub=9012 --f=/home/baraa/.local/share/jupyter/runtime/kernel-v2-261323AOvTtMclSsgz.json > ~/miniconda3/envs/auto-sklearn/bin/python -m ipykernel_launcher --ip=127.0.0.1 --stdin=9011 --control=9009 --hb=9008 --Session.signature_scheme="hmac-sha256" --Session.key=b"39e77f25-eae0-4712-8a1a-628305c2ff03" --shell=9010 --transport="tcp" --iopub=9012 --f=/home/baraa/.local/share/jupyter/runtime/kernel-v2-261323AOvTtMclSsgz.json info 16:24:11.910: Process Execution: cwd: ~/Documents/Python/Testing/Search cwd: ~/Documents/Python/Testing/Search info 16:24:12.237: ipykernel version & path 6.21.2, ~/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/ipykernel/__init__.py for /home/baraa/miniconda3/envs/auto-sklearn/bin/python info 16:24:13.131: Started Kernel auto-sklearn (Python 3.8.16) (pid: 263555) info 16:24:13.182: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/pythonFiles/printJupyterDataDir.py > ~/miniconda3/envs/auto-sklearn/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/pythonFiles/printJupyterDataDir.py error 16:25:00.281: Disposing session as kernel process died ExitCode: undefined, Reason: /home/baraa/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/traitlets/traitlets.py:2548: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5. warn( /home/baraa/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/traitlets/traitlets.py:2499: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '39e77f25-eae0-4712-8a1a-628305c2ff03' instead of 'b"39e77f25-eae0-4712-8a1a-628305c2ff03"'. warn( info 16:25:00.284: Dispose Kernel process 263555. error 16:25:00.284: Raw kernel process exited code: undefined error 16:25:00.301: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done at t.KernelShellFutureHandler.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:33213) at /home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:52265 at Map.forEach (<anonymous>) at y._clearKernelState (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:52250) at y.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:45732) at /home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:139244 at Z (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:1608939) at Kp.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:139221) at qp.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:146518) at process.processTicksAndRejections (node:internal/process/task_queues:96:5)] warn 16:25:00.301: Cell completed with errors { message: 'Canceled future for execute_request message before replies were done' } info 16:25:00.302: Cancel all remaining cells true || Error || undefined
已尝试以下方法但问题未解决:
- 重装ipykernel
- 降级traitlets和pyzmq至19.0.2
- 升级Python版本
- 重装Miniconda
- 更换conda环境
内容的提问来源于stack exchange,提问作者Baraa Zaid
相关产品推荐
相关产品推荐

