Docker-Compose部署MLRun后无法查看训练图,报AttributeError错误求助
解决MLRun部署后查看训练图表出现500错误的方案
问题详情
通过docker-compose部署MLRun,完成示例训练任务后,管理页面查看训练结果图表时返回500内部服务器错误,错误详情:
{ "data": { "detail": "AttributeError(\"'K8sHelper' object has no attribute 'v1api'\")" }, "status": 500, "statusText": "Internal Server Error", "headers": { "connection": "keep-alive", "content-length": "76", "content-type": "application/json", "date": "Fri, 07 Apr 2023 09:45:51 GMT", "server": "nginx/1.21.6" }, "config": { "url": "projects/quick-tutorial-root/files", "method": "get", "headers": { "Accept": "application/json, text/plain, */*" }, "params": { "path": "/auto-trainer-train/0/confusion-matrix.html" }, "baseURL": "/mlrun/api/v1", "transformRequest": [ null ], "transformResponse": [ null ], "timeout": 0, "xsrfCookieName": "XSRF-TOKEN", "xsrfHeaderName": "X-XSRF-TOKEN", "maxContentLength": -1, "maxBodyLength": -1, "transitional": { "silentJSONParsing": true, "forcedJSONParsing": true, "clarifyTimeoutError": false } }, "request": {} }
解决步骤
切换MLRun稳定版本镜像
替换docker-compose.yml中的MLRun镜像为最新稳定版(比如mlrun/mlrun:1.3.0,可根据官方发布选择),然后重新部署:docker-compose down docker-compose pull docker-compose up -d检查MLRun API服务日志
查看mlrun-api容器的初始化日志,排查K8sHelper相关错误:docker-compose logs mlrun-api若日志显示K8s客户端初始化失败,确认部署环境的Kubernetes集群(如kind/minikube)是否正常运行,或检查docker-compose.yml中K8s相关环境变量配置是否正确。
清理旧任务缓存并重跑训练
删除示例任务的缓存文件后重新提交训练:# 假设mlrun数据存储目录为当前目录下的mlrun-data rm -rf ./mlrun-data/projects/quick-tutorial-root/auto-trainer-train验证所有服务状态
确认docker-compose启动的所有服务均正常运行:docker-compose ps若有服务异常,查看对应服务日志定位问题。
内容的提问来源于stack exchange,提问作者sequoia00
相关产品推荐
相关产品推荐

