如何解决Docker容器内mpirun调用Python脚本失败的问题?
问题:Docker SDK执行mpirun命令找不到python可执行文件
我尝试用以下Python脚本通过Docker SDK在容器内执行runScript_v2.py:
import docker # Initialize the Docker client client = docker.from_env() # Define Docker run options docker_run_options = { 'name': 'container-name', 'user': 'dafoamuser', 'mounts': [ { 'type': 'bind', 'source': 'D:/Dafoam/oneraM6', 'target': '/home/dafoamuser/mount' } ], 'working_dir': '/home/dafoamuser/mount', 'detach': True, 'tty': True, 'stdin_open': True, 'command': 'bash' } # Run the Docker container with specified options container = client.containers.run( 'dafoamimage:latest', **docker_run_options ) # Execute the mpirun command inside the container mpirun_command = 'mpirun -np 8 python runScript_v2.py 2>&1 | tee logOpt.txt' exec_cmd = container.exec_run(mpirun_command, tty=True, privileged=True) print(f"Command: {mpirun_command}\nOutput:\n{exec_cmd.output.decode('utf-8')}") # Stop and remove the container (if needed) container.stop() container.remove()
运行后出现如下错误:
C:\Users\yesbo\.conda\envs\py39env\lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, Command: mpirun -np 8 python runScript_v2.py 2>&1 | tee logOpt.txt Output: -------------------------------------------------------------------------- mpirun was unable to find the specified executable file, and therefore did not launch the job. This error was first reported for process rank 0; it may have occurred for other processes as well. NOTE: A common cause for this error is misspelling a mpirun command line parameter option (remember that mpirun interprets the first unrecognized command line token as the executable). Node: c537205732b5 Executable: python -------------------------------------------------------------------------- 8 total processes failed to start
直接在容器终端运行该mpirun命令可正常执行,求解决办法。
解决方案
使用python的绝对路径
先在容器终端执行which python获取python的绝对路径(比如/usr/bin/python),然后修改mpirun命令:mpirun_command = 'mpirun -np 8 /usr/bin/python runScript_v2.py 2>&1 | tee logOpt.txt'原因:exec_run执行命令时的环境变量PATH可能和终端登录时不同,导致无法找到python可执行文件。
通过shell执行命令
exec_run默认直接执行命令,不经过shell,管道|和重定向2>&1无法生效,同时环境变量也不完整。修改命令为通过bash执行:mpirun_command = '/bin/bash -c "mpirun -np 8 python runScript_v2.py 2>&1 | tee logOpt.txt"' exec_cmd = container.exec_run(mpirun_command, tty=True, privileged=True)显式指定环境变量
在exec_run时传入完整的PATH环境变量,确保能找到python:exec_cmd = container.exec_run( mpirun_command, tty=True, privileged=True, environment={"PATH": "/usr/local/bin:/usr/bin:/bin:/home/dafoamuser/.local/bin"} )路径可根据容器内实际环境调整。
内容的提问来源于stack exchange,提问作者Mohsin Bashir
相关产品推荐
相关产品推荐

