AWS EC2实例中通过Conda YAML创建环境时进程挂起求助
问题:Conda通过YAML重建环境时挂起超时
在AWS EC2 Linux实例上执行以下操作时,通过导出的YAML文件重建Conda环境会无报错挂起,最终超时被终止:
conda create -n myenv python=3.10.4 conda activate myenv conda list # 输出见下文 conda env export > newenv.yml # YAML文件内容见下文,此步骤正常 conda deactivate conda env create -n newenv --f newenv.yml # 此处挂起/冻结 # 输出: # Collecting package metadata (repodata.json): - # Collecting package metadata (repodata.json): - Killed # 约5分钟后
已尝试的操作:
- 删除原环境后同名重建
- 确认
anaconda3/envs文件夹权限为775,创建环境无权限问题 - 更新、清理、重新初始化Conda
- 多次重启EC2实例
Conda list输出
(myenv) [ec2-user@ip-172-31-93-141 ~]$ conda list # packages in environment at /home/ec2-user/anaconda3/envs/myenv: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge bzip2 1.0.8 h7f98852_4 conda-forge ca-certificates 2022.9.24 ha878542_0 conda-forge ld_impl_linux-64 2.39 hcc3a1bd_1 conda-forge libffi 3.3 h58526e2_2 conda-forge libgcc-ng 12.2.0 h65d4601_19 conda-forge libgomp 12.2.0 h65d4601_19 conda-forge libsqlite 3.40.0 h753d276_0 conda-forge libstdcxx-ng 12.2.0 h46fd767_19 conda-forge libuuid 1.41.5 h5eee18b_0 libzlib 1.2.13 h166bdaf_4 conda-forge ncurses 6.3 h27087fc_1 conda-forge openssl 1.1.1s h0b41bf4_1 conda-forge pip 22.3.1 pyhd8ed1ab_0 conda-forge python 3.10.4 h12debd9_0 readline 8.1.2 h0f457ee_0 conda-forge setuptools 65.5.1 pyhd8ed1ab_0 conda-forge sqlite 3.40.0 h4ff8645_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge tzdata 2022g h191b570_0 conda-forge wheel 0.38.4 pyhd8ed1ab_0 conda-forge xz 5.2.6 h166bdaf_0 conda-forge zlib 1.2.13 h166bdaf_4 conda-forge
YAML文件内容
name: myenv channels: - conda-forge - defaults dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_gnu - bzip2=1.0.8=h7f98852_4 - ca-certificates=2022.9.24=ha878542_0 - ld_impl_linux-64=2.39=hcc3a1bd_1 - libffi=3.3=h58526e2_2 - libgcc-ng=12.2.0=h65d4601_19 - libgomp=12.2.0=h65d4601_19 - libsqlite=3.40.0=h753d276_0 - libstdcxx-ng=12.2.0=h46fd767_19 - libuuid=1.41.5=h5eee18b_0 - libzlib=1.2.13=h166bdaf_4 - ncurses=6.3=h27087fc_1 - openssl=1.1.1s=h0b41bf4_1 - pip=22.3.1=pyhd8ed1ab_0 - python=3.10.4=h12debd9_0 - readline=8.1.2=h0f457ee_0 - setuptools=65.5.1=pyhd8ed1ab_0 - sqlite=3.40.0=h4ff8645_0 - tk=8.6.12=h27826a3_0 - tzdata=2022g=h191b570_0 - wheel=0.38.4=pyhd8ed1ab_0 - xz=5.2.6=h166bdaf_0 - zlib=1.2.13=h166bdaf_4 prefix: /home/ec2-user/anaconda3/envs/myenv
解决思路
精简YAML文件:导出的YAML包含大量底层依赖,部分依赖可能存在源访问或版本冲突问题。可先保留核心依赖(如仅
python=3.10.4),让Conda自动解决其他依赖,再逐步添加所需包。修改后的YAML示例:name: newenv channels: - conda-forge - defaults dependencies: - python=3.10.4切换镜像源加速:EC2实例访问默认Conda源可能较慢,可在创建命令中直接指定国内镜像源:
conda env create -n newenv --file newenv.yml -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/ -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/也可修改
~/.condarc配置文件永久替换为国内源:channels: - defaults show_channel_urls: true default_channels: - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2 custom_channels: conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud检查实例内存资源:"Killed"提示大概率是实例内存不足导致进程被终止。执行
free -h查看内存使用情况,若内存不足,升级EC2实例规格(如t2.medium及以上)。跳过依赖检查(谨慎使用):使用
--no-deps参数跳过依赖检查直接安装指定包,但可能导致环境缺失必要依赖:conda env create -n newenv --file newenv.yml --no-deps清理Conda缓存:缓存的repodata可能损坏,执行以下命令清理后重试:
conda clean --all -y
内容的提问来源于stack exchange,提问作者KXO2022
相关产品推荐
相关产品推荐

