Python3.11+Windows11环境下依赖包安装失败及兼容问题求助

环境信息 #

• PyCharm 2022.2.3（专业版）
• Python 3.11.0（疑问：该Python版本与PyTorch是否存在兼容性问题？）
• Windows 11 x64系统

系统版本及Python版本验证：

Microsoft Windows [Version 10.0.22621.674]
(c) Microsoft Corporation. All rights reserved.

C:\Users\donhu>python
Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

requirements.txt内容 #

tensorflow~=2.10.0
numpy~=1.23.3
# torch==1.12.1
PyYAML~=6.0
pandas~=1.5.0
transformers~=4.22.1
TorchCRF~=1.1.0
tqdm~=4.64.1
seqeval~=1.2.2
viet-text-tools==0.1.1

运行报错 #

Package requirements 'tensorflow~=2.10.0', 'transformers~=4.22.1', 'TorchCRF~=1.1.0' are not satisfied

训练代码train.py #

import argparse
import yaml
import pandas as pd
import torch
from TorchCRF import CRF
import transformers
from data import Dataset
from engines import train_fn
import warnings

warnings.filterwarnings("ignore")
parser = argparse.ArgumentParser()
parser.add_argument("--data_file", type=str)
parser.add_argument("--hyps_file", type=str)
args = parser.parse_args()
data_file = yaml.load(open(args.data_file), Loader=yaml.FullLoader)
hyps_file = yaml.load(open(args.hyps_file), Loader=yaml.FullLoader)
train_loader = torch.utils.data.DataLoader(
    Dataset(
        df=pd.read_csv(data_file["train_df_path"]),
        tag_names=data_file["tag_names"],
        tokenizer=transformers.AutoTokenizer.from_pretrained(hyps_file["encoder"], use_fast=False),
    ),
    num_workers=hyps_file["num_workers"],
    batch_size=hyps_file["batch_size"],
    shuffle=True,
)
val_loader = torch.utils.data.DataLoader(
    Dataset(
        df=pd.read_csv(data_file["val_df_path"]),
        tag_names=data_file["tag_names"],
        tokenizer=transformers.AutoTokenizer.from_pretrained(hyps_file["encoder"], use_fast=False),
    ),
    num_workers=hyps_file["num_workers"],
    batch_size=hyps_file["batch_size"] * 2,
)
loaders = {
    "train": train_loader,
    "val": val_loader,
}
model = transformers.RobertaForTokenClassification.from_pretrained(hyps_file["encoder"],
                                                                   num_labels=data_file["num_tags"])
if hyps_file["use_crf"]:
    criterion = CRF(num_tags=data_file["num_tags"], batch_first=True)
else:
    criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=float(hyps_file["lr"]))
train_fn(
    loaders, model, torch.device(hyps_file["device"]), hyps_file["device_ids"],
    criterion,
    optimizer,
    epochs=hyps_file["epochs"],
    ckp_path="../ckps/{}.pt".format(hyps_file["encoder"].split("/")[-1]),
)

依赖安装错误信息 #

ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch
WARNING: You are using pip version 21.3.1; however, version 22.3 is available.
You should consider upgrading via the 

升级pip后问题仍未解决。

疑问 #

请问该如何解决这些问题？Python 3.11.0与PyTorch是否存在兼容性问题？

1. 核心问题说明 #

Python 3.11在2022年10月发布，PyTorch 1.12.1（你注释的版本）不支持该Python版本，直到PyTorch 2.0才正式适配Python 3.11。同时TensorFlow 2.10.0、transformers 4.22.1也不支持Python 3.11，这是导致安装失败的根本原因。

2. 具体解决步骤 #

提供两种可行方案，按需选择：

方案A：降级Python到3.10.x（推荐，适配原有依赖版本）

1. 修改requirements.txt，取消torch的注释：

tensorflow~=2.10.0
numpy~=1.23.3
torch==1.12.1
PyYAML~=6.0
pandas~=1.5.0
transformers~=4.22.1
TorchCRF~=1.1.0
tqdm~=4.64.1
seqeval~=1.2.2
viet-text-tools==0.1.1

2. 安装适配的PyTorch版本：
   • GPU版本：

     pip3 install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
     

   • CPU版本：

     pip3 install torch==1.12.1+cpu torchvision==0.13.1+cpu torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cpu
     

3. 安装剩余依赖：

   pip install -r requirements.txt
   

方案B：保留Python 3.11，升级依赖到兼容版本

1. 修改requirements.txt为兼容版本：

tensorflow~=2.11.0
numpy~=1.24.0
torch>=2.0.0
PyYAML~=6.0
pandas~=2.0.0
transformers>=4.26.0
TorchCRF~=1.1.0
tqdm~=4.64.1
seqeval~=1.2.2
viet-text-tools==0.1.1

2. 安装PyTorch 2.0+：
   • GPU版本：

     pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
     

   • CPU版本：

     pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
     

3. 安装剩余依赖：

   pip install -r requirements.txt
   

步骤3：验证依赖安装

运行以下命令确认关键依赖已正确安装：

python -c "import torch; print(torch.__version__)"
python -c "import tensorflow; print(tensorflow.__version__)"
python -c "import transformers; print(transformers.__version__)"
python -c "import TorchCRF; print(TorchCRF.__version__)"

3. 额外注意事项 #

• 在PyCharm中通过File > Settings > Project: xxx > Python Interpreter切换到对应版本的Python解释器。
• 建议使用虚拟环境安装依赖，避免全局环境冲突。

内容的提问来源于stack exchange，提问作者Đỗ Như Vỹ