已安装requests仍报ModuleNotFoundError的网页爬取问题排查

爬取代码 #

import requests
from bs4 import BeautifulSoup

try:
    source = requests.get('https://www.tripadvisor.com.sg/Hotels-g294265-Singapore-Hotels.html')
    source.raise_for_status()

    soup = BeautifulSoup(source.text, 'html.parser')
    print(soup)

    hotel = soup.find('div', class_="prw_rup prw_meta_hsx_responsive_listing ui_section listItem")

    print(hotel)

except Exception as e:
    print(e)

命令行运行日志 #

PS C:\Users\keeha\PycharmProjects\INF1002_Draft> py main1.py
Traceback (most recent call last):
  File "C:\Users\keeha\PycharmProjects\INF1002_Draft\main1.py", line 1, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
PS C:\Users\keeha\PycharmProjects\INF1002_Draft> cd C:\Users\keeha\AppData\Local\Programs\Python\Python310\Scripts          
PS C:\Users\keeha\AppData\Local\Programs\Python\Python310\Scripts> .\pip install requests                                                     
Requirement already satisfied: requests in c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages (2.28.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages (from requests) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages (from requests) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages (from requests) (2022.9.24)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages (from requests) (1.26.12)
WARNING: You are using pip version 21.3.1; however, version 22.2.2 is available.
You should consider upgrading via the 'C:\Users\keeha\AppData\Local\Programs\Python\Python310\Scripts\python.exe -m pip install --upgrade pip' command.
PS C:\Users\keeha\AppData\Local\Programs\Python\Python310\Scripts> cd c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages
PS C:\users\keeha\appdata\local\programs\python\python310\lib\site-packages> py C:\Users\keeha\PycharmProjects\INF1002_Draft\main1.py                   
Traceback (most recent call last):
  File "C:\Users\keeha\PycharmProjects\INF1002_Draft\main1.py", line 1, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
PS C:\users\keeha\appdata\local\programs\python\python310\lib\site-packages>

问题说明 #

已在Python环境和PyCharm中安装requests模块，且能在PyCharm设置中找到该模块，但运行代码时反复出现ModuleNotFoundError: No module named 'requests'。

排查原因及解决步骤 #

• Python版本环境不匹配：pip安装的是Python3.10的requests，但py命令可能指向其他Python版本。验证及解决：
  • 运行py --version查看当前py对应的Python版本
  • 运行python3.10 -m pip list确认requests在3.10环境中存在
  • 改用python3.10 main1.py运行代码，确保使用安装了requests的环境
• PyCharm解释器配置错误：全局安装了requests，但PyCharm未使用对应解释器。检查步骤：
  1. 进入File > Settings > Project: [你的项目名] > Python Interpreter
  2. 确认选中的解释器是Python3.10（安装requests的版本）
  3. 若不是，点击齿轮图标选择Add，找到Python3.10的安装路径添加
• 虚拟环境冲突：项目若使用虚拟环境，requests可能仅安装在全局环境。解决：
  • 激活项目虚拟环境，运行pip install requests
  • 或在PyCharm解释器设置中切换到项目虚拟环境解释器
• 路径识别问题：Python未识别到requests所在的site-packages路径。验证及解决：
  • 在代码开头添加以下代码打印路径：

    import sys
    print(sys.path)
    

  • 若输出无c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages，手动添加：

    import sys
    sys.path.append('c:\users\keeha\appdata\local\programs\python\python310\lib\site-packages')
    

内容的提问来源于stack exchange，提问作者KeeHanXiang