Selenium脚本在PythonAnywhere运行异常的问题求助

我开发了一个脚本，计划在每月1号（测试阶段设为当日）发送下月即将到来的跑步赛事与马拉松信息邮件。该脚本在PyCharm IDE中运行正常，但部署到PythonAnywhere后无法定位所需交互的页面元素（每次均输出'First ERR'），有时还会崩溃，页面加载存在异常。请问我哪里操作有误？

相关代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.wait import WebDriverWait
from smtplib import SMTP
from datetime import datetime
import requests
import os
import time
from bs4 import BeautifulSoup

EMAIL = 'bocaj.a31@gmail.com'
PASSWORD = os.environ.get('PASSWORD')
print(PASSWORD)

response = requests.get('https://www.runguides.com/')
response.raise_for_status()
# print(response.text)
soup = BeautifulSoup(response.text, 'html.parser')
link_extension = soup.select_one(selector='#footer > div.wrapper.footer-bottom > div > div.col-12.hide-on-tablet > div > ul > li:nth-child(66) > a').get('href')
print(link_extension)

chrome_options = webdriver.ChromeOptions()
#I tried to add these to pythonanywhere to fix a different problem
#---------------------------------------
# chrome_options.add_argument("--no-sandbox")
# chrome_options.add_argument("--headless")
# chrome_options.add_argument("--disable-gpu")
# chrome_options.add_argument('--disable-dev-shm-usage')
#------------------------------------------
chrome_options.add_experimental_option('detach',True)
main_driver = webdriver.Chrome(options=chrome_options)

if (datetime.now().day == 31):
    main_driver.get(f'https://www.runguides.com{link_extension}')
    main_driver.maximize_window()
    try:
        WebDriverWait(main_driver, 20).until(expected_conditions.element_to_be_clickable((By.XPATH,'/html/body/div[2]/div[3]/div/div[6]/div[4]/md-menu[1]/button')))
    except TimeoutException:
        print("First ERR")
    else:
        print('Success')
        main_driver.find_element(By.XPATH, value='//*[@id="main-listing-container"]/div[4]/md-menu[1]/button').click()

        try:
            WebDriverWait(main_driver, 120).until(expected_conditions.presence_of_element_located((By.CSS_SELECTOR,'md-menu-content md-menu-item .md-button .ng-binding')))
        except TimeoutException:
            print("ERROR")
        else:
            months_elements = main_driver.find_elements(By.CSS_SELECTOR,
                                                    value='md-menu-content md-menu-item .md-button .ng-binding')
            print(months_elements)
            names = [item.text for item in months_elements]
            print(names)
            try:
                month_names = months_elements[names.index('January'):]
            except ValueError:
                time.sleep(5)
                names = [item.text for item in months_elements]
                month_names = months_elements[names.index('January'):]

            print(month_names)
            month_names[datetime.now().month].click()
            try:
                WebDriverWait(main_driver, 120).until(expected_conditions.presence_of_element_located(
                    (By.CSS_SELECTOR, 'tbody tr')))
            except TimeoutException:
                    print('Error')
            else:
                print('Success')
                try:
                    all_race_dates = [race_date.text for race_date in main_driver.find_elements(By.CSS_SELECTOR, value='tbody .listing-container .listing-date .clickable') if 'Ontario' not in race_date.text]
                    print(all_race_dates)
                    all_race_names = [race_names.text for race_names in main_driver.find_elements(By.CSS_SELECTOR, value='tbody .listing-container .listing-title .clickable')]
                    print(all_race_names)
                    email_string = ''
                    for i in range(len(all_race_names)):
                        email_string +=f'{all_race_names[i]} on {all_race_dates[i]} \n'
                    if email_string != '':
                        with SMTP('smtp.gmail.com') as connection:
                            connection.starttls()
                            connection.login(user=EMAIL, password=PASSWORD)
                            connection.sendmail(from_addr=EMAIL, to_addrs=EMAIL, msg=f'Subject: Races next Month!!\n\n{email_string}')
                except:
                    print('Password Error')
    main_driver.quit()

#Obviously if it works, there should be a "Password Error" because that is an environment variable

#I've tried looking at documentation, adding the arguments I did to get the driver to not crash, and I tried adding print statemnts in various places to debug

排查与修复建议 #

• 启用无头模式及服务器必需配置：PythonAnywhere是无桌面环境的服务器，必须启用无头模式并添加必要参数，把你注释掉的以下代码取消注释，同时删掉detach=True（服务器环境下无用，还可能导致进程残留）：

  chrome_options.add_argument("--no-sandbox")
  chrome_options.add_argument("--headless=new")  # 用新版本无头模式更稳定
  chrome_options.add_argument("--disable-gpu")
  chrome_options.add_argument('--disable-dev-shm-usage')
  

• 替换不可靠的绝对定位：你使用的绝对路径XPATH（如/html/body/div[2]/div[3]/div/div[6]/div[4]/md-menu[1]/button）极易因为页面结构变化（比如服务器端渲染差异、弹窗/广告加载）失效，换成基于元素特征的相对定位，比如：

  # 替换原XPATH，改用按钮的aria-label属性定位
  WebDriverWait(main_driver, 20).until(expected_conditions.element_to_be_clickable((By.CSS_SELECTOR, 'md-menu button[aria-label*="Month"]')))
  

• 优化页面等待逻辑：服务器网络环境与本地不同，页面加载可能更慢。建议：
  • 设置页面加载策略为eager，减少等待完全加载的时间：chrome_options.page_load_strategy = 'eager'
  • 优先使用visibility_of_element_located而非element_to_be_clickable，后者条件更严格，容易触发超时
• 统一请求环境：不要混合使用requests和Selenium获取页面链接，两者的请求头、环境完全不同，可能导致服务器返回的页面结构不一致。直接用Selenium打开首页，再提取链接：

  main_driver.get('https://www.runguides.com/')
  link_extension = main_driver.find_element(By.CSS_SELECTOR, '#footer > div.wrapper.footer-bottom > div > div.col-12.hide-on-tablet > div > ul > li:nth-child(66) > a').get_attribute('href')
  

• 检查依赖与版本匹配：确认PythonAnywhere上已安装所有依赖包（selenium、beautifulsoup4、requests），并且ChromeDriver版本与服务器上的Chrome版本完全一致——版本不匹配会直接导致驱动崩溃
• 精准捕获异常：把宽泛的except:改成捕获具体异常（如SMTPAuthenticationError、NoSuchElementException、TimeoutException），方便定位真实问题，比如：

  except SMTPAuthenticationError:
      print('邮箱认证失败，请检查密码或授权码')
  except Exception as e:
      print(f'未知错误：{str(e)}')
  

内容的提问来源于stack exchange，提问作者j. astroff