求助：基于AI的Selenium Canvas验证码绕过方案

问题背景 #

我正在编写自动创建Proton Mail邮箱的爬虫脚本，完成邮箱名、密码输入后卡在了Canvas拼图验证码环节。原本以为拼图块和背景是独立HTML元素，实际二者都在同一个Canvas节点（//*[@id="root"]/div/div/div/div/div/div[1]/canvas）中。我尝试用OpenCV识别坐标但拖拽逻辑无效，且缺乏AI开发经验，需要指导如何实现该验证码的绕过。

当前代码的核心问题 #

1. CV识别逻辑不稳定：依赖固定轮廓面积和尺寸筛选，易受验证码样式变化影响，误判率高
2. 拖拽行为不真实：直接触发DragEvent未模拟人类鼠标轨迹，易被反爬机制检测
3. iframe处理冗余：重复切换iframe且未明确上下文，可能导致元素定位失败
4. 错误处理不完善：识别失败后的重试逻辑存在资源泄漏风险

解决方案步骤 #

一、非AI视觉识别优化（无需AI开发经验） #

1. 优化CV识别逻辑

改用颜色分割+轮廓排序的组合方案，针对Proton验证码的明暗差异特征提取目标：

• 通过HSV颜色阈值过滤背景，突出拼图和缺口区域
• 按轮廓面积排序，取前两个最大的轮廓作为拼图和缺口
• 根据位置判断（拼图通常靠左）区分二者

2. 模拟真实拖拽行为

使用Selenium的ActionChains生成带随机扰动的鼠标轨迹：

• 加入悬停停顿、分段拖拽、坐标随机偏移，模拟人类操作习惯
• 处理网页缩放比例，确保坐标转换准确

3. 规范iframe操作

明确嵌套iframe的切换层级，完成操作后无需切回主文档，避免上下文混乱

二、入门级AI辅助方案 #

如果需要引入AI，无需从零开发，可采用轻量开源模型：

• 使用YOLOv8 Tiny模型，标注20-30张验证码样本（拼图、缺口），训练一个轻量检测器直接输出坐标
• 调用Hugging Face预训练图像检测API，无需本地训练即可实现目标识别

修改后的代码示例 #

import os
import time
import random
import string
import cv2
import numpy as np
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

def get_puzzle_and_hole(image_path):
    """优化的CV识别：颜色分割+轮廓排序"""
    img = cv2.imread(image_path)
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    
    # 颜色阈值（可根据验证码实际颜色调整）
    lower = np.array([0, 0, 0])
    upper = np.array([180, 255, 100])
    mask = cv2.inRange(hsv, lower, upper)
    
    # 提取并排序轮廓
    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = sorted(contours, key=cv2.contourArea, reverse=True)[:2]
    
    if len(contours) != 2:
        return None, None
    
    # 根据位置区分拼图和缺口（拼图通常在左侧）
    cnt1, cnt2 = contours
    x1, y1, w1, h1 = cv2.boundingRect(cnt1)
    x2, y2, w2, h2 = cv2.boundingRect(cnt2)
    
    piece_coords = (x1, y1, w1, h1) if x1 < x2 else (x2, y2, w2, h2)
    hole_coords = (x2, y2, w2, h2) if x1 < x2 else (x1, y1, w1, h1)
    
    return piece_coords, hole_coords

def simulate_real_drag(driver, canvas, start_x, start_y, end_x, end_y):
    """模拟人类鼠标拖拽轨迹"""
    actions = ActionChains(driver)
    # 悬停到起点
    actions.move_to_element_with_offset(canvas, start_x, start_y).pause(random.uniform(0.2, 0.5))
    # 按下左键
    actions.click_and_hold()
    
    # 生成带扰动的拖拽轨迹
    steps = random.randint(10, 20)
    delta_x = end_x - start_x
    delta_y = end_y - start_y
    
    for i in range(steps):
        current_x = start_x + delta_x * (i/steps) + random.randint(-5, 5)
        current_y = start_y + delta_y * (i/steps) + random.randint(-3, 3)
        actions.move_to_element_with_offset(canvas, current_x, current_y).pause(random.uniform(0.05, 0.1))
    
    # 到达终点后释放鼠标
    actions.move_to_element_with_offset(canvas, end_x, end_y).pause(random.uniform(0.2, 0.4))
    actions.release()
    actions.perform()

def create_proton_account():
    os.environ['WDM_LOG_LEVEL'] = '0'
    option = webdriver.ChromeOptions()
    option.add_argument('--log-level=3')
    option.add_argument("--disable-blink-features=AutomationControlled")
    option.add_experimental_option("excludeSwitches", ["enable-automation"])
    driver = webdriver.Chrome(options=option)
    wait = WebDriverWait(driver, 60)
    
    try:
        driver.get("https://account.proton.me/mail/signup?plan=free&ref=mail_plus_intro-mailpricing-2")
        
        # 生成随机账号密码
        mail_length = random.randint(10, 20)
        mdp_length = random.randint(13, 15)
        alphabet = string.ascii_letters + string.digits
        alphabet1 = string.ascii_letters + string.digits + string.punctuation
        mail_base = ''.join(random.choice(alphabet) for _ in range(mail_length))
        mdp = ''.join(random.choice(alphabet1) for _ in range(mdp_length))
        print(f"生成账号: {mail_base}@proton.me | 密码: {mdp}")
        
        # 填写注册信息
        iframe = wait.until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div[4]/div[1]/main/div[1]/div[2]/form/iframe")))
        driver.switch_to.frame(iframe)
        wait.until(EC.element_to_be_clickable((By.ID, "email"))).send_keys(mail_base)
        driver.switch_to.default_content()
        
        wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="password"]'))).send_keys(mdp)
        wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="repeat-password"]'))).send_keys(mdp)
        wait.until(EC.element_to_be_clickable((By.XPATH, '//button[@type="submit"]'))).click()
        
        # 切换到验证码嵌套iframe
        wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[@id="key_0"]/iframe')))
        wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '/html/body/iframe')))
        
        # 获取Canvas并截图
        canvas = wait.until(EC.presence_of_element_located((By.XPATH, '//*[@id="root"]/div/div/div/div/div/div[1]/canvas')))
        screenshot_path = "captcha_screenshot.png"
        with open(screenshot_path, 'wb') as f:
            f.write(canvas.screenshot_as_png)
        
        # 识别拼图和缺口
        piece_coords, hole_coords = get_puzzle_and_hole(screenshot_path)
        if not piece_coords or not hole_coords:
            print("识别失败，重新尝试")
            return
        
        # 处理网页缩放比例
        canvas_rect = driver.execute_script("""
            var rect = arguments[0].getBoundingClientRect();
            return {width: rect.width};
        """, canvas)
        img_width = cv2.imread(screenshot_path).shape[1]
        scale = canvas_rect["width"] / img_width
        
        # 转换为Canvas相对坐标
        piece_x, piece_y, piece_w, piece_h = piece_coords
        hole_x, hole_y, hole_w, hole_h = hole_coords
        
        start_x = (piece_x + piece_w//2) * scale
        start_y = (piece_y + piece_h//2) * scale
        end_x = (hole_x + hole_w//2) * scale
        end_y = (hole_y + hole_h//2) * scale
        
        # 执行拖拽
        simulate_real_drag(driver, canvas, start_x, start_y, end_x, end_y)
        print("拖拽完成，等待验证结果")
        time.sleep(3)
        
    except Exception as e:
        print(f"执行出错: {str(e)}")
    finally:
        # 清理临时文件
        if os.path.exists(screenshot_path):
            os.remove(screenshot_path)
        # driver.quit()  # 如需自动关闭浏览器，取消注释

if __name__ == "__main__":
    create_proton_account()

注意事项 #

• 颜色阈值需根据验证码实际显示效果调整，可使用OpenCV工具提取HSV范围
• 拖拽轨迹的随机参数可根据反爬强度调整，越贴近人类行为通过率越高
• 频繁自动化注册可能触发账号封禁，建议控制请求频率并添加代理IP轮换
• Proton的验证码特征可能随时更新，需定期调整识别逻辑

内容的提问来源于stack exchange，提问作者Look_thisSTREAMING