PyAutoGUI在Roblox中点击无响应/偏移，求替代自动化工具

一、PyAutoGUI的替代工具 #

1. pynput #

直接操作系统输入事件，能绕过部分游戏的输入拦截机制，对Roblox这类游戏兼容性较好，支持跨平台，API简洁直观。

示例代码（模拟鼠标点击）：

from pynput.mouse import Controller, Button
import time

mouse = Controller()
# 移动到目标坐标并执行左键点击
mouse.position = (457, 480)
time.sleep(0.1)
mouse.click(Button.left, 1)

2. PyDirectInput #

专为DirectX游戏设计，解决PyAutoGUI在3D游戏中点击失效的问题（Roblox基于DirectX渲染）。它会模拟原生DirectInput指令，而非系统级鼠标事件，游戏识别度更高。

示例代码：

import PyDirectInput
import pygetwindow as gw

# 获取Roblox窗口并激活
roblox_window = gw.getWindowsWithTitle("Roblox")[0]
roblox_window.activate()

# 将窗口相对坐标转换为全局屏幕坐标
global_x = roblox_window.left + 457
global_y = roblox_window.top + 480

# 移动鼠标并点击
PyDirectInput.moveTo(global_x, global_y, duration=0.5)
PyDirectInput.click()

3. ctypes（Windows平台专属） #

直接调用Windows API发送鼠标消息到指定窗口，实现精准的窗口内坐标点击，完全不受全局屏幕缩放影响，适合对精度要求高的场景。

示例代码（发送左键点击到Roblox窗口）：

import ctypes
import pygetwindow as gw

# Windows API常量定义
WM_LBUTTONDOWN = 0x0201
WM_LBUTTONUP = 0x0202
MK_LBUTTON = 0x0001

# 获取Roblox窗口句柄
roblox_window = gw.getWindowsWithTitle("Roblox")[0]
hwnd = roblox_window._hWnd

# 窗口内相对坐标
x = 457
y = 480

# 转换为Windows API所需参数格式
l_param = (y << 16) | (x & 0xFFFF)

# 发送鼠标按下和抬起消息
ctypes.windll.user32.SendMessageW(hwnd, WM_LBUTTONDOWN, MK_LBUTTON, l_param)
ctypes.windll.user32.SendMessageW(hwnd, WM_LBUTTONUP, 0, l_param)

二、现有代码的优化建议（无需换工具也能解决问题） #

1. 修复坐标偏移问题 #

高DPI屏幕下，系统缩放会导致ImageGrab和pyautogui的坐标不匹配，需进行缩放转换：

import pyautogui

# 获取系统缩放比例
scale_factor = pyautogui.screenshot().size[0] / pyautogui.size()[0]

# 将脚本内坐标转换为实际屏幕坐标
def convert_coords(x, y):
    return int(x * scale_factor), int(y * scale_factor)

# 使用示例
check_position = convert_coords(455, 502)
click_position = convert_coords(457, 480)

2. 确保权限一致 #

如果Roblox以管理员权限运行，Python脚本也需要以管理员权限启动，否则输入事件会被系统拦截。

3. 优化颜色检测准确性 #

游戏像素可能存在抗锯齿或颜色细微变化，可调整颜色匹配容差，或改用RGB直接对比：

def rgb_distance(rgb1, rgb2):
    return sum((a - b)**2 for a, b in zip(rgb1, rgb2))**0.5

# RGB距离小于50时判定为匹配（可根据实际情况调整）
if rgb_distance(pixel_color, hex_to_rgb(target_color)) < 50:
    # 执行点击逻辑
    pass

优化后的完整代码示例（结合PyDirectInput） #

import PyDirectInput
import pygetwindow as gw
from PIL import ImageGrab
import time
import colorsys

def hex_to_rgb(hex_color):
    hex_color = hex_color.lstrip("#")
    return tuple(int(hex_color[i:i+2], 16) for i in (0, 2, 4))

def rgb_to_hsl(rgb_color):
    return colorsys.rgb_to_hls(rgb_color[0]/255.0, rgb_color[1]/255.0, rgb_color[2]/255.0)

def color_distance(hsl1, hsl2):
    return sum((a-b)**2 for a,b in zip(hsl1, hsl2))**0.5

def get_window_global_coords(window, x, y):
    # 转换窗口相对坐标为全局屏幕坐标
    return window.left + x, window.top + y

def detect_color_and_click(target_color, check_and_click_positions, final_click_position):
    try:
        roblox_window = gw.getWindowsWithTitle("Roblox")[0]
        roblox_window.activate()
    except IndexError:
        print("未找到Roblox窗口，请确保游戏已启动")
        return
    
    target_hsl = rgb_to_hsl(hex_to_rgb(target_color))
    # 只截取Roblox窗口区域的截图，提升效率和准确性
    window_bbox = (roblox_window.left, roblox_window.top, roblox_window.right, roblox_window.bottom)
    screenshot = ImageGrab.grab(bbox=window_bbox)

    for i, ((check_x, check_y), (click_x, click_y)) in enumerate(check_and_click_positions):
        # 直接使用窗口内相对坐标获取像素
        pixel_color = screenshot.getpixel((check_x, check_y))
        pixel_hsl = rgb_to_hsl(pixel_color)
        distance = color_distance(target_hsl, pixel_hsl)

        if distance < 0.2:
            print(f"[{i+1}/{len(check_and_click_positions)}] 在位置({check_x},{check_y})检测到目标颜色，执行点击")
            global_click_x, global_click_y = get_window_global_coords(roblox_window, click_x, click_y)
            PyDirectInput.moveTo(global_click_x, global_click_y, duration=0.5)
            PyDirectInput.click()
            time.sleep(1)

    global_final_x, global_final_y = get_window_global_coords(roblox_window, *final_click_position)
    print(f"所有检测完成，移动到最终位置({final_click_position})并点击")
    PyDirectInput.moveTo(global_final_x, global_final_y, duration=0.5)
    PyDirectInput.click()
    time.sleep(1)

target_color = "#A73DEF"
check_and_click_positions = [((455, 502), (457, 480)), ((700, 502), (700, 476))]
final_click_position = (497, 137)

while True:
    detect_color_and_click(target_color, check_and_click_positions, final_click_position)
    time.sleep(2)

内容的提问来源于stack exchange，提问作者Niki