Python3.6+PyAutoGUI自动化脚本无法捕获弹窗列表文本求助

Hey there, let's figure out how to fix this issue you're facing. PyAutoGUI is fantastic for simulating mouse clicks and keyboard inputs, but it operates at the pixel level—it doesn't have built-in capabilities to interact with the underlying GUI controls or extract their text. That's why you can't grab the script names from that pop-up list. Here are a few actionable approaches to solve this:

1. Use Pywinauto (Control-Level Automation) #

This is the most reliable method if your target app uses standard Windows GUI controls. Pywinauto lets you directly interact with application windows, controls, and extract their text without relying on screen pixels.

First, install the package:

pip install pywinauto

Then, use this sample code to connect to your app, locate the pop-up, and extract list items:

from pywinauto import Application

# Connect to your application—use either the window title or process name
app = Application(backend="uia").connect(title="Your Main App Window Title")

# Locate the pop-up window (replace with your pop-up's actual title)
popup_window = app.window(title="Select Script File")

# Find the list control (use Windows' built-in Inspect.exe to identify the control type/name)
script_list = popup_window.ListBox

# Extract all script names from the list
available_scripts = [item.text() for item in script_list.items()]
print("Available scripts:", available_scripts)

# Select your target script and click OK
script_list.select("Target Script Name")
popup_window.OK.click()

Pro tip: Use Windows' Inspect.exe (search for it in the Start Menu) to inspect the pop-up's controls. It will show you the exact control type, name, and properties needed to locate elements with pywinauto.

2. Fallback: OCR with PyTesseract #

If your app uses custom, non-standard controls that pywinauto can't detect, you can use optical character recognition (OCR) to extract text from the screen region containing the list.

First, install the required packages:

pip install pyautogui pytesseract pillow

You'll also need to install the Tesseract OCR engine (add it to your system PATH after downloading).

Here's how to implement it:

import pyautogui
import pytesseract
from PIL import Image

# Define the screen region of the pop-up list (use pyautogui.displayMousePosition() to get coordinates)
list_region = (100, 200, 400, 300)  # Replace with your actual x, y, width, height

# Capture the region and preprocess for better OCR accuracy
screenshot = pyautogui.screenshot(region=list_region)
grayscale_screenshot = screenshot.convert("L")  # Convert to grayscale

# Extract text from the screenshot
raw_text = pytesseract.image_to_string(grayscale_screenshot)
# Clean up the text to get individual script names
script_names = [line.strip() for line in raw_text.split("\n") if line.strip()]
print("Detected scripts:", script_names)

Note: OCR accuracy depends on the clarity of the text and screen resolution. You might need to adjust the region, add thresholding to the image, or tweak Tesseract's parameters for better results.

3. Other Options (App-Specific) #

• If your application is built with Qt, you can use QAccessible (part of PyQt) to access UI elements programmatically.
• If the pop-up is embedded in a web view, consider using Selenium to interact with it (though this is less likely for a desktop app).

Pick the method that best fits your app's GUI framework—pywinauto is the most robust choice for standard Windows apps, while OCR works for custom or hard-to-access interfaces.

内容的提问来源于stack exchange，提问作者codename_panda