SikuliX调整分辨率与DPI缩放后图像检测失败求助

我正在使用SikuliX和Robot Framework开展GUI自动化测试，需要SikuliX在屏幕分辨率或Windows DPI缩放变化时仍能检测界面元素。

已执行操作 #

• 在1920x1080分辨率、125% DPI缩放的系统上采集参考图像；
• 编写基于OpenCV的Python脚本，动态检测当前屏幕分辨率与DPI缩放比例；
• 在运行SikuliX前按比例调整参考图像尺寸，使其匹配当前屏幕设置。

调试操作 #

• 用SSIM（结构相似性指数）对比原图与缩放后图像，结果为1.00（像素级完全一致）；
• 通过Robot Framework截图验证SikuliX的“视野”与屏幕完全一致；
• 添加“Reset Roi”确保SikuliX检测范围为全屏；
• 执行Robot Framework测试获取匹配分数，结果为0.0。

图像缩放Python代码片段 #

import os
import cv2
import pyautogui
import  ctypes 

IMAGE_FOLDER = "C:\\SikuliX.sikuli\\GPIO_TEST.sikuli"
RESIZED_IMAGE_FOLDER = "C:\\SikuliX.sikuli\\RESIZED_GPIO_TEST"

os.makedirs(RESIZED_IMAGE_FOLDER, exist_ok=True)

REFERENCE_WIDTH = 1920
REFERENCE_HEIGHT = 1080

CURRENT_WIDTH, CURRENT_HEIGHT = pyautogui.size()
scale_x = CURRENT_WIDTH / REFERENCE_WIDTH
scale_y = CURRENT_HEIGHT / REFERENCE_HEIGHT

def resize_image(image_path, scale_x, scale_y):
    img = cv2.imread(image_path)
    if img is None:
        print(f"Skipping: {image_path} (Not an image)")
        return False

    new_width = int(img.shape[1] * scale_x)
    new_height = int(img.shape[0] * scale_y)

    # Use INTER_NEAREST to avoid blurring
    resized_img = cv2.resize(img, (new_width, new_height), interpolation=cv2.INTER_NEAREST)

    resized_image_path = os.path.join(RESIZED_IMAGE_FOLDER, os.path.basename(image_path))

    # Save PNG without compression
    cv2.imwrite(resized_image_path, resized_img, [cv2.IMWRITE_PNG_COMPRESSION, 100])

    print(f"Image saved without compression: {resized_image_path} (Size: {os.path.getsize(resized_image_path)} bytes)")

    return True

def get_windows_scaling():
    user32 = ctypes.windll.user32
    dc = user32.GetDC(0)
    dpi = ctypes.windll.gdi32.GetDeviceCaps(dc, 88)  # Get DPI scaling
    return dpi / 96.0  # 96 DPI = 100%, 120 DPI = 125%, etc.

scaling_factor = get_windows_scaling()
print(f"Windows Scaling Detected: {scaling_factor * 100:.0f}%")

# Save scaling factor for Robot Framework
with open("scaling_factor.txt", "w") as f:
    f.write(str(scaling_factor))

for filename in os.listdir(IMAGE_FOLDER):
    if filename.endswith(".png") or filename.endswith(".jpg"):
        image_path = os.path.join(IMAGE_FOLDER, filename)
        resize_image(image_path, scale_x, scale_y)

print("Image resizing finished!")      

后续尝试的Robot Framework关键字 #

*** Keywords ***
Similarity reach
    [Arguments]    ${image_path}    ${max_similarity}=0.95    ${min_similarity}=0.6    ${step}=1
    ${similarity}    Set Variable    ${max_similarity}
    
    WHILE    ${similarity} >= ${min_similarity}
        ${score}    Get Match Score    ${image_path}
        Log    Match score for ${image_path} at similarity ${similarity}: ${score}
        
        IF    ${score} > 0.0
            Log     Image found with similarity: ${similarity}
            RETURN    ${score}
        END
        ${similarity}    Evaluate    ${similarity} - ${step}
    END
    Log    ${similarity}
    Log    Image not found even at lowest similarity threshold (${min_similarity})
    RETURN    0.0

日志显示所有阈值下匹配分数均为0.0，请问：SikuliX检测失败的原因是什么？有没有更好的方法处理分辨率与缩放变化？

检测失败的核心原因 #

1. 色彩空间不匹配：OpenCV默认以BGR格式读取图像，而SikuliX使用RGB色彩空间，颜色通道完全颠倒，直接导致匹配失败。
2. DPI感知缺失：SikuliX默认未开启Windows DPI感知，即使图像尺寸缩放正确，它获取的屏幕坐标与实际显示像素存在偏移，匹配逻辑无法对齐。
3. 关键字步长错误：Similarity reach中${step}设为1，会直接从0.95跳到负数，跳过所有合理的中间相似度阈值，实际未按预期尝试更低匹配度。

优化的分辨率与缩放处理方法 #

方法1：修复图像色彩空间 #

在Python缩放脚本中，将OpenCV读取的BGR图像转换为RGB后保存：

# 在resize_image函数的img = cv2.imread(image_path)后添加
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

方法2：开启SikuliX的DPI感知 #

• 若通过jar包启动SikuliX，添加启动参数：-Dsun.java2d.dpiaware=true
• 若使用Robot Framework的SikuliLibrary，初始化时设置：

Set SikuliX Setting    DPI_AWARE    True

开启后SikuliX会自动适配Windows DPI缩放，无需手动调整图像。

方法3：使用SikuliX原生缩放API #

直接通过Settings.Scale参数动态设置全局缩放比例，无需提前修改图像文件：

${scaling_factor}    Get File    scaling_factor.txt
Set SikuliX Setting    Scale    ${scaling_factor}

方法4：修正Robot Framework关键字步长 #

将${step}默认值改为0.05，确保逐步降低相似度阈值：

Similarity reach
    [Arguments]    ${image_path}    ${max_similarity}=0.95    ${min_similarity}=0.6    ${step}=0.05

内容的提问来源于stack exchange，提问作者Safaa Mhirsi