使用UIAutomation模拟Windows鼠标拖拽移动文件失败求助

我编写了一个模拟两元素间鼠标拖拽的do_drag方法，代码如下：

public void do_drag(IUIAutomationElement from_element, IUIAutomationElement to_element) {
    int from_x = (int)(from_element.CurrentBoundingRectangle.left + from_element.CurrentBoundingRectangle.right) / 2;
    int from_y = (int)(from_element.CurrentBoundingRectangle.top + from_element.CurrentBoundingRectangle.bottom) / 2;
    int to_x = (int)(to_element.CurrentBoundingRectangle.left + to_element.CurrentBoundingRectangle.right) / 2;
    int to_y = (int)(to_element.CurrentBoundingRectangle.top + to_element.CurrentBoundingRectangle.bottom) / 2;
    var screenBounds = Screen.PrimaryScreen.Bounds;
    SetCursorPos(from_x, from_y); 
    mouse_event(MOUSEEVENTTF_ABSOLUTE | MOUSEEVENTTF_LEFTDOWN, from_x, from_y, 0, 0);
    mouse_event(MOUSEEVENTTF_ABSOLUTE | MOUSEEVENTTF_MOVE, to_x * 65535 / screenBounds.Width, to_y * 65535 / screenBounds.Height, 0, 0);
    mouse_event(MOUSEEVENTTF_ABSOLUTE | MOUSEEVENTTF_LEFTUP, to_x, to_y, 0, 0);
    return;
}

这个方法在Excel中模拟单元格拖拽选中多个网格时正常工作，但模拟将文件拖拽到文件夹完成移动操作时，鼠标虽按预期移动，却未实际完成文件移动。

• API兼容性问题：mouse_event是较旧的Windows API，对于资源管理器这类现代UI，系统更依赖SendInput处理输入事件，后者能更准确模拟真实用户输入，避免被系统输入拦截机制忽略。
• 坐标不一致：代码中LEFTDOWN和LEFTUP用原始屏幕坐标，MOVE用绝对坐标转换（乘以65535/屏幕宽高），混合坐标模式会导致系统识别拖拽动作时出现偏差。
• 缺少必要延迟：真实拖拽过程存在时间间隔，连续调用mouse_event会让系统来不及处理拖拽中间状态（比如资源管理器需要时间识别拖拽对象、触发悬停逻辑）。
• 元素坐标准确性：需确认IUIAutomationElement的CurrentBoundingRectangle是否返回正确屏幕坐标，尤其是资源管理器中文件、文件夹的实际显示位置（比如是否受隐藏标题栏、滚动偏移影响）。

1. 替换为SendInput API #

SendInput是更可靠的输入模拟API，先定义必要结构体和导入：

[StructLayout(LayoutKind.Sequential)]
public struct INPUT
{
    public uint Type;
    public MOUSEINPUT MouseInput;
}

[StructLayout(LayoutKind.Sequential)]
public struct MOUSEINPUT
{
    public int X;
    public int Y;
    public uint MouseData;
    public uint Flags;
    public uint Time;
    public IntPtr ExtraInfo;
}

[DllImport("user32.dll", SetLastError = true)]
public static extern uint SendInput(uint nInputs, INPUT[] pInputs, int cbSize);

private const uint INPUT_MOUSE = 0;
private const uint MOUSEEVENTF_ABSOLUTE = 0x8000;
private const uint MOUSEEVENTF_LEFTDOWN = 0x0002;
private const uint MOUSEEVENTF_LEFTUP = 0x0004;
private const uint MOUSEEVENTF_MOVE = 0x0001;

2. 修改do_drag方法，统一坐标处理并添加延迟 #

public void do_drag(IUIAutomationElement from_element, IUIAutomationElement to_element)
{
    var fromRect = from_element.CurrentBoundingRectangle;
    int from_x = (int)(fromRect.Left + fromRect.Right) / 2;
    int from_y = (int)(fromRect.Top + fromRect.Bottom) / 2;
    
    var toRect = to_element.CurrentBoundingRectangle;
    int to_x = (int)(toRect.Left + toRect.Right) / 2;
    int to_y = (int)(toRect.Top + toRect.Bottom) / 2;
    
    var screenBounds = Screen.PrimaryScreen.Bounds;
    // 统一转换为绝对坐标（0-65535范围）
    int absFromX = (int)(from_x * 65535.0 / screenBounds.Width);
    int absFromY = (int)(from_y * 65535.0 / screenBounds.Height);
    int absToX = (int)(to_x * 65535.0 / screenBounds.Width);
    int absToY = (int)(to_y * 65535.0 / screenBounds.Height);

    // 移动到起始位置并等待系统识别
    MoveMouse(absFromX, absFromY);
    Thread.Sleep(100);

    // 按下左键并短暂停顿
    MouseEvent(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_ABSOLUTE, absFromX, absFromY);
    Thread.Sleep(50);

    // 分步骤移动鼠标，贴近真实拖拽动作
    MoveMouseGradually(absFromX, absFromY, absToX, absToY, 10);
    Thread.Sleep(100); // 悬停目标，触发资源管理器接收逻辑

    // 松开左键
    MouseEvent(MOUSEEVENTF_LEFTUP | MOUSEEVENTF_ABSOLUTE, absToX, absToY);
    Thread.Sleep(50);
}

private void MoveMouse(int x, int y)
{
    INPUT[] input = new INPUT[1];
    input[0].Type = INPUT_MOUSE;
    input[0].MouseInput.X = x;
    input[0].MouseInput.Y = y;
    input[0].MouseInput.Flags = MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
    SendInput(1, input, Marshal.SizeOf(typeof(INPUT)));
}

private void MouseEvent(uint flags, int x, int y)
{
    INPUT[] input = new INPUT[1];
    input[0].Type = INPUT_MOUSE;
    input[0].MouseInput.X = x;
    input[0].MouseInput.Y = y;
    input[0].MouseInput.Flags = flags;
    SendInput(1, input, Marshal.SizeOf(typeof(INPUT)));
}

private void MoveMouseGradually(int startX, int startY, int endX, int endY, int steps)
{
    int stepX = (endX - startX) / steps;
    int stepY = (endY - startY) / steps;
    int currentX = startX;
    int currentY = startY;

    for (int i = 0; i < steps; i++)
    {
        currentX += stepX;
        currentY += stepY;
        MoveMouse(currentX, currentY);
        Thread.Sleep(20);
    }
    // 确保最终到达目标坐标
    MoveMouse(endX, endY);
}

3. 额外检查项 #

• 用UIAutomation工具验证from_element和to_element的CurrentBoundingRectangle是否为资源管理器中文件、文件夹的准确屏幕坐标。
• 多显示器环境下，使用元素所在屏幕的Bounds而非PrimaryScreen，避免坐标转换错误。
• 确保程序以足够权限运行（比如管理员权限），避免被资源管理器的输入验证拦截。

内容的提问来源于stack exchange，提问作者Even