基于Selenium与WebSocket的连续序列录制器开发问题

背景与核心需求 #

• 开发序列录制器：客户端提交URL，服务器端通过Selenium启动浏览器打开该URL，通过WebSocket同步页面PNG到客户端供用户交互，最终生成符合规范的.side格式录制文件
• 必须满足：页面刷新、跳转（如触发登录表单加载）时，录制不能中断，需持续捕获所有用户操作

目标.side文件格式示例 #

{  
  "id": "38b09777-d8d1-4e07-a985-5a4a300004bf",
  "version": "2.0",
  "name": "test",
  "url": "http://testphp.vulnweb.com",
  "tests": [
    {
      "id": "3d6ea84f-d64f-4109-92f2-c0c96bafda65",
      "name": "stop",
      "commands": [
        {
          "id": "8f5c8c81-3628-4505-b767-856de504999a",
          "comment": "",
          "command": "open",
          "target": "/",
          "targets": [],
          "value": ""
        },
        {
          "id": "75978eab-8205-4fe1-84a4-e4030f96afe2",
          "comment": "",
          "command": "setWindowSize",
          "target": "1760x1055",
          "targets": [],
          "value": ""
        },
        {
          "id": "2b218310-7214-45b5-addf-6b137ad04b5e",
          "comment": "",
          "command": "click",
          "target": "linkText=Your profile",
          "targets": [
            ["linkText=Your profile", "linkText"],
            ["css=li:nth-child(5) > a", "css:finder"],
            ["xpath=//a[contains(text(),'Your profile')]", "xpath:link"],
            ["xpath=//div[@id='sectionLinks']/ul/li[5]/a", "xpath:idRelative"],
            ["xpath=//a[contains(@href, 'userinfo.php')]", "xpath:href"],
            ["xpath=//li[5]/a", "xpath:position"],
            ["xpath=//a[contains(.,'Your profile')]", "xpath:innerText"]
          ],
          "value": ""
        }
      ]
    }
  ]
}

可行的Playwright实现参考 #

Playwright通过事件委托绑定到body，能自动捕获动态元素点击，且页面刷新/跳转后可重新注入脚本维持录制：

document.body.addEventListener('click', function (event) {
    const clickedElement = event.target;
    const eventDetails = {
        id: generateUUID(),
        command: 'click',
        target: clickedElement.tagName.toLowerCase() + (clickedElement.id ? '#' + clickedElement.id : ''),
        value: ''
    };
    console.log(eventDetails);
    window.localStorage.setItem('recordedEvents', JSON.stringify(window.localStorage.getItem('recordedEvents') ? JSON.parse(window.localStorage.getItem('recordedEvents')).concat(eventDetails) : [eventDetails]));
});

原Selenium代码的问题 #

原代码给页面每个元素单独绑定click事件，页面刷新/跳转后，旧DOM元素被销毁，绑定的事件监听也随之失效，导致录制中断：

const clickableElements = await driver.findElements(By.css("body *")); // 选择所有元素
for (const element of clickableElements) {
    const tagName = await element.getTagName();

    await driver.executeScript(`
        const element = arguments[0];
        element.addEventListener('click', function() {
            const eventDetails = {
                id: "${generateUUID()}",
                command: "click",
                target: "${tagName.toLowerCase()}" + (element.id ? "#" + element.id : ""),
                value: ""
            };
            window.localStorage.setItem('recordedEvents', JSON.stringify(JSON.parse(window.localStorage.getItem('recordedEvents') || '[]').concat(eventDetails)));
        });
    `, element);
}

解决步骤与代码实现 #

1. 改用事件委托捕获所有点击事件 #

将事件监听绑定到document，利用事件冒泡机制捕获所有元素的点击，包括动态加载的元素：

await driver.executeScript(`
    // 页面内定义UUID生成函数
    window.generateUUID = function() {
        return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
            var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
            return v.toString(16);
        });
    };

    // 绑定事件委托到document
    document.addEventListener('click', function(event) {
        const clickedElement = event.target;
        const eventDetails = {
            id: window.generateUUID(),
            command: "click",
            target: clickedElement.tagName.toLowerCase() + (clickedElement.id ? "#" + clickedElement.id : ""),
            value: ""
        };
        // 存储事件到localStorage
        const existingEvents = JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
        existingEvents.push(eventDetails);
        window.localStorage.setItem('recordedEvents', JSON.stringify(existingEvents));
    });
`);

2. 监听页面导航，自动重新注入脚本 #

页面刷新或跳转后，之前注入的脚本会被清空，通过Selenium监听navigation事件，在每次页面加载完成后重新注入录制脚本：

const { until } = require('selenium-webdriver');

// 注册导航完成监听
driver.on('navigation', async () => {
    await driver.wait(until.documentIsReady());
    // 重新注入事件委托脚本
    await driver.executeScript(`
        window.generateUUID = function() {
            return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
                var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
                return v.toString(16);
            });
        };

        document.addEventListener('click', function(event) {
            const clickedElement = event.target;
            const eventDetails = {
                id: window.generateUUID(),
                command: "click",
                target: clickedElement.tagName.toLowerCase() + (clickedElement.id ? "#" + clickedElement.id : ""),
                value: ""
            };
            const existingEvents = JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
            existingEvents.push(eventDetails);
            window.localStorage.setItem('recordedEvents', JSON.stringify(existingEvents));
        });
    `);
});

// 初始页面加载后注入脚本
await driver.wait(until.documentIsReady());
await driver.executeScript(`
    window.generateUUID = function() {
        return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
            var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
            return v.toString(16);
        });
    };

    document.addEventListener('click', function(event) {
        const clickedElement = event.target;
        const eventDetails = {
            id: window.generateUUID(),
            command: "click",
            target: clickedElement.tagName.toLowerCase() + (clickedElement.id ? "#" + clickedElement.id : ""),
            value: ""
        };
        const existingEvents = JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
        existingEvents.push(eventDetails);
        window.localStorage.setItem('recordedEvents', JSON.stringify(existingEvents));
    });
`);

3. 扩展录制范围（可选） #

若需要捕获输入、表单提交等更多操作，添加对应事件监听：

// 注入扩展事件监听的脚本
await driver.executeScript(`
    // 监听输入事件
    document.addEventListener('input', function(event) {
        const target = event.target;
        if (['input', 'textarea'].includes(target.tagName.toLowerCase())) {
            const eventDetails = {
                id: window.generateUUID(),
                command: "type",
                target: target.tagName.toLowerCase() + (target.id ? "#" + target.id : ""),
                value: target.value
            };
            const existingEvents = JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
            existingEvents.push(eventDetails);
            window.localStorage.setItem('recordedEvents', JSON.stringify(existingEvents));
        }
    });

    // 监听表单提交事件
    document.addEventListener('submit', function(event) {
        const target = event.target;
        if (target.tagName.toLowerCase() === 'form') {
            const eventDetails = {
                id: window.generateUUID(),
                command: "submit",
                target: target.tagName.toLowerCase() + (target.id ? "#" + target.id : ""),
                value: ""
            };
            const existingEvents = JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
            existingEvents.push(eventDetails);
            window.localStorage.setItem('recordedEvents', JSON.stringify(existingEvents));
        }
    });
`);

4. 导出为.side格式文件 #

录制完成后，从页面localStorage读取事件，整理成.side规范的结构并导出：

const fs = require('fs');

// 服务器端UUID生成函数
function generateUUID() {
    return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
        var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
        return v.toString(16);
    });
}

// 获取页面录制的事件
const recordedEvents = await driver.executeScript(`
    return JSON.parse(window.localStorage.getItem('recordedEvents') || '[]');
`);

// 构建.side格式数据
const sideFile = {
    id: generateUUID(),
    version: "2.0",
    name: "recording",
    url: await driver.getCurrentUrl(),
    tests: [
        {
            id: generateUUID(),
            name: "test_session",
            commands: recordedEvents
        }
    ]
};

// 写入文件（树莓派本地存储）
fs.writeFileSync('./recording.side', JSON.stringify(sideFile, null, 2));

树莓派环境注意事项 #

• 确保安装适配ARM架构的浏览器驱动（如ChromiumDriver for ARM）
• 若使用无头模式，需配置浏览器参数支持WebSocket和页面截图功能

内容的提问来源于stack exchange，提问作者Nomadyb