Java中HtmlUnit点击元素被忽略：目标元素未显示的解决方案

问题 #

在Java中使用HtmlUnit尝试点击网页上的<span>元素（文本为+），该元素点击后应增加计数器并解锁提交按钮，但操作失败。日志提示目标元素未显示，但在浏览器中该元素可见且可点击。已尝试等待5秒，无效，猜测可能和元素处于带滚动溢出的div末尾有关。

相关代码 #

// Find all services to select "Sonstiges" service
List<DomElement> services = secondPage.getElementsByName("services");

for (DomElement service : services) {
    // Get correct service
    if (service.getNextElementSibling().getTextContent().equals("Sonstiges")) {
        service.getNextElementSibling().getNextElementSibling().getNextElementSibling()
                .getNextElementSibling().getChildElements().forEach(domElement -> {
            if (domElement.getTextContent().equals("+")) {
                // Press add service button
                try {
                    domElement.click();
                } catch (IOException e) {
                    logger.log(Level.WARNING, "Could not add service:
" + Util.exceptionToString(e));
                }
            }
        });
        break;
    }
}

// Click submit
DomElement secondSubmitButton = secondPage.getElementById("forward-service");
HtmlPage thirdPage = secondSubmitButton.click();

日志信息 #

Nov. 06, 2023 3:56:17 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:13 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:17 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:13 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:8762] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:8762] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:17897] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [59:17897] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:15911] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:15911] Ignoring the whole rule.
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler error
WARNUNG: CSS error: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:62364] Error in class selector. (Invalid token "and". Was expecting one of: "only", "inherit", <IDENT>.)
Nov. 06, 2023 3:56:17 PM org.htmlunit.DefaultCssErrorHandler warning
WARNUNG: CSS warning: 'https://onlinetermine.kaiserslautern.de/libs/semantic-ui-2.4.1/semantic.min.css' [75:62364] Ignoring the whole rule.
Nov. 06, 2023 3:56:18 PM org.htmlunit.WebConsole info
INFORMATION: [2023-11-06T14:56:18.683Z] WRN: module: clu.core is obselete use now rc.core, will removed in version 1.2.58
Nov. 06, 2023 3:56:19 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNUNG: Cookie rejected [rc_sid99f7e7c958a84cb084fd362b95200aaa="48f8b876f10b4be18c670eb8fb37998d7b3898197b76443bab04b27a7fae1e9f", version:0, domain:.callup.kaiserslautern.de, path:/m/Fuehrerscheinstelle, expiry:Mon Nov 06 16:16:15 CET 2023] Illegal 'domain' attribute ".callup.kaiserslautern.de". Domain of origin: "onlinetermine.kaiserslautern.de"
Nov. 06, 2023 3:56:19 PM org.htmlunit.WebConsole info
INFORMATION: [2023-11-06T14:56:19.143Z] WRN: module: clu.core is obselete use now rc.core, will removed in version 1.2.58
Nov. 06, 2023 3:56:24 PM org.htmlunit.html.DomElement click
WARNUNG: Calling click() ignored because the target element 'HtmlSpan[<span class="counterButton" onclick="changecap(1,2,'bd59a4e2-288d-4117-b6df-45c694d076fd')">]' is not displayed.

解决方案 #

1. 强制触发元素点击（绕过显示检查） #

HtmlUnit的默认click()会校验元素可见性，可调用重载方法强制执行点击：

if (domElement.getTextContent().equals("+")) {
    try {
        // 强转为HtmlElement，调用忽略可见性的click方法
        ((HtmlElement) domElement).click(true);
    } catch (IOException e) {
        logger.log(Level.WARNING, "Could not add service: " + Util.exceptionToString(e));
    }
}

2. 滚动到元素位置（标记为显示状态） #

如果元素在滚动容器内，先将其滚动到视口再点击：

if (domElement.getTextContent().equals("+")) {
    try {
        HtmlElement spanElement = (HtmlElement) domElement;
        // 滚动到元素所在位置，false表示滚动至元素底部
        spanElement.scrollIntoView(false);
        // 等待页面JS执行完成
        secondPage.waitForBackgroundJavaScript(2000);
        spanElement.click();
    } catch (IOException e) {
        logger.log(Level.WARNING, "Could not add service: " + Util.exceptionToString(e));
    }
}

3. 直接执行onclick脚本 #

绕过元素显示检查，直接触发绑定的业务逻辑：

if (domElement.getTextContent().equals("+")) {
    String onClickScript = domElement.getAttribute("onclick");
    if (onClickScript != null && !onClickScript.isEmpty()) {
        // 执行元素的onclick脚本
        secondPage.executeJavaScript(onClickScript);
        // 等待脚本执行，确保计数器更新和按钮解锁
        secondPage.waitForBackgroundJavaScript(3000);
    }
}

4. 优化元素定位逻辑（避免链式调用的脆弱性） #

多次getNextElementSibling()依赖DOM结构，改用CSS选择器更稳定：

// 直接用CSS选择器定位Sonstiges对应的+按钮
HtmlElement addButton = secondPage.querySelector("span:contains('Sonstiges') ~ .counterButton:contains('+')");
if (addButton != null) {
    try {
        addButton.click(true);
    } catch (IOException e) {
        logger.log(Level.WARNING, "Could not add service: " + Util.exceptionToString(e));
    }
}

注：contains()是HtmlUnit支持的扩展选择器，需确保使用的版本兼容。

内容的提问来源于stack exchange，提问作者Jonas Krug