Puppeteer无法点击SAP UI页面按钮？ABB招聘页爬取咨询

Let's break down why your script isn't clicking the "More" button on ABB's SAP UI5-powered job board, plus how to fix it and successfully scrape the page.

Why Your Clicks Are Failing #

SAP UI5 sites have unique quirks that throw off standard Puppeteer interactions:

• Dynamic rendering delays: SAP UI5 loads components asynchronously. Just waiting for the selector to exist doesn't mean the element is fully interactive—event handlers might still be attaching, or the element could be hidden off-screen.
• Viewport visibility: The "More" button is often positioned below the initial viewport. Puppeteer can't interact with elements that aren't visible on the screen by default.
• Custom event handling: SAP UI5 uses its own press event for button-like components instead of the native browser click event. A plain e.click() might not trigger the intended action.

Fixing the Button Click #

Here's how to adjust your script to work with SAP UI5:

1. Wait for Full Element Readiness #

Instead of just checking for the selector, wait until the element is visible and interactive. Increase the timeout to account for SAP UI5's slower loading:

await page.waitForSelector("#__component0---map_search--tableJobs-trigger", {
  visible: true,
  timeout: 60000 // 60-second timeout to handle slow loads
});

2. Scroll the Element Into View #

Since the button is likely off-screen, scroll to it before trying to click:

await page.evaluate(selector => {
  const element = document.querySelector(selector);
  if (element) {
    element.scrollIntoView({ behavior: 'smooth', block: 'center' });
  }
}, "#__component0---map_search--tableJobs-trigger");
// Add a short delay to let the scroll complete
await page.waitForTimeout(1000);

3. Trigger SAP UI5's Custom Event #

Use SAP UI5's native fireEvent method to trigger the button's action instead of a standard click:

await page.evaluate(selector => {
  const element = document.querySelector(selector);
  if (element) {
    element.fireEvent("press"); // This is what SAP UI5 listens for
    console.log("Successfully triggered the 'More' action");
  }
}, "#__component0---map_search--tableJobs-trigger");

Full Modified Script #

Here's the complete working script with all fixes:

const puppeteer = require('puppeteer');

let URL = "https://jobs.abb.com/jobsearch/?sap-language=EN#";

const ping = async () => {
  const browser = await puppeteer.launch({ 
    headless: false, 
    args:['--start-maximized' ] 
  });
  const page = await browser.newPage();
  await page.setViewport({ width: 1366, height: 768 });
  await page.setDefaultNavigationTimeout(0);
  
  // Use networkidle2 for better readiness than just "load"
  await page.goto(URL, { waitUntil: "networkidle2" });
  
  // Wait for the button to be visible
  await page.waitForSelector("#__component0---map_search--tableJobs-trigger", {
    visible: true,
    timeout: 60000
  });
  console.log("Selector found and visible");
  
  // Scroll to the button
  await page.evaluate(selector => {
    const element = document.querySelector(selector);
    if (element) {
      element.scrollIntoView({ behavior: 'smooth', block: 'center' });
    }
  }, "#__component0---map_search--tableJobs-trigger");
  await page.waitForTimeout(1000);
  
  // Trigger the SAP UI5 press event
  await page.evaluate(selector => {
    const element = document.querySelector(selector);
    if (element) {
      element.fireEvent("press");
    }
  }, "#__component0---map_search--tableJobs-trigger");
  
  // Wait for new jobs to load
  await page.waitForTimeout(3000);
  
  // Add your scraping logic here (e.g., extract job titles, links)
  
  // await browser.close();
}

ping();

Can You Successfully Scrape This Page? #

Absolutely! SAP UI5 sites are scrapeable with Puppeteer, just keep these tips in mind:

• Repeat pagination: After clicking "More", check if the button is still visible/inactive to know when all jobs are loaded.
• Avoid anti-scraping blocks: Add small delays between actions and don't send requests too rapidly.
• Handle dynamic content: Always wait for elements to be fully visible/interactive before scraping them.

内容的提问来源于stack exchange，提问作者Vithu Jey