Puppeteer(Node.js)点击大学LMS面板列表项失败，求解决方法

问题背景 #

我通过Puppeteer（Node.js）登录大学LMS系统爬取数据，进入控制面板后，尝试点击第5个文本为“View My Current Enrollments”的列表项，但操作失败。试过用span、a标签、链接、类选择器以及page.evaluate等多种方式定位点击，均提示选择器不存在或未定义（已反复确认选择器正确）；尝试鼠标移动操作，但需要选择器获取坐标。

页面列表HTML代码 #

<ul class="row yic">

    <li class="col-md-3">
         <a href="https://my.kfueit.edu.pk/users/activeEnrollment_datesheet" target="_blank"><i      class="fa fa-calendar"></i> 
         <span>View My Roll No Slip</span>
        </a>
    </li>

    <li class="col-md-3">
          <a href="https://my.kfueit.edu.pk/examination/my_surveys"><i class="fa fa-newspaper-o"></i> 
         <span>View My Survey</span>
        <img style="display: block;position: absolute;width: 58px;right: 6px;top: 6px;" src="https://my.kfueit.edu.pk/assets/images/new_icon.gif">
    </a>
    </li>

    <li class="col-md-3">
        <a href="https://my.kfueit.edu.pk/users/transcript"><i class="fa  fa-address-book"></i>  <span>View My Transcript</span>
    </a>
    </li>


        <li class="col-md-3">                                                                                        <a href="https://my.kfueit.edu.pk/users/planofstudy"><i class="fa fa-list"></i> 
<span>View My Plan of Study</span>                                                          </a>                                                                                                    </li>
                
        <li class="col-md-3">
         <a href="https://my.kfueit.edu.pk/users/activeEnrollment"><i class="fa fa-book"></i> <span>View My Current Enrollments</span>
        </a>
    </li>

    <li class="col-md-3">
        <a href="https://my.kfueit.edu.pk/finance/challan"><i class="fa fa-money"></i>
         <span> Semester Fee Voucher</span>
        </a>
    </li>

    <li class="col-md-3">
           <a href="https://my.kfueit.edu.pk/users/hostelportal"><i class="fa fa-building"></i>
        <span>My Hostel</span>
         </a>                                                                                               </li>

         <li class="col-md-3">
            <a href="https://my.kfueit.edu.pk/studentcourses/course_registration"><i class="fa fa-book"></i> 
            <span>Course Registration</span>
            </a>
          </li>

         <li class="col-md-3">
            <a href="https://my.kfueit.edu.pk/users/student_clearance"><i class="fa fa-check"></i> <span>My Clearance</span>
            </a>
         </li>
                            

        <li class="col-md-3">
        <a href="https://my.kfueit.edu.pk/users/medical_profile"><i class="fa fa-medkit"></i> <span>Medical Profile</span>
    </a>
    </li>

        <li class="col-md-3">
    <a href="https://my.kfueit.edu.pk/users/issue_listing/cosc241103020"><i class="fa fa-podcast"></i>
<span>Create Token(SFSC)</span>
    </a>
        </li>

    <li class="col-md-3">
        <a href="https://my.kfueit.edu.pk/finance/download_facilitation_challan"><i class="fa fa-money"></i>
<span>Generate Challan</span>   
    </a>
        </li>

    </ul>

当前Puppeteer代码 #

(async()=>{
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();

  await page.goto('https://my.kfueit.edu.pk');

  await page.type('#username', 'COSC241103020');
  await page.type('#password', 'kfu#1122');

  const captchaElement = await page.$('#captImg img');
  const screenshotPath = 'captcha.png';
  await captchaElement.screenshot({ path: screenshotPath });

  const captchaText = await solveCaptchaWithOCR(screenshotPath);
  console.log('--captchatext----',captchaText)

  const answer = parseMathProblem(captchaText);
  console.log('=====answer====', answer);

  if (answer !== null) {
    await page.type('.form-control', answer.toString());
  } else {
    console.error('Could not solve the captcha');
  }

  console.log('------set----')

  await page.click('#submit');
  await page.waitForNavigation({ waitUntil: 'networkidle2' });

  await page.evaluate(() => {
    console.log("Found elements:", document.querySelectorAll("li.col-md-3").length);
  });
})();

可行的解决方法 #

• 等待元素完全加载：登录后的页面可能存在异步加载，waitForNavigation的networkidle2未必能确保元素就绪，改用waitForSelector等待目标元素：

  // 用href定位（最可靠，文本可能有空格差异）
  await page.waitForSelector('a[href="https://my.kfueit.edu.pk/users/activeEnrollment"]', { timeout: 10000 });
  

• 通过文本精准匹配定位：在page.evaluate中遍历元素，匹配文本内容后点击父元素：

  await page.evaluate(() => {
    const targetSpan = Array.from(document.querySelectorAll('li.col-md-3 span'))
      .find(span => span.textContent.trim() === 'View My Current Enrollments');
    if (targetSpan) targetSpan.parentElement.click();
    else console.error('目标元素未找到');
  });
  

• 通过索引定位第5个列表项：直接选择第5个li元素（CSS选择器中:nth-child从1开始计数）：

  await page.waitForSelector('ul.row.yic li.col-md-3:nth-child(5) a');
  await page.click('ul.row.yic li.col-md-3:nth-child(5) a');
  

• 模拟真实鼠标点击：如果直接click无效，先获取元素坐标，用鼠标模拟点击：

  const element = await page.$('a[href="https://my.kfueit.edu.pk/users/activeEnrollment"]');
  const box = await element.boundingBox();
  await page.mouse.click(box.x + box.width/2, box.y + box.height/2);
  

• 检查是否存在iframe：部分LMS会把内容嵌入iframe，需先切换到对应iframe再操作：

  // 替换成实际iframe的选择器
  const iframe = await page.waitForSelector('iframe[name="content-frame"]');
  const frame = await page.frame(iframe);
  await frame.click('a[href="https://my.kfueit.edu.pk/users/activeEnrollment"]');
  

• 设置真实浏览器UA：避免被网站识别为爬虫，设置符合真实浏览器的User-Agent：

  await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36');
  

内容的提问来源于stack exchange，提问作者Faisal Majeed