如何使用Cheerio遍历爬取数据生成正确的JSON数组？

Hey there! Let's figure out why your matchDate field is acting up when scraping with Cheerio and generating that JSON file. It sounds like the date increment logic you added is causing unintended changes—let's break this down and fix it.

What's Probably Causing the matchDate Error #

Most often, this happens because your date variable has the wrong scope or you're incrementing it in the wrong place. For example, if you initialize a date outside your loop and increment it every time you process a single match, you'll end up with a date that gets bigger for every item—even if multiple matches belong to the same day.

Here's a common version of the problematic code:

const cheerio = require('cheerio');
const fs = require('fs');

const html = '<your-scraped-html-content>';
const $ = cheerio.load(html);

let currentDate = new Date('2024-05-01');
const matches = [];

// 🔴 错误：每个match都让日期自增，导致日期连续累加
$('.match-item').each((index, element) => {
  currentDate.setDate(currentDate.getDate() + 1);
  
  matches.push({
    matchDate: currentDate.toISOString().split('T')[0],
    team1: $(element).find('.team1').text().trim(),
    team2: $(element).find('.team2').text().trim()
  });
});

fs.writeFileSync('matches.json', JSON.stringify(matches, null, 2));

In this case, if you have 5 matches on the same day, you'll end up with dates from 2024-05-02 to 2024-05-06 instead of all showing 2024-05-01.

Fixes to Get Correct matchDate Values #

Let's go over two common scenarios and their fixes:

Scenario 1: Dates are already on the page (best practice!)

If each match (or a group of matches) has its date displayed in the HTML, just scrape that directly instead of generating it manually. This is way more reliable.

const cheerio = require('cheerio');
const fs = require('fs');

const html = '<your-scraped-html-content>';
const $ = cheerio.load(html);

const matches = [];

$('.match-item').each((index, element) => {
  // 🟢 直接从页面元素中提取日期
  const rawDate = $(element).find('.match-date').text().trim();
  // 如果需要格式化日期（比如转成YYYY-MM-DD）
  const formattedDate = new Date(rawDate).toISOString().split('T')[0];

  matches.push({
    matchDate: formattedDate,
    team1: $(element).find('.team1').text().trim(),
    team2: $(element).find('.team2').text().trim()
  });
});

fs.writeFileSync('matches.json', JSON.stringify(matches, null, 2));

If matches are grouped by date (e.g., a section for "May 1" with multiple matches), adjust the logic to grab the group's date first:

$('.date-group').each((groupIndex, groupElement) => {
  // 获取当前日期组的日期
  const groupDate = $(groupElement).find('.date-header').text().trim();
  const formattedDate = new Date(groupDate).toISOString().split('T')[0];

  // 遍历该日期下的所有赛事
  $(groupElement).find('.match-item').each((matchIndex, matchElement) => {
    matches.push({
      matchDate: formattedDate, // 所有该组的赛事用同一个日期
      team1: $(matchElement).find('.team1').text().trim(),
      team2: $(matchElement).find('.team2').text().trim()
    });
  });
});

Scenario 2: You need to generate sequential dates (e.g., daily matches)

If you're scraping matches for consecutive days and the page doesn't show dates, only increment the date after processing all matches for a single day, not per match.

const cheerio = require('cheerio');
const fs = require('fs');

const html = '<your-scraped-html-content>';
const $ = cheerio.load(html);

const matches = [];
let currentDate = new Date('2024-05-01');

// 假设页面按天划分，每天一个section
$('.daily-section').each((dayIndex, dayElement) => {
  const formattedDate = currentDate.toISOString().split('T')[0];

  // 处理当天的所有赛事
  $(dayElement).find('.match-item').each((matchIndex, matchElement) => {
    matches.push({
      matchDate: formattedDate,
      team1: $(matchElement).find('.team1').text().trim(),
      team2: $(matchElement).find('.team2').text().trim()
    });
  });

  // 🟢 处理完当天所有赛事后，再自增日期
  currentDate.setDate(currentDate.getDate() + 1);
});

fs.writeFileSync('matches.json', JSON.stringify(matches, null, 2));

Key Takeaway #

The core issue is making sure your date logic aligns with how the data is structured on the page. Always prefer scraping dates directly from the HTML if possible—this avoids manual date generation errors entirely. If you do need to generate dates, make sure you increment them only when you move to a new day's data, not for every individual match.

内容的提问来源于stack exchange，提问作者breakslow_