如何用Puppeteer将HTML交易数据提取为指定嵌套JSON结构？

问题描述 #

我用Puppeteer抓取交易历史数据时，无法将抓取到的元素innerText转换成指定的嵌套JSON结构。

预期JSON结构 #

[
  {
    "date": "Today",
    "transactions": [
      { "amount": "-28,000" },
      { "amount": "+5,000" },
      { "amount": "-5,000" }
    ]
  },
  {
    "date": "Yesterday",
    "transactions": [
      { "amount": "-24,000" },
      { "amount": "-141,000" },
      { "amount": "+50,000" }
    ]
  }
]

目标HTML结构 #

<div class="transaction-list-group">
    <div class="transaction-list-group__header">Today</div>
    <div class="transaction-list-group__body">
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">Bobby Timmy</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount">-28,000</div>
                </div>
            </div>
        </div>
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">John Doe</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount incoming">+5,000</div>
                </div>
            </div>
        </div>
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">Outgoing money</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount">-5,000</div>
                </div>
            </div>
        </div>
    </div>
</div>
<div class="transaction-list-group">
    <div class="transaction-list-group__header">Yesterday</div>
    <div class="transaction-list-group__body">
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">Adam Crash</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount">-24,000</div>
                </div>
            </div>
        </div>
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">Alexi Pattim</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount">-141,000</div>
                </div>
            </div>
        </div>
        <div class="transaction-list clickable">
            <div class="transaction-list__content">
                <div class="transaction-info">
                    <div class="transaction-info__party-name">McKenzy Smithy</div>
                    <div class="transaction-info__description">
                        Lorem ipsum dolor sit amet consectetur adipisicing elit. Rerum, distinctio?
                    </div>
                </div>
                <div class="transaction-amount">
                    <div class="transaction-amount__currency-amount incoming">+50,000</div>
                </div>
            </div>
        </div>
    </div>
</div>

当前问题代码 #

await page.$$eval('.transaction-list-group', (nodes) => nodes.map(element => ({
     date: element.querySelector(".transaction-list-group__header").innerText,
     transactions: [
         {
             amount: element.querySelector(".transaction-amount__currency-amount").innerText
         }
     ]
   })
))

解决方案 #

你当前代码的问题是用querySelector只获取了每个分组下的第一个交易金额，而非所有交易。要实现预期的嵌套结构，需要用querySelectorAll选中当前分组下的所有交易金额元素，再转换为数组后映射成目标格式。

修正后的代码如下：

await page.$$eval('.transaction-list-group', (groups) => 
  groups.map(group => {
    const date = group.querySelector('.transaction-list-group__header').innerText;
    // 获取当前分组下所有交易金额元素
    const amountElements = group.querySelectorAll('.transaction-amount__currency-amount');
    // 转换为数组并映射成目标结构
    const transactions = Array.from(amountElements).map(el => ({
      amount: el.innerText.trim() // 可选：用trim()去除文本中多余空格
    }));
    return { date, transactions };
  })
);

代码说明 #

• querySelectorAll('.transaction-amount__currency-amount')：选中当前transaction-list-group下的所有交易金额元素
• Array.from(amountElements)：把NodeList转换为可遍历的数组
• .map(el => ({ amount: el.innerText.trim() }))：将每个金额元素转换为{ amount: ... }的对象，trim()用于清理文本中的多余空格（可选）

运行这段代码后，就能得到你预期的嵌套JSON结构。

内容的提问来源于stack exchange，提问作者Joker Dark Knight