使用Javascript、Cheerio和Axios爬取Transfermarkt时出现选择器未终止错误

问题场景 #

使用Javascript、Cheerio和Axios爬取Transfermarkt英超2022赛季射手榜数据时，出现如下报错：

throw new Error("Attribute selector didn't terminate");
Error: Attribute selector didn't terminate

原代码如下：

const fetchData = async(url) => {
    const result = await axios.get(url)
    return result.data
}

const main = async () => {
    const content = await fetchData("https://www.transfermarkt.com.br/premier-league/torschuetzenliste/wettbewerb/GB1/saison_id/2022")
    const $ = cheerio.load(content)
    let artilheiros = []

    $('table.items tbody tr').each((i, e) => {
        const AllElements = $(e).find('td')
        const nomeArtilheiro = $(`${AllElements[1]} > table > tbody > tr.hauptlink > a`).text();
        // const clubeArtilheiro
        // const jogos 
        // const gols 
        const data = {nomeArtilheiro}
        artilheiros.push(data)
    })

    console.log(artilheiros)
}

main()

报错原因 #

代码中错误地将Cheerio DOM对象AllElements[1]直接转成字符串拼接到选择器模板中，导致生成的选择器语法完全不符合规范，Cheerio无法解析这种混乱的选择器，因此抛出“属性选择器未终止”的错误。

修复方案 #

方案1：基于已有DOM对象链式调用find方法 #

不需要拼接选择器，直接在AllElements[1]对应的元素上调用find方法查找子元素：

$('table.items tbody tr').each((i, e) => {
    const AllElements = $(e).find('td')
    // 直接在AllElements[1]的元素上执行find
    const nomeArtilheiro = $(AllElements[1]).find('table > tbody > tr.hauptlink > a').text();
    const data = {nomeArtilheiro}
    artilheiros.push(data)
})

方案2：直接使用精准的CSS选择器 #

跳过AllElements的中间变量，直接在遍历的行元素中通过nth-child定位目标列，再查找子元素：

$('table.items tbody tr').each((i, e) => {
    const nomeArtilheiro = $(e).find('td:nth-child(2) table tbody tr.hauptlink a').text();
    // 补充其他字段的爬取逻辑示例：
    const clubeArtilheiro = $(e).find('td:nth-child(3) a').text();
    const jogos = $(e).find('td:nth-child(4)').text().trim();
    const gols = $(e).find('td:nth-child(5)').text().trim();
    const data = {nomeArtilheiro, clubeArtilheiro, jogos, gols}
    artilheiros.push(data)
})

额外提示 #

爬取Transfermarkt时建议添加请求头模拟浏览器访问，避免被反爬拦截，示例：

const fetchData = async(url) => {
    const result = await axios.get(url, {
        headers: {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
        }
    })
    return result.data
}

内容的提问来源于stack exchange，提问作者G4rutti