如何用Nokogiri按指定格式提取HTML表格数据？

问题情况 #

提取HTML表格数据时，所有列都重复了全部单元格内容，输出格式不符合预期。

原代码 #

class HtmlTablePage
    include PageObject

    page_url 'https://www.w3schools.com/html/html_tables.asp'

    table(:table, id: 'customers')

    def get_table_data
        page = Nokogiri::HTML.parse(table_element.html)
        table = page.css('table#customers tbody tr th')
        data = {}
        for i in table do
            data[i.text] = page.css("table#customers tbody tr td").map(&:text)
        end
        p data
    end
end

当前输出 #

{"Company"=>["Alfreds Futterkiste", "Maria Anders", "Germany", "Centro comercial Moctezuma", "Francisco Chang", "Mexico", "Ernst Handel", "Roland Mendel", "Austria", "Island Trading", "Helen Bennett", "UK", "Laughing Bacchus Winecellars", "Yoshi Tannamuri", "Canada", "Magazzini Alimentari Riuniti", "Giovanni Rovelli", "Italy"], "Contact"=>["Alfreds Futterkiste", "Maria Anders", "Germany", "Centro comercial Moctezuma", "Francisco Chang", "Mexico", "Ernst Handel", "Roland Mendel", "Austria", "Island Trading", "Helen Bennett", "UK", "Laughing Bacchus Winecellars", "Yoshi Tannamuri", "Canada", "Magazzini Alimentari Riuniti", "Giovanni Rovelli", "Italy"], "Country"=>["Alfreds Futterkiste", "Maria Anders", "Germany", "Centro comercial Moctezuma", "Francisco Chang", "Mexico", "Ernst Handel", "Roland Mendel", "Austria", "Island Trading", "Helen Bennett", "UK", "Laughing Bacchus Winecellars", "Yoshi Tannamuri", "Canada", "Magazzini Alimentari Riuniti", "Giovanni Rovelli", "Italy"]}

期望输出 #

{'Company': ['Alfreds Futterkiste', 'Centro comercial Moctezuma', 'Ernst Handel', 'Island Trading', 'Laughing Bacchus Winecellars', 'Magazzini Alimentari Riuniti'], 'Country': ['Germany', 'Mexico', 'Austria', 'UK', 'Canada', 'Italy'], 'Contact': ['Maria Anders', 'Francisco Chang', 'Roland Mendel', 'Helen Bennett', 'Yoshi Tannamuri', 'Giovanni Rovelli']}

解决方案 #

问题原因 #

原代码循环时，每个表头字段都被赋值了所有<td>的文本内容，没有按列分组提取数据，导致所有列内容重复。

修正代码 #

class HtmlTablePage
    include PageObject

    page_url 'https://www.w3schools.com/html/html_tables.asp'

    table(:table, id: 'customers')

    def get_table_data
        page = Nokogiri::HTML.parse(table_element.html)
        # 获取表头文本
        headers = page.css('table#customers tbody tr th').map(&:text)
        # 初始化数据结构，每个表头对应空数组
        data = headers.each_with_object({}) { |header, hash| hash[header] = [] }
        # 遍历所有数据行（跳过表头行）
        page.css('table#customers tbody tr:not(:has(th))').each do |row|
            # 获取当前行的所有单元格文本
            cells = row.css('td').map(&:text)
            # 将每个单元格文本对应到表头列中
            headers.each_with_index do |header, index|
                data[header] << cells[index]
            end
        end
        p data
    end
end

说明 #

1. 先提取所有表头文本，初始化每个表头对应的空数组；
2. 遍历所有不含表头的数据行，提取每行的单元格文本；
3. 按索引将单元格文本对应到对应的表头数组中，实现按列分组存储数据。

内容的提问来源于stack exchange，提问作者spectator