如何使用Python BeautifulSoup提取HTML代码中的表格

问题描述 #

我正尝试用Python的BeautifulSoup库从给定的HTML代码中提取表格数据，以下是待处理的HTML代码示例，期望生成指定格式的Markdown表格输出。

待处理HTML代码：

{"html":"<table width=\"100%\" cellspacing=\"0\" cellpadding=\"0\" class=\"gr_table_b1\">\n  <tbody>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\" class=\"gr_table_colm10\">Bond Type<\/td>\n      <td valign=\"top\" align=\"right\">\n         US Corporate Debentures\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Debt Type<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Senior Unsecured Note\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Industry Group<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  Industrial\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Industry Sub Group<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Transportation\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\" class=\"gr_table_colm10\">Sub-Product Asset<\/td>\n      <td valign=\"top\" align=\"right\">\n         CORP\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Sub-Product Asset Type<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Corporate Bond\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">State<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  &amp;mdash;\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Use of Proceeds<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t &amp;mdash;\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Security Code<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t &amp;mdash;\n      <\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n<div class=\"gr_row_b6 gr_table_title\">Special Characteristics<\/div>\n<div class=\"gr_section_b1\">\n  <table width=\"100%\" cellspacing=\"0\" cellpadding=\"0\" class=\"gr_table_b1\">\n   <tbody>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Medium Term Note<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  N\n      <\/td>\n    <\/tr>\n   <\/tbody>\n  <\/table>\n <\/div>"}

期望输出：

| Bond Type              | US Corporate Debentures  |
| Debt Type              | Senior Unsecured Note    |
| Industry Group         | Industrial               |
| Industry Sub Group     | Transportation           |
| Sub-Product Asset      | CORP                     |
| Sub-Product Asset Type | Corporate Bond           |
| State                  | —                        |
| Use of Proceeds        | —                        |
| Security Code          | —                        |
|                        |                          |

解决方案 #

通过BeautifulSoup解析HTML，遍历目标表格行，提取单元格文本并处理格式，最终生成符合要求的Markdown表格。

实现代码：

from bs4 import BeautifulSoup
import json

# 解析JSON获取HTML内容
raw_data = json.loads('{"html":"<table width=\"100%\" cellspacing=\"0\" cellpadding=\"0\" class=\"gr_table_b1\">\n  <tbody>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\" class=\"gr_table_colm10\">Bond Type<\/td>\n      <td valign=\"top\" align=\"right\">\n         US Corporate Debentures\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Debt Type<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Senior Unsecured Note\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Industry Group<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  Industrial\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Industry Sub Group<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Transportation\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\" class=\"gr_table_colm10\">Sub-Product Asset<\/td>\n      <td valign=\"top\" align=\"right\">\n         CORP\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Sub-Product Asset Type<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t Corporate Bond\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">State<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  &amp;mdash;\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Use of Proceeds<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t &amp;mdash;\n      <\/td>\n    <\/tr>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Security Code<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t &amp;mdash;\n      <\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n<div class=\"gr_row_b6 gr_table_title\">Special Characteristics<\/div>\n<div class=\"gr_section_b1\">\n  <table width=\"100%\" cellspacing=\"0\" cellpadding=\"0\" class=\"gr_table_b1\">\n   <tbody>\n    <tr class=\"gr_table_row4\">\n      <td valign=\"top\">Medium Term Note<\/td>\n      <td valign=\"top\" align=\"right\">\n      \t  N\n      <\/td>\n    <\/tr>\n   <\/tbody>\n  <\/table>\n <\/div>"}')
soup = BeautifulSoup(raw_data['html'], 'html.parser')

# 获取所有目标表格行
table_rows = soup.find_all('tr', class_='gr_table_row4')

# 生成Markdown表格内容
markdown_lines = []
for row in table_rows:
    cells = row.find_all('td')
    if len(cells) == 2:
        # 清理文本空白并替换HTML实体
        col1 = cells[0].get_text(strip=True)
        col2 = cells[1].get_text(strip=True).replace('&mdash;', '—')
        # 按格式拼接行
        markdown_lines.append(f"| {col1:<22} | {col2:<24} |")

# 添加要求的空行
markdown_lines.append("|                        |                          |")

# 输出结果
print('\n'.join(markdown_lines))

代码说明：

1. 先解析JSON字符串，提取出核心HTML内容；
2. 使用find_all定位所有目标表格行，遍历每行提取两个单元格的文本；
3. 清理文本前后的空白字符，将HTML实体—替换为视觉上的破折号—；
4. 按照指定的对齐格式拼接成Markdown表格行，最后添加要求的空行。

内容的提问来源于stack exchange，提问作者Rong Ren