使用BeautifulSoup解析HTML时完整单词被拆分的问题求助

问题分析 #

当HTML标签嵌入单个单词内部（如<p>Hell<font>o</font></p>），使用get_text(separator=" ")会强制在标签位置添加空格，导致完整单词被拆分。而浏览器渲染时会自动忽略标签对单词的分割，因此需要模拟这种行为，同时保留段落分隔。

解决思路 #

1. 保留段落结构：先按<p>标签遍历内容，确保段落之间的分隔不受影响；
2. 智能拼接文本：对每个段落内的文本片段，仅当前后文本的边界本身包含空格时才添加分隔空格，否则直接拼接，避免拆分被标签打断的单词；
3. 过滤冗余空白：使用stripped_strings去除每个文本节点前后的冗余空白，同时保留文本内部的有效空格。

代码实现 #

from bs4 import BeautifulSoup
import html

data = """<p style='text-align:justify;margin-top:0pt;margin-bottom:0pt;line-height:12pt;' >
      <font style='font-family:Times New Roman;font-size:11pt;font-style:italic;margin-left:0pt;' >Accounting Pronouncements Adopted</font>
    </p>
    <p style='text-align:justify;line-height:12pt;' ></p>
    <p style='text-align:left;margin-top:0pt;margin-bottom:0pt;line-height:12pt;' >
      <font style='font-family:Times New Roman;font-size:11pt;margin-left:18pt;' >In March 2016, the Financial Accounting Standard Board (“FASB”) issued Accounting Standards Update (“ASU”) No. 2016-09, “Stock Compensation” (Topic 718) (“ASU 2016-09”).  ASU 2016-09 contains amended guidance for </font>
      <font style='font-family:Times New Roman;font-size:11pt;' >share-based payment accounting.  We adopted the provisions of this standard during the first quarter of 2017.  </font>
    </p>
    <p style='text-align:left;line-height:12pt;' ></p>
    <p style='text-align:left;margin-top:0pt;margin-bottom:0pt;line-height:12pt;' >
      <font style='font-family:Times New Roman;font-size:11pt;margin-left:18pt;' >Under ASU 2016-09, all excess tax benefits and tax deficiencies resulting from the difference between the deduction for tax purposes and the st</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >ock-based compensation cost recognized for financial reporting purposes are included as a component of income tax expense as of January 1, 2017.  Prior to the implementation of ASU 2016-09, excess tax benefits w</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >ere recorded as a component of A</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >dditional pai</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >d-</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >in capital and tax deficiencies were recognized either as an offset to accumulated excess tax benefits or in the income statement if there were no accumulated excess tax benefits.  The adoption of ASU 2016-09 reduced income tax expense by approximately $</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >19.</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >6</font>
      <font style='font-family:Times New Roman;font-size:11pt;' > million for the </font>
      <font style='font-family:Times New Roman;font-size:11pt;' >year</font>
      <font style='font-family:Times New Roman;font-size:11pt;' > ended </font>
      <font style='font-family:Times New Roman;font-size:11pt;' >Decem</font>
      <font style='font-family:Times New Roman;font-size:11pt;' >ber 30, 2017.</font>
    </p>"""

soup = BeautifulSoup(data, features="html.parser") 

# 处理每个段落，保留结构并拼接被标签打断的单词
result = []
for p in soup.find_all('p'):
    p_text_fragments = []
    for string in p.stripped_strings:
        if p_text_fragments:
            # 检查前后文本是否需要空格分隔
            last_fragment = p_text_fragments[-1]
            # 如果前一段末尾不是空格且当前段开头不是空格，直接拼接
            if not (last_fragment[-1].isspace() or string[0].isspace()):
                p_text_fragments[-1] += string
            else:
                p_text_fragments.append(string)
        else:
            p_text_fragments.append(string)
    # 拼接段落内的文本，空段落保留占位
    if p_text_fragments:
        result.append(' '.join(p_text_fragments))
    else:
        result.append('')

# 段落之间用空行分隔，并转义HTML字符
final_result = '\n\n'.join(result)
final_result = html.unescape(final_result)

print(final_result)

运行结果 #

Accounting Pronouncements Adopted


In March 2016, the Financial Accounting Standard Board (“FASB”) issued Accounting Standards Update (“ASU”) No. 2016-09, “Stock Compensation” (Topic 718) (“ASU 2016-09”).  ASU 2016-09 contains amended guidance for share-based payment accounting.  We adopted the provisions of this standard during the first quarter of 2017.


Under ASU 2016-09, all excess tax benefits and tax deficiencies resulting from the difference between the deduction for tax purposes and the stock-based compensation cost recognized for financial reporting purposes are included as a component of income tax expense as of January 1, 2017.  Prior to the implementation of ASU 2016-09, excess tax benefits were recorded as a component of Additional paid-in capital and tax deficiencies were recognized either as an offset to accumulated excess tax benefits or in the income statement if there were no accumulated excess tax benefits.  The adoption of ASU 2016-09 reduced income tax expense by approximately $19.6 million for the year ended December 30, 2017.

内容的提问来源于stack exchange，提问作者doorpop