如何在Python中移除网页内的accordion/fusion-panel相关CSS代码
问题描述
我用Python结合BeautifulSoup编写了如下代码,提取指定class的article标签下的div内容,处理后可得到所需文本,但偶尔会混入accordion/fusion-panel相关的CSS样式代码,请问如何删除这些内容?
原代码:
toggle=[] for i in soup.find_all('article', {'class': 'post post-113572 type-post status-publish format-standard has-post-thumbnail hentry category-kurzprogramm'}): per_toggle=i.find('div') print(per_toggle) toggle.append(per_toggle) New_toggle=[] for each in toggle: new_each = str(each).replace('<br/', '') pattern = re.compile('<.*?>') result = re.sub(pattern, '', new_each) print (result) New_toggle.append(result) df=pd.DataFrame(New_toggle)
混入的CSS代码示例:
#accordion-113572-1 .fusion-panel:hover{ border-color: #d4d4d4 } #accordion-113572-1 .fusion-panel { border-color:#d4d4d4; }.fusion-accordian #accordion-113572-1 .panel-title a .fa-fusion-box{ color: #ffffff;}.fusion-accordian #accordion-113572-1 .panel-title a .fa-fusion-box:before{ font-size: 13px; width: 13px;}.fusion-accordian #accordion-113572-1 .panel-title a{font-size:19px;color:#191919;font-family:""Barlow Condensed"";font-weight:500;}.fusion-accordian #accordion-113572-1 .toggle-content{font-size:20px;color:#000000;font-family:""Barlow"";font-weight:regular;}.fusion-accordian #accordion-113572-1 .fa-fusion-box { background-color: #009ee0;border-color: #009ee0;}.fusion-accordian #accordion-113572-1 .panel-title a:not(.active):hover, #accordion-113572-1 .fusion-toggle-boxed-mode:hover .panel-title a { color: #009ee0;}.fusion-accordian #accordion-113572-1 .panel-title .active .fa-fusion-box,.fusion-accordian #accordion-113572-1 .panel-title a:not(.active):hover .fa-fusion-box { background-color: #009ee0!important;border-color: #009ee0!important;}.fusion-accordian #accordion-113572-1 .panel-f0b74f27ef9a21ca8 .panel-title a{}.fusion-accordian #accordion-113572-1 .panel-f0b74f27ef9a21ca8 .toggle-content{}
解决方案
方法1:提前移除含CSS的标签(推荐)
这些CSS通常存放在<style>标签,或是带有特定标识的容器里,在解析阶段直接删除这些元素,从根源避免冗余内容混入:
import re import pandas as pd from bs4 import BeautifulSoup # 先移除页面所有style标签 for style_tag in soup.find_all('style'): style_tag.decompose() # 移除所有包含fusion-panel或accordion关键词的容器 for panel in soup.find_all(class_=re.compile(r'fusion-panel|accordion')): panel.decompose() # 再提取目标内容 toggle = [] for article in soup.find_all('article', {'class': 'post post-113572 type-post status-publish format-standard has-post-thumbnail hentry category-kurzprogramm'}): per_toggle = article.find('div') if per_toggle: toggle.append(per_toggle) New_toggle = [] for each in toggle: # 用get_text直接提取纯文本,自动去除HTML标签 pure_text = each.get_text(strip=True, separator=' ') New_toggle.append(pure_text) df = pd.DataFrame(New_toggle, columns=['提取内容'])
方法2:正则匹配清除已混入的CSS文本
如果CSS已经混入到提取的文本中,可以用正则表达式匹配并删除符合特征的CSS代码:
# 在原有文本处理步骤后添加以下代码 css_pattern = re.compile(r'#accordion-\d+-\d+ .*?\{\s*.*?\s*\}|.[a-z0-9-]+ .*?\{\s*.*?\s*\}') clean_result = css_pattern.sub('', result) New_toggle.append(clean_result)
方法3:优化原有文本提取逻辑
原代码将标签转成字符串再处理容易引入冗余,改用BeautifulSoup原生的get_text()方法提取纯文本,能大幅减少垃圾内容:
New_toggle = [] for each in toggle: # 提取纯文本,去除首尾空白并统一分隔符 pure_text = each.get_text(strip=True, separator=' ') # 清理可能残留的CSS css_pattern = re.compile(r'#accordion-\d+-\d+ .*?\{\s*.*?\s*\}') clean_text = css_pattern.sub('', pure_text) New_toggle.append(clean_text) df = pd.DataFrame(New_toggle)
内容的提问来源于stack exchange,提问作者AnGeL
相关产品推荐
相关产品推荐

