如何用BeautifulSoup解除HTML注释包裹的内容?
如何用BeautifulSoup高效提取被HTML注释包裹的目标元素
我正在使用bs4 4.10.0版本的BeautifulSoup进行网页爬取,遇到一个问题:部分目标元素被HTML注释包裹,需要将注释内的HTML节点还原为正常可解析的节点,而非删除注释内容。
原始HTML示例
<div class="h-[125] js-scroll-hidden" id="link-index-40"> <!-- <a href="/md5/dc0bbd5373a5ada24373640dab8defb3" class="custom-a flex items-center relative left-[-10] px-[10] py-2 hover:bg-[#00000011] "> <div class="flex-none"> <div class="relative overflow-hidden w-[72] h-[108] flex flex-col justify-center"> <div class="absolute w-[100%] h-[90]" style="background-color: hsl(63deg 43% 73%)"></div> <img class="relative inline-block" src="https://libgen.rs/covers/2274000/dc0bbd5373a5ada24373640dab8defb3-g.jpg" alt="" referrerpolicy="no-referrer" onerror="this.parentNode.removeChild(this)" loading="lazy" decoding="async"/> </div> </div> <div class="relative top-[-1] pl-4 grow overflow-hidden"> <div class="truncate text-xs text-gray-500">English [en], pdf, 11.7MB, "The Dale Carnegie course in effective spea - Dale Carnegie.pdf"</div> <h3 class="truncate text-xl font-bold">The Dale Carnegie course in effective speaking, human relations and developing courage and confidence, improving your memory, leadership training : how the course is conducted and what you do at each session</h3> <div class="truncate text-sm">Dale Carnegie, 1989</div> <div class="truncate italic">Dale Carnegie & Associates, Inc.</div> </div> </a> --> </div>
已尝试的方案
- 搜索到的多数方案是直接移除注释内容,不符合我的需求
- 尝试修改标签内容,但会破坏后续需要使用的
.find()和.find_all()方法 - 手动处理注释符号,但未找到可行的方式
- 现有可行方法资源消耗过大,而我需要处理200+这类元素,希望找到高效实现方式
期望结果
<div class="h-[125] js-scroll-hidden" id="link-index-40"> <a href="/md5/dc0bbd5373a5ada24373640dab8defb3" class="custom-a flex items-center relative left-[-10] px-[10] py-2 hover:bg-[#00000011] "> <div class="flex-none"> <div class="relative overflow-hidden w-[72] h-[108] flex flex-col justify-center"> <div class="absolute w-[100%] h-[90]" style="background-color: hsl(63deg 43% 73%)"></div> <img class="relative inline-block" src="https://libgen.rs/covers/2274000/dc0bbd5373a5ada24373640dab8defb3-g.jpg" alt="" referrerpolicy="no-referrer" onerror="this.parentNode.removeChild(this)" loading="lazy" decoding="async"/> </div> </div> <div class="relative top-[-1] pl-4 grow overflow-hidden"> <div class="truncate text-xs text-gray-500">English [en], pdf, 11.7MB, "The Dale Carnegie course in effective spea - Dale Carnegie.pdf"</div> <h3 class="truncate text-xl font-bold">The Dale Carnegie course in effective speaking, human relations and developing courage and confidence, improving your memory, leadership training : how the course is conducted and what you do at each session</h3> <div class="truncate text-sm">Dale Carnegie, 1989</div> <div class="truncate italic">Dale Carnegie & Associates, Inc.</div> </div> </a> </div>
请问是否有BeautifulSoup原生、低资源消耗的方法实现上述需求?也可推荐其他易用工具。
内容的提问来源于stack exchange,提问作者Ethan
相关产品推荐
相关产品推荐

