如何用Python从本地HTML提取主流视频平台链接并导出JSON？

核心思路 #

优先采用DOM结构解析处理HTML内容，因为HTML标签嵌套复杂，正则容易出现漏匹配或误匹配；正则仅作为补充，提取DOM解析未覆盖到的零散链接。最终将去重后的视频链接保存为JSON文件，供youtube-dl批量调用。

一、基于DOM结构解析的方案 #

Python中常用的HTML解析工具：

• BeautifulSoup：API友好，能兼容不规范的HTML代码，上手成本低
• lxml：解析速度快，支持XPath查询，适合处理大型HTML文件

具体实现步骤 #

1. 遍历本地目标目录下的所有HTML文件
2. 用解析器加载HTML内容，构建DOM树
3. 针对不同平台的嵌入规则提取链接：
   • 提取<video>标签的src属性
   • 提取iframe的src属性（多数平台采用iframe嵌入视频）
   • 提取平台专属标签的核心属性（比如YouTube的<youtube-player>标签的video-id），再拼接成标准可识别链接

代码示例 #

import os
import json
from bs4 import BeautifulSoup

# 定义各平台的DOM提取规则
PLATFORM_RULES = {
    "youtube": {
        "selectors": ["iframe[src*='youtube.com/embed']", "youtube-player[video-id]"],
        "processor": lambda elem: (
            elem["src"] if "src" in elem.attrs 
            else f"https://www.youtube.com/watch?v={elem['video-id']}"
        )
    },
    "rumble": {
        "selectors": ["iframe[src*='rumble.com/embed']"],
        "processor": lambda elem: elem["src"].split("?")[0]
    },
    "bitchute": {
        "selectors": ["iframe[src*='bitchute.com/embed']", "video[src*='bitchute.com']"],
        "processor": lambda elem: elem["src"]
    },
    "brighteon": {
        "selectors": ["iframe[src*='brighteon.com/embed']", "video[src*='brighteon.com']"],
        "processor": lambda elem: elem["src"]
    },
    "odysee": {
        "selectors": ["iframe[src*='odysee.com/$']", "video[src*='odysee.com']"],
        "processor": lambda elem: elem["src"]
    },
    "vimeo": {
        "selectors": ["iframe[src*='player.vimeo.com/video']"],
        "processor": lambda elem: f"https://vimeo.com/{elem['src'].split('/')[-1].split('?')[0]}"
    },
    "dailymotion": {
        "selectors": ["iframe[src*='dailymotion.com/embed/video']"],
        "processor": lambda elem: f"https://www.dailymotion.com/video/{elem['src'].split('/')[-1].split('?')[0]}"
    }
}

def extract_video_links(html_path):
    links = set()  # 用集合自动去重
    with open(html_path, "r", encoding="utf-8", errors="ignore") as f:
        soup = BeautifulSoup(f.read(), "html.parser")
    
    for platform, rule in PLATFORM_RULES.items():
        for selector in rule["selectors"]:
            elements = soup.select(selector)
            for elem in elements:
                try:
                    link = rule["processor"](elem)
                    links.add(link)
                except KeyError:
                    continue
    return list(links)

def main(html_dir, output_json):
    all_links = set()
    # 遍历目录下所有HTML/HTM文件
    for root, _, files in os.walk(html_dir):
        for file in files:
            if file.endswith((".html", ".htm")):
                file_path = os.path.join(root, file)
                links = extract_video_links(file_path)
                all_links.update(links)
    
    # 保存为JSON文件
    with open(output_json, "w", encoding="utf-8") as f:
        json.dump(list(all_links), f, indent=2)
    print(f"共提取到{len(all_links)}个视频链接，已保存到{output_json}")

if __name__ == "__main__":
    # 替换为你的HTML文件目录和输出JSON路径
    main("./html_files", "./video_links.json")

二、正则表达式方案 #

适合提取HTML中零散存在、未被标签规范包裹的平台链接，作为DOM解析的补充手段。

各平台正则表达式 #

import re

PLATFORM_PATTERNS = [
    # YouTube：匹配watch、embed、share及短链格式
    r"https?://(www\.)?(youtube\.com/(watch\?v=|embed/|share\?v=)|youtu\.be/)[\w-]+",
    # Rumble
    r"https?://(www\.)?rumble\.com/(embed/|v/)[\w-]+",
    # Bitchute
    r"https?://(www\.)?bitchute\.com/(video/|embed/)[\w-]+",
    # Brighteon
    r"https?://(www\.)?brighteon\.com/(video/|embed/)[\w-]+",
    # Odysee
    r"https?://(www\.)?odysee\.com/[$\w-]+",
    # Vimeo
    r"https?://(www\.)?(vimeo\.com/|player\.vimeo\.com/video/)[\d]+",
    # Dailymotion
    r"https?://(www\.)?(dailymotion\.com/video/|dailymotion\.com/embed/video/)[\w-]+"
]

def extract_links_with_regex(html_content):
    links = set()
    for pattern in PLATFORM_PATTERNS:
        matches = re.findall(pattern, html_content, re.IGNORECASE)
        for match in matches:
            # 拼接完整链接，过滤空分组内容
            full_link = "".join(filter(None, match))
            if not full_link.startswith("http"):
                full_link = f"https://{full_link}"
            links.add(full_link)
    return list(links)

整合到DOM解析流程 #

在extract_video_links函数中加入正则提取逻辑，实现双重覆盖：

def extract_video_links(html_path):
    links = set()
    with open(html_path, "r", encoding="utf-8", errors="ignore") as f:
        content = f.read()
        soup = BeautifulSoup(content, "html.parser")
    
    # DOM解析提取链接
    for platform, rule in PLATFORM_RULES.items():
        for selector in rule["selectors"]:
            elements = soup.select(selector)
            for elem in elements:
                try:
                    link = rule["processor"](elem)
                    links.add(link)
                except KeyError:
                    continue
    
    # 正则补充提取零散链接
    regex_links = extract_links_with_regex(content)
    links.update(regex_links)
    return list(links)

三、注意事项 #

1. 编码兼容：读取HTML文件时添加errors="ignore"，避免因编码问题中断程序
2. 链接去重：始终用集合存储链接，自动过滤重复项
3. youtube-dl兼容性：优先提取平台的标准播放页链接（比如YouTube的watch?v=格式），比embed链接兼容性更好
4. 规则扩展：如果遇到新的嵌入方式，只需在PLATFORM_RULES或PLATFORM_PATTERNS中添加对应规则即可

内容的提问来源于stack exchange，提问作者Galaxy