如何从网页Script标签的JS变量中提取数据（非Selenium方案）

我完全懂你的困扰——用BeautifulSoup拿到了Script标签内容，但eval()要么因为缩进、冗余代码报错，还存在安全风险。这里给你一个更靠谱的Python方案，用正则定位目标内容，再用JSON解析提取数据，完美避开这些问题。

核心思路 #

你的目标是提取userhome()函数里的JSON对象（也就是最后一行的{"items": [...]}），具体步骤很清晰：

1. 从Script内容中精准匹配出userhome()的参数部分（就是大括号包裹的JSON数据）
2. 把匹配到的字符串用Python标准库的json模块解析成字典
3. 直接提取items字段就能拿到你要的数据

完整代码示例 #

import re
import json

# 假设这是你通过BeautifulSoup获取到的Script标签内容
script_content = """barLoadGoogleFont('Open Sans'); barCssLoad('/global/pics/js/jquery/royalSlider/skins/universal/rs-universal.css?v=e449c4'); barCssLoad('/global/pics/css/material-icons.css?v=e6d856'); barCssLoad('/user/pics/css/user.css?v=eced9d'); barCssLoad('/user/pics/css/userIcons.css?v=6f9a03'); barCssLoad('/timeline/pics/css/timeline.css?v=8ec2ca'); barJsLibraryLoad('/global/pics/js/jquery/jquery.royalslider.min.js?v=515a43'); barJsLibraryLoad('/anketa/pics/js/utilsAnketa.js?v=9383d5'); barJsLibraryLoad('/znamky/pics/js/utilsZnamky.js?v=7afc9e'); barJsLibraryLoad('/exam/pics/js/utilsExam.js?v=033d55'); barJsLibraryLoad('/timeline/pics/js/utilsTimeline.js?v=29cf0e'); barJsLibraryLoad('/timeline/pics/js/timelineItemCreator.js?v=c37c99'); barJsLibraryLoad('/timeline/pics/js/timelineInputbox.js?v=2fde70'); barJsLibraryLoad('/timeline/pics/js/timelineViewer.js?v=f35e45'); barJsLibraryLoad('/user/pics/js/DailyPlan.js?v=e81fb9'); barJsLibraryLoad('/user/pics/js/userHomeEtest.js?v=6166f3'); $j(document).ready(function() { $j('#jwbcddd3da_md').userhome({"items":[{"timelineid":"2140963","timestamp":"2020-12-09 09:59:13","reakcia_na":"692638","typ":"h_clearplany","user":"Plan5077","target_user":null,"user_meno":"Kvarta aj2","ineid":"clearplany","text":"","cas_pridania":"2020-12-09 09:59:13","cas_udalosti":null,"data":"null","vlastnik":"Ucitel8678605","vlastnik_meno":"Barbora Drugajová","pocet_reakcii":"0","posledna_reakcia":"","pomocny_zaznam":"1","removed":"0","cas_pridania_btc":"2020-12-09 09:59:13","posledna_reakcia_btc":null},{"timelineid":"2287814","timestamp":"2020-12-09 09:59:12","reakcia_na":"2290613","typ":"h_dailyplan","user":"Trieda8694210","target_user":null,"user_meno":"Kvarta A","ineid":"daily2020-12-09","text":"","cas_pridania":"2020-12-09 09:59:12","cas_udalosti":null,"data":"[]","vlastnik":"Ucitel8678605","vlastnik_meno":"Barbora Drugajová","pocet_reakcii":"0","posledna_reakcia":"","pomocny_zaznam":"1","removed":"0","cas_pridania_btc":"2020-12-09 09:59:12","posledna_reakcia_btc":null},{"timelineid":"1439827","timestamp":"2020-12-09 08:56:57","reakcia_na":null,"typ":"h_clearplany","user":"*","target_user":null,"user_meno":"Celá škola","ineid":"clearplany","text":"","cas_pridania":"2020-12-09 08:56:57","cas_udalosti":null,"data":"null","vlastnik":"Ucitel16434","vlastnik_meno":"Ivor Dian","pocet_reakcii":"0","posledna_reakcia":"","pomocny_zaznam":null,"removed":"0","cas_pridania_btc":"2020-12-09 08:56:57","posledna_reakcia_btc":null},{"timelineid":"2290324","timestamp":"2020-12-09 08:37:22","reakcia_na":null,"typ":"sprava","user":"CustPlan5075","target_user":null,"user_meno":"Kvarta A+Kvarta B - nj4 · nemecký jazyk","ineid":null,"text":"Ahojte, zajtra..."}]})});"""

# 1. 匹配userhome函数中的JSON参数
# re.DOTALL让.匹配换行符，确保能捕获多行的JSON内容
match_result = re.search(r'userhome\((\{.*?\})\)', script_content, re.DOTALL)

if match_result:
    # 2. 提取JSON字符串
    json_data_str = match_result.group(1)
    
    try:
        # 3. 解析JSON为Python字典
        parsed_data = json.loads(json_data_str)
        # 4. 提取目标items数组
        target_items = parsed_data.get('items', [])
        
        # 输出结果示例
        print("成功提取到items数据：")
        for item in target_items:
            print(f"时间线ID: {item['timelineid']}, 用户名称: {item['user_meno']}")
            
    except json.JSONDecodeError as e:
        print(f"JSON解析失败，错误信息: {e}")
else:
    print("未找到userhome函数的参数内容")

为什么这个方案更优？ #

• 安全：彻底避开eval()执行未知代码的安全隐患
• 稳定：正则匹配精准定位目标内容，不受其他冗余函数、格式缩进的影响
• 轻量：只用到Python标准库，不需要额外安装任何依赖

备选方案（处理复杂JS场景） #

如果遇到JS里有非JSON格式的内容（比如内嵌函数、特殊语法），可以用js2py库直接执行JS代码提取数据：

import re
import js2py

# 同上的script_content
# 修改JS代码，把userhome的参数赋值给全局变量data
modified_js = re.sub(r'userhome\((\{.*?\})\)', r'window.target_data = \1', script_content, flags=re.DOTALL)

# 执行JS代码
js_context = js2py.EvalJs()
js_context.execute(modified_js)

# 获取数据
extracted_data = js_context.target_data
target_items = extracted_data.get('items', [])
print(target_items)

需要先安装依赖：pip install js2py

内容的提问来源于stack exchange，提问作者Tomáš Čikovský