Python运行Quizlet爬虫脚本报JSONDecodeError错误该怎么修复？

错误原因 #

json.decoder.JSONDecodeError: Extra data报错由页面结构更新导致，原代码的两处硬编码逻辑已经失效：

• 硬编码取script标签的索引[-6]：Quizlet官网迭代后页面script标签的排序发生变化，取到的标签已经不包含目标闪卡数据
• 硬编码字符串切片[44:-152]：目标script标签的内容格式发生变化，切片后得到的字符串不是合法JSON格式，存在多余字符导致解析失败

修复方案 #

替换原代码中QuizletParser函数里的JSON提取逻辑，改为动态定位目标标签、自动提取JSON内容，无需依赖硬编码规则：

def QuizletParser(link: str):
    session = HTMLSession()
    resp = session.get(link)
    soup = BeautifulSoup(resp.content, features="lxml")
    # 遍历所有script标签定位包含闪卡数据的目标标签
    target_script = None
    for script in soup.find_all('script'):
        script_content = script.string
        if not script_content:
            continue
        # 用闪卡数据特有字段termIdToTermsMap匹配目标标签
        if 'termIdToTermsMap' in script_content:
            target_script = script_content
            break
    if not target_script:
        raise Exception("页面未找到闪卡数据，官网结构可能已再次更新")
    # 提取JSON内容：定位第一个{到最后一个}的区间，避开前面的变量声明和后面的多余代码
    json_start = target_script.find('{')
    json_end = target_script.rfind('}')
    json_str = target_script[json_start:json_end+1]
    data = json.loads(json_str)
    # 以下原有逻辑保持不变
    flashcards = []
    for i in list(data['termIdToTermsMap'].values()):
        i = {
            'index': i['rank'],
            'id': i['id'],
            'term': i['word'],
            'definition': i['definition'],
            'setId': i['setId'],
            'image': i['_imageUrl'],
            'termTts': 'https://quizlet.com'+i['_wordTtsUrl'],
            'termTtsSlow': 'https://quizlet.com'+i['_wordSlowTtsUrl'],
            'definitionTts': 'https://quizlet.com'+i['_definitionTtsUrl'],
            'definitionTtsSlow': 'https://quizlet.com'+i['_definitionSlowTtsUrl'],
            'lastModified': i['lastModified'],
        }
        flashcards.append(i)

    output = {
        'title': data['set']['title'],
        'flashcards': flashcards,
        'author': {
            'name': data['creator']['username'],
            'id': data['creator']['id'],
            'timestamp': data['creator']['timestamp'],
            'lastModified': data['creator']['lastModified'],
            'image': data['creator']['_imageUrl'],
            'timezone': data['creator']['timeZone'],
            'isAdmin': data['creator']['isAdmin'],
        },
        'id': data['set']['id'],
        'link': data['set']['_webUrl'],
        'thumbnail': data['set']['_thumbnailUrl'],

        'timestamp': data['set']['timestamp'],
        'lastModified': data['set']['lastModified'],
        'publishedTimestamp': data['set']['publishedTimestamp'],
        'authorsId': data['set']['creatorId'],
        'termLanguage': data['set']['wordLang'],
        'definitionLanguage': data['set']['defLang'],
        'description': data['set']['description'],
        'numTerms': data['set']['numTerms'],
        'hasImages': data['set']['hasImages'],
        'hasUploadedImage': data['hasUploadedImage'],
        'hasDiagrams': data['set']['hasDiagrams'],
        'hasImages': data['set']['hasImages'],

    }
    return Box(output)

内容的提问来源于stack exchange，提问作者John Anthony