使用Tweepy时连接错误处理失效，求助解决脚本崩溃问题

使用Tweepy库编写Python脚本，通过无限循环长时间持续抓取推文，但每当出现连接失败等问题时，脚本直接崩溃，需手动重启。尝试用try...except语句增强鲁棒性，甚至捕获tweepy.TweepyException也无效，模拟断网测试时脚本依然崩溃。

示例代码：

consumer_key = ""
consumer_secret = ""
access_token = ""
access_token_secret = ""

auth = tweepy.OAuth1UserHandler(
   consumer_key, consumer_secret,
   access_token, access_token_secret
)
api = tweepy.API(auth)

city = 'LX'
country = 'PT'
coord = '38.736946,-9.142685'
radius = '25km'

while True:
    today = datetime.now()

    if len(str(today.month)) == 1:
        month = str(0) + str(today.month)
    else:
        month = str(today.month)    
    n=10
    tweets=tweepy.Cursor(api.search_tweets,q="",geocode=coord+','+radius,tweet_mode="extended").items(n)
 
    fname = f'{country}_{city}_{today.day}{month}{str(today.year)[2:]}.json'

    with open(fname, 'a') as outfile:
        for tweet in tweets:
            try: 
                outfile.write(json.dumps(tweet._json))
                outfile.write('\n')
            except:
                os.system(f'echo error >> log_file.log')
                print('error')
                pass
        outfile.close()
    time.sleep(10)

（注：示例代码可能存在拼写错误）

你的核心问题是异常捕获范围过小——当前的try...except仅包裹了文件写入逻辑，而连接失败、API请求错误等异常发生在tweepy.Cursor初始化和遍历推文的阶段，完全没被捕获到。

以下是修复后的代码，同时增加了重试机制和更完善的日志记录：

import tweepy
import datetime
import json
import os
import time
from requests.exceptions import RequestException

consumer_key = ""
consumer_secret = ""
access_token = ""
access_token_secret = ""

auth = tweepy.OAuth1UserHandler(
    consumer_key, consumer_secret,
    access_token, access_token_secret
)
api = tweepy.API(auth, wait_on_rate_limit=True)  # 开启自动处理限流

city = 'LX'
country = 'PT'
coord = '38.736946,-9.142685'
radius = '25km'

while True:
    try:
        today = datetime.datetime.now()
        
        # 简化月份格式化逻辑
        month = f"{today.month:02d}"
        n = 10
        
        # 初始化Cursor并请求推文
        tweets = tweepy.Cursor(
            api.search_tweets,
            q="",
            geocode=f"{coord},{radius}",
            tweet_mode="extended"
        ).items(n)
        
        fname = f'{country}_{city}_{today.day}{month}{str(today.year)[2:]}.json'
        
        with open(fname, 'a') as outfile:
            for tweet in tweets:
                try:
                    outfile.write(json.dumps(tweet._json))
                    outfile.write('\n')
                except Exception as e:
                    # 记录具体写入异常
                    error_msg = f"[{datetime.datetime.now()}] 写入失败: {str(e)}"
                    os.system(f'echo "{error_msg}" >> log_file.log')
                    print(error_msg)
                    pass
        
        time.sleep(10)
    
    except tweepy.TweepyException as e:
        # 捕获Tweepy相关异常（限流、API错误等）
        error_msg = f"[{datetime.datetime.now()}] Tweepy异常: {str(e)}"
        os.system(f'echo "{error_msg}" >> log_file.log')
        print(error_msg)
        time.sleep(60)  # 异常后延迟60秒再重试
    
    except RequestException as e:
        # 捕获网络类异常（断网、超时等）
        error_msg = f"[{datetime.datetime.now()}] 网络连接异常: {str(e)}"
        os.system(f'echo "{error_msg}" >> log_file.log')
        print(error_msg)
        time.sleep(120)  # 断网后延迟更久再重试
    
    except Exception as e:
        # 兜底捕获其他未知异常
        error_msg = f"[{datetime.datetime.now()}] 未知异常: {str(e)}"
        os.system(f'echo "{error_msg}" >> log_file.log')
        print(error_msg)
        time.sleep(60)

关键修复点： #

• 扩大异常捕获范围：将整个API请求、推文遍历、文件写入的逻辑都放入外层try块，确保所有可能的异常都能被捕获
• 明确异常类型：分别捕获Tweepy自身异常、网络请求异常，以及兜底的通用异常，便于针对性处理
• 增强限流处理：初始化API时添加wait_on_rate_limit=True，Tweepy会自动处理API限流等待
• 完善日志记录：记录异常发生时间和具体信息，方便排查问题
• 优化重试延迟：针对不同异常设置不同的重试间隔，避免无效频繁请求
• 简化代码：用f"{today.month:02d}"简化月份格式化，移除多余的outfile.close()（with语句会自动关闭文件）

内容的提问来源于stack exchange，提问作者Filipa