如何使用Python Backoff库根据HTTP 408错误码实现重试？

针对你的需求，结合backoff库，提供两种可行的实现方式，你可以根据代码习惯选择：

方案一：扩展@backoff.on_exception处理HTTP 408异常 #

requests的response.raise_for_status()方法会将4xx/5xx状态码转为requests.exceptions.HTTPError，我们可以把该异常加入重试列表，再通过giveup参数过滤出仅408的情况，其他HTTP错误不触发重试。

修改后的完整代码 #

def should_giveup(exc):
    # 仅当HTTPError的状态码不是408时，放弃重试
    if isinstance(exc, requests.exceptions.HTTPError):
        return exc.response.status_code != 408
    # 超时类异常继续重试
    return False

@backoff.on_exception(
    backoff.expo,
    (requests.exceptions.ReadTimeout, requests.exceptions.ConnectTimeout, requests.exceptions.HTTPError),
    max_tries=MAX_TIMEOUT_RETRY_COUNT,
    giveup=should_giveup
)
def get_response_http_post(data, params, headers, url) -> dict:
    """ Returns the response as json for a http post request """
    response_dict = {}
    try:
        response = requests.get(url, data=data, params=params, headers=headers, timeout=REQUEST_TIMEOUT)
        response.raise_for_status()  # 自动将408转为HTTPError
        if response.text:
            response_dict = json.loads(response.text)
        response_dict['status_code'] = response.status_code
        if response.reason:
            response_dict['reason'] = response.reason
        return response_dict
    except requests.exceptions.ReadTimeout as err:
        logger.exception(f'ReadTimeout: {err}')
        raise
    except requests.exceptions.ConnectTimeout as err:
        logger.exception(f'ConnectTimeout: {err}')
        raise
    except requests.exceptions.ConnectionError as err:
        logger.exception(f'ConnectionError: {err}')
        raise
    except requests.exceptions.HTTPError as err:
        logger.exception(f'HTTP Error: {err}')
        raise
    except requests.exceptions.RequestException as err:
        logger.exception(f'Exception occurred while sending http request: {err}')
        raise

关键说明 #

• 把requests.exceptions.HTTPError加入重试异常列表，配合giveup函数精准控制仅408触发重试
• 原代码中raise requests.exceptions.ReadTimeout会丢失异常上下文，改为直接raise保留完整异常信息

方案二：叠加@backoff.on_predicate基于返回值重试 #

如果希望通过函数返回的status_code触发重试，可以叠加@backoff.on_predicate装饰器，检查返回字典中的状态码是否为408。

修改后的完整代码 #

@backoff.on_predicate(
    backoff.expo,
    lambda resp: resp.get('status_code') == 408,
    max_tries=MAX_TIMEOUT_RETRY_COUNT
)
@backoff.on_exception(
    backoff.expo,
    (requests.exceptions.ReadTimeout, requests.exceptions.ConnectTimeout),
    max_tries=MAX_TIMEOUT_RETRY_COUNT
)
def get_response_http_post(data, params, headers, url) -> dict:
    """ Returns the response as json for a http post request """
    response_dict = {}
    try:
        response = requests.get(url, data=data, params=params, headers=headers, timeout=REQUEST_TIMEOUT)
        # 注意：这里不能调用response.raise_for_status()，否则408会转为异常进入分支，无法返回带408的字典
        if response.text:
            response_dict = json.loads(response.text)
        response_dict['status_code'] = response.status_code
        if response.reason:
            response_dict['reason'] = response.reason
        if response.status_code == 408:
            logger.warning(f'Received HTTP 408 Timeout for URL: {url}')
        return response_dict
    except requests.exceptions.ReadTimeout as err:
        logger.exception(f'ReadTimeout: {err}')
        raise
    except requests.exceptions.ConnectTimeout as err:
        logger.exception(f'ConnectTimeout: {err}')
        raise
    except requests.exceptions.ConnectionError as err:
        logger.exception(f'ConnectionError: {err}')
        raise
    except requests.exceptions.RequestException as err:
        logger.exception(f'Exception occurred while sending http request: {err}')
        raise

关键说明 #

• @backoff.on_predicate会检查函数返回值，当返回的字典中status_code为408时触发重试
• 必须去掉response.raise_for_status()，否则408会被转为异常，无法进入返回值判断逻辑

推荐选择 #

优先使用方案一，它更贴合requests的异常处理逻辑，能统一管理HTTP错误，同时保留完整的异常上下文，代码结构更清晰。

内容的提问来源于stack exchange，提问作者Kulasangar