psycopg2.errors.UniqueViolation错误：try-except捕获位置咨询

报错信息（已翻译） #

DEBUG 2022-10-06 09:53:43,747 1 ui.rabbit 声明sfm_exchange交换机
DEBUG 2022-10-06 09:53:44,037 1 ui.config 为集合设置接收器。
DEBUG 2022-10-06 09:53:44,038 1 ui.config 为导出设置接收器。
DEBUG 2022-10-06 09:53:44,038 1 ui.config 未运行调度器
DEBUG 2022-10-06 09:53:44,038 1 ui.config 设置删除接收器
DEBUG 2022-10-06 09:53:44,038 1 ui.config 设置集合接收器
INFO 2022-10-06 09:53:45,955 1 message_consumer.sfm_ui_consumer 更新ID为44477b7d22164d78a4b6375ca924f06b的采集任务：执行成功完成
INFO 2022-10-06 09:55:42,729 1 message_consumer.sfm_ui_consumer 更新ID为793a8a9d9fda4655a738a16ecb4e8462的采集任务：运行中
DEBUG 2022-10-06 09:55:51,497 1 message_consumer.sfm_ui_consumer Warc文件ID为d7b068dc965f43bf915a79ae1f1158d3
INFO 2022-10-06 09:55:52,571 1 message_consumer.sfm_ui_consumer 更新ID为793a8a9d9fda4655a738a16ecb4e8462的采集任务：运行中
ERROR 2022-10-06 09:55:52,579 1 message_consumer.sfm_ui_consumer 重复键值违反唯一约束"ui_seed_collection_id_uid_token_744a7e90_uniq"
详细信息: 键(collection_id, uid, token)=(15, 34743251, SpaceX)已存在。
回溯（最近的调用最后）:
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: 重复键值违反唯一约束"ui_seed_collection_id_uid_token_744a7e90_uniq"
详细信息: 键(collection_id, uid, token)=(15, 34743251, SpaceX)已存在。


上述异常是以下异常的直接原因：

回溯（最近的调用最后）:
  文件"/opt/sfm-ui/sfm/message_consumer/sfm_ui_consumer.py", 第47行, in on_message
    self._on_harvest_status_message()
  文件"/opt/sfm-ui/sfm/message_consumer/sfm_ui_consumer.py", 第112行, in _on_harvest_status_message
    seed.save()
  文件"/opt/sfm-ui/sfm/ui/models.py", 第613行, in save
    return history_save(self, *args, **kw)
  文件"/opt/sfm-ui/sfm/ui/models.py", 第82行, in history_save
    return super(self.__class__, self).save(*args, **kw)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第743行, in save
    self.save_base(using=using, force_insert=force_insert,
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第780行, in save_base
    updated = self._save_table(
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第853行, in _save_table
    updated = self._do_update(base_qs, using, pk_val, values, update_fields,
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第903行, in _do_update
    return filtered._update(values) > 0
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/query.py", 第760行, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", 第1471行, in execute_sql
    cursor = super().execute_sql(result_type)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", 第1142行, in execute_sql
    cursor.execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第99行, in execute
    return super().execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第67行, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第76行, in _execute_with_wrappers
    return executor(sql, params, many, context)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/utils.py", 第89行, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: 重复键值违反唯一约束"ui_seed_collection_id_uid_token_744a7e90_uniq"
详细信息: 键(collection_id, uid, token)=(15, 34743251, SpaceX)已存在。

回溯（最近的调用最后）:
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: 重复键值违反唯一约束"ui_seed_collection_id_uid_token_744a7e90_uniq"
详细信息: 键(collection_id, uid, token)=(15, 34743251, SpaceX)已存在。


上述异常是以下异常的直接原因：

回溯（最近的调用最后）:
  文件"/usr/local/lib/python3.8/site-packages/kombu/messaging.py", 第624行, in _receive_callback
    return on_m(message) if on_m else self.receive(decoded, message)
  文件"/usr/local/lib/python3.8/site-packages/kombu/messaging.py", 第590行, in receive
    [callback(body, message) for callback in callbacks]
  文件"/usr/local/lib/python3.8/site-packages/kombu/messaging.py", 第590行, in <listcomp>
    [callback(body, message) for callback in callbacks]
  文件"/opt/sfm-utils/sfmutils/consumer.py", 第111行, in _callback
    self.on_message()
  文件"/opt/sfm-ui/sfm/message_consumer/sfm_ui_consumer.py", 第57行, in on_message
    raise e
  文件"/opt/sfm-ui/sfm/message_consumer/sfm_ui_consumer.py", 第47行, in on_message
    self._on_harvest_status_message()
  文件"/opt/sfm-ui/sfm/message_consumer/sfm_ui_consumer.py", 第112行, in _on_harvest_status_message
    seed.save()
  文件"/opt/sfm-ui/sfm/ui/models.py", 第613行, in save
    return history_save(self, *args, **kw)
  文件"/opt/sfm-ui/sfm/ui/models.py", 第82行, in history_save
    return super(self.__class__, self).save(*args, **kw)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第743行, in save
    self.save_base(using=using, force_insert=force_insert,
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第780行, in save_base
    updated = self._save_table(
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第853行, in _save_table
    updated = self._do_update(base_qs, using, pk_val, values, update_fields,
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/base.py", 第903行, in _do_update
    return filtered._update(values) > 0
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/query.py", 第760行, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", 第1471行, in execute_sql
    cursor = super().execute_sql(result_type)
  文件"/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", 第1142行, in execute_sql
    cursor.execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第99行, in execute
    return super().execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第67行, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第76行, in _execute_with_wrappers
    return executor(sql, params, many, context)
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
  文件"/usr/local/lib/python3.8/site-packages/django/db/utils.py", 第89行, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  文件"/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", 第84行, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: 重复键值违反唯一约束"ui_seed_collection_id_uid_token_744a7e90_uniq"
详细信息: 键(collection_id, uid, token)=(15, 34743251, SpaceX)已存在。


上述异常是以下异常的直接原因：

回溯（最近的调用最后）:
  文件"/opt/sfm-ui/sfm/manage.py", 第10行, in <module>
    execute_from_command_line(sys.argv)
  文件"/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", 第381行, in execute_from_command_line
    utility.execute()
  文件"/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", 第375行, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  文件"/usr/local/lib/python3.8/site-packages/django/core/management/base.py", 第323行, in run_from_argv
    self.execute(*args, **cmd_options)
  文件"/usr/local/lib/python3.8/site-packages/django/core/management/base.py", 第364行, in execute
    output = self.handle(*args, **options)
  文件"/opt/sfm-ui/sfm/message_consumer/management/commands/startconsumer.py", 第20行, in handle
    consumer.run()
  文件"/usr/local/lib/python3.8/site-packages/kombu/mixins.py", 第175行, in run
    for _ in self.consume(limit=None, **kwargs):
  文件"/usr/local/lib/python3.8/site-packages/kombu/mixins.py", 第197行, in consume
    conn.drain_events(timeout=safety_interval)
  文件"/usr/local/lib/python3.8/site-packages/librabbitmq/__init__.py", 第227行, in drain_events
    self._basic_recv(timeout)
SystemError: <method '_basic_recv' of '_librabbitmq.Connection' objects> 返回结果时设置了错误

问题分析 #

从报错栈可以明确两个核心问题：

1. 异常类型转换：底层的psycopg2.errors.UniqueViolation会被Django自动包装为django.db.utils.IntegrityError抛出，若你之前仅捕获psycopg2的异常，会导致捕获失败。
2. 错误被上层重新抛出：即使在seed.save()、models.save或history_save中捕获了异常，sfm_ui_consumer.py的on_message方法第57行的raise e会将未处理的异常再次抛出，最终导致错误暴露。

正确的捕获方案 #

方案1：在seed.save()调用处捕获（推荐） #

修改sfm_ui_consumer.py第112行附近的代码，同时捕获Django包装后的异常：

import django.db.utils
import logging

logger = logging.getLogger(__name__)

# ... 其他代码 ...

# 在调用seed.save()的位置添加try-except
try:
    seed.save()
except django.db.utils.IntegrityError as e:
    # 根据业务需求处理，比如记录日志、跳过重复数据等
    logger.error(f"唯一键冲突，无法保存Seed: {e}")
    # 若不需要向上抛出异常，此处无需写raise

方案2：在on_message方法中统一捕获 #

如果需要处理该方法内所有数据库异常，可以修改sfm_ui_consumer.py的on_message方法：

import django.db.utils
import logging

logger = logging.getLogger(__name__)

def on_message(self):
    try:
        self._on_harvest_status_message()
    except django.db.utils.IntegrityError as e:
        logger.error(f"处理采集状态消息时发生唯一键冲突: {e}")
        # 阻止异常继续向上抛出，避免consumer崩溃
    except Exception as e:
        # 处理其他未知异常，可选择重新抛出
        logger.error(f"处理消息时发生未知错误: {e}")
        raise

额外优化建议 #

除了捕获异常，还可以在保存前主动检查是否存在重复记录，从根源避免触发异常：

from ui.models import Seed

# 保存前检查唯一键组合是否已存在
if not Seed.objects.filter(
    collection_id=seed.collection_id,
    uid=seed.uid,
    token=seed.token
).exists():
    seed.save()
else:
    logger.info(f"Seed已存在，跳过保存: collection_id={seed.collection_id}, uid={seed.uid}, token={seed.token}")

内容的提问来源于stack exchange，提问作者Adhithya Kiran