You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

PostgreSQL插入好友表触发唯一约束重复键错误求助

问题描述

我是SQL与PostgreSQL新手,若问题较为基础还请见谅。

我创建了两张表:

CREATE TABLE    usertable (  
user_id VARCHAR PRIMARY KEY,
...);

CREATE TABLE hasFriend(
userID          VARCHAR,
friendID            VARCHAR ,
PRIMARY KEY (userID,friendID),
FOREIGN KEY (userID) REFERENCES usertable(user_id),
FOREIGN KEY (friendID) REFERENCES usertable(user_id));

我正通过Python脚本解析大型JSON文件,向usertable和hasFriend表填充数据。用户表的解析与插入功能正常完成并关闭了JSON文件,之后调用好友表插入函数,重新从JSON文件开头读取数据,函数代码如下:

def insert2FriendTable():
    with open('yelp_user.JSON','r') as f:    
        line = f.readline()
        count_line = 0

        try:
             #edited out the connect statement since it has personal info on it.
            conn = psycopg2.connect(...)
        except:
            print('Unable to connect to the database!')
        cur = conn.cursor()
        while line:
            data = json.loads(line)
            for friend in data['friends']:
                sql_str="INSERT INTO hasFriend (userID, friendID) VALUES ('" + data['user_id']  + "','" + friend + "');"
                try:
                    cur.execute(sql_str)
                except Exception as error:
                    print("Insert to hasFriend failed!", error)
                    return

运行时触发错误:

Insert to hasFriend failed! duplicate key value violates unique constraint "hasfriend_pkey"
DETAIL:  Key (userid, friendid)=(om5ZiponkpRqUNa3pVPiRg, U_sn0B-HWdTSlHNXIl_4XA) already exists.

JSON文件首行示例:

{"average_stars": 3.94, "compliment_cool": 1556, "compliment_cute": 211, "compliment_funny": 1556, "compliment_hot": 1285, "compliment_list": 101, "compliment_more": 134, "compliment_note": 1295, "compliment_photos": 162, "compliment_plain": 2134, "compliment_profile": 74, "compliment_writer": 402, "cool": 40110, "elite": [2014, 2017, 2011, 2012, 2015, 2009, 2013, 2007, 2016, 2006, 2010, 2008], "fans": 835, "friends": ["U_sn0B-HWdTSlHNXIl_4XA", "pnfVIB7UhvCQ7X2K0Q2XIw", "jVYzrVblDFSuL3GHtt8ZSA", "Z7bpqY89ZiBHXdo7UN1kiw", "8Aqr35f254lOeitNowt7ig", "zjcN27kCVeK8K2ONe9Qt4g", "8drMKNHWavs2g6uf0pLtvg", "_K2ViyfmVq6nzIitR0TIlg", "rUV1FUhji5xMjNBpcq5SXg", "yrGIgk5eaWy-eewLNv4KHQ", "3Vd_ATdvvuVVgn_YCpz8fw", "ebC_pH92K4uxyDenoXb5bg", "RJrGgtBXkpX2oEHM4hSqXg", "sdpIz4-s15T239CZ4Bd6Ag", "A0j21z2Q1HGic7jW6e9h7A", "AvC5XQAElcGAAn_Wr5auEg", "JlkHKBnHKdK8Tpls0AF5Aw", "8AG5MctcxTjP4svmUrt0yQ", "bKxdvn7KpmWjMzlmBvp-Xw", "VVMS74JyUk2h53yfC-xNsA", "-ro7OG3jjCSKnF6OJinKjg", "K7thO1n-vZ9PFYiC7nTR2w", "pRBzWnFzaCEtqhYyJ2ZTDQ", "nwESZ8e-KzXt2fKkOuRdIQ", "WNZfkL4DBspueoGSUOMAqA", "uU6fQWadr7Hx_MP0Vmy3kQ", "XiLxIJThWsE0x4d0IeSPsg", "Y9LBTbwO4g0BmdBIi0D3CA", "jGbj8fl575EIQJcfaA1FKQ", "nxWrhF_hyX0wwjrEkQX8uQ", "670k6Gr6V4VqLIKtVEmDuQ", "o5STsEtfvD1Ig0J7Z-1uxA", "x13yoEggBL0pIE7-KMnhDQ", "rCx7tb3toOJUsvdOeqYY0g", "nkN_do3fJ9xekchVC-v68A", "KzHRsFwryS7b5Fog8kkkGA", "_pBzBgtCTN9PNUPfgPDI8A", "PU5QaMADa6N_9ZoQ04ZjOw", "FkfpHzqoDRChwOYhA6NPnQ", "CaQy-zz10ajG7KkNSbXi5w", "brQ7OjB6f9nXWGk45A9A3g"], "funny": 10882, "name": "Andrea", "review_count": 2559, "useful": 83681, "user_id": "om5ZiponkpRqUNa3pVPiRg", "yelping_since": "2006-01-18"}

我完全不清楚该如何解决此问题,恳请各位提供建议,谢谢!

解决方案

1. 跳过重复记录:使用INSERT ... ON CONFLICT DO NOTHING

PostgreSQL原生支持冲突处理语法,当遇到唯一键重复时直接跳过该条插入,不会中断整个流程。同时改用参数化查询,避免SQL注入风险,还能兼容含特殊字符的ID:

sql_str = "INSERT INTO hasFriend (userID, friendID) VALUES (%s, %s) ON CONFLICT DO NOTHING;"
cur.execute(sql_str, (data['user_id'], friend))

2. 清理已有重复数据(可选)

如果之前已经执行过插入操作,表中已有部分数据,可先清空表再重新插入(注意:此操作会删除所有现有数据):

TRUNCATE TABLE hasFriend;

3. 修复脚本中的其他问题

  • 缺少事务提交:执行插入后需要调用conn.commit()才能将数据持久化到数据库
  • 异常处理直接终止流程:把return改为continue,跳过错误继续处理后续数据
  • 无限循环问题:循环末尾需添加line = f.readline()读取下一行数据

修改后的完整函数示例:

def insert2FriendTable():
    with open('yelp_user.JSON','r') as f:    
        line = f.readline()

        try:
            conn = psycopg2.connect(...)
        except Exception as e:
            print('Unable to connect to the database!', e)
            return
        cur = conn.cursor()
        
        while line:
            try:
                data = json.loads(line)
                for friend in data['friends']:
                    sql_str = "INSERT INTO hasFriend (userID, friendID) VALUES (%s, %s) ON CONFLICT DO NOTHING;"
                    cur.execute(sql_str, (data['user_id'], friend))
                conn.commit()
            except Exception as error:
                print("Insert to hasFriend failed!", error)
                conn.rollback()
            line = f.readline()
        
        cur.close()
        conn.close()

内容的提问来源于stack exchange,提问作者Ely Meyers

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.25 00:18:22