Python迭代生成嵌套字典时数据被覆盖问题求助
问题:CSV转JSON时所有图书的info/ratings内容被最后一条覆盖
我正尝试将符合条件的图书数据从CSV文件写入JSON文件,目标结构是顶级键为bookID、值为包含info和ratings嵌套字典的结构。遍历字典列表jsonArray时,发现顶级的bookID能正确切换,但每个bookID对应的info和ratings内容始终被数据集的最后一条条目覆盖。我确认循环中info_dict和rating_dict能生成单条图书的正确数据,但它们在json_dict中被后续迭代的内容覆盖。bookID是唯一的,不清楚如何避免这种覆盖问题。
当前代码
def retrieve_popular_books(csv_in, json_out): jsonArray = [] json_dict = {} info_list = [] rating_list = [] book_id = [] info_dict = {} rating_dict = {} with open(csv_in, newline='') as csv_file: csv_dict_reader = csv.DictReader(csv_file) for row in csv_dict_reader: average_rating = float(row["average_rating"]) num_pages = int(row["num_pages"]) ratings_count = int(row["ratings_count"]) if average_rating > 4.50 and num_pages > 50 and ratings_count > 1000: jsonArray.append(row) for i in jsonArray: book_id = i['bookID'] json_dict[book_id] = {} for k,v in i.items(): if k != "bookID" and k!= "average_rating" and k!="ratings_count" and k!= "text_reviews_count": info_list.append({k:v}) if k== "average_rating" or k=="ratings_count" or k== "text_reviews_count": rating_list.append({k:v}) for i in info_list: for k,v in i.items(): info_dict[k] = v json_dict[book_id]["info"] = info_dict for i in rating_list: for k,v in i.items(): rating_dict[k] = v json_dict[book_id]["ratings"] = rating_dict with open(json_out, 'w') as jsonf: json.dump(json_dict, jsonf, indent=4) retrieve_popular_books('books_data/books_v2.csv', 'books_data/books_v2_copy.json')
当前错误结果(部分)
{ "863f3fd8-b4f5-43b8-8ec1-eb870faa334e": { "info": { "title": "Fullmetal Alchemist Vol. 6 (Fullmetal Alchemist #6)", "authors": "Hiromu Arakawa/Akira Watanabe", "isbn": "1421503190", "isbn13": "9781421503196", "language_code": "eng", "num_pages": "200", "publication_date": "3/21/2006", "publisher": "VIZ Media LLC" }, "ratings": { "average_rating": "4.58", "ratings_count": "10052", "text_reviews_count": "201" } }, "b343cec8-66dc-470a-8efe-47858e37aab5": { "info": { "title": "Fullmetal Alchemist Vol. 6 (Fullmetal Alchemist #6)", "authors": "Hiromu Arakawa/Akira Watanabe", "isbn": "1421503190", "isbn13": "9781421503196", "language_code": "eng", "num_pages": "200", "publication_date": "3/21/2006", "publisher": "VIZ Media LLC" }, "ratings": { "average_rating": "4.58", "ratings_count": "10052", "text_reviews_count": "201" } } }
问题原因
核心问题是**info_dict、rating_dict、info_list、rating_list这些容器是在循环外部初始化的**:
- 每次处理新图书时,你只是往同一个
info_list/rating_list里追加数据,修改同一个info_dict/rating_dict的键值对 - Python中字典是引用类型,所有bookID对应的
"info"和"ratings"都指向同一个字典对象,后续修改会覆盖之前的内容,最终所有条目都显示最后一次循环的结果
修正方案
把字典和列表的初始化移到遍历单本图书的循环内部,确保每本书都有独立的容器实例,同时可以简化逻辑,直接构建目标字典:
def retrieve_popular_books(csv_in, json_out): json_dict = {} with open(csv_in, newline='') as csv_file: csv_dict_reader = csv.DictReader(csv_file) for row in csv_dict_reader: # 过滤条件 average_rating = float(row["average_rating"]) num_pages = int(row["num_pages"]) ratings_count = int(row["ratings_count"]) if average_rating > 4.50 and num_pages > 50 and ratings_count > 1000: book_id = row['bookID'] # 为当前图书创建独立的info和ratings字典 info_dict = {} rating_dict = {} # 拆分数据到对应字典 for k, v in row.items(): if k == "bookID": continue elif k in ["average_rating", "ratings_count", "text_reviews_count"]: rating_dict[k] = v else: info_dict[k] = v # 加入最终字典 json_dict[book_id] = { "info": info_dict, "ratings": rating_dict } with open(json_out, 'w') as jsonf: json.dump(json_dict, jsonf, indent=4) retrieve_popular_books('books_data/books_v2.csv', 'books_data/books_v2_copy.json')
修正说明
- 移除了冗余的
jsonArray、info_list、rating_list,直接在读取CSV行时处理,节省内存 - 每处理一本符合条件的图书,就新建
info_dict和rating_dict,确保每个bookID对应独立的字典实例 - 简化了键值对的拆分逻辑,直接通过判断键名分配到对应字典
- 避免了引用类型导致的覆盖问题,每个图书的info和ratings都是独立的对象
内容的提问来源于stack exchange,提问作者Nathan
相关产品推荐
相关产品推荐

