如何基于catalog_uuid配对多语言CSV行并生成POST请求Payload

问题描述 #

现有包含英文（en）和阿拉伯文（ar）配对行的CSV文件，部分行仅含单一语言。需实现以下逻辑：

• 若同一catalog_uuid同时存在en和ar行，生成包含row（对应en行）和nextRow（对应ar行）的Payload
• 若仅存在en或ar行，生成仅含row的Payload
• 将生成的Payload发送至指定API接口

修正后的Python代码 #

import pandas as pd
import requests
import json
import ast

HEADERS = {"Accept": "*/*", "Content-Type": "application/json"}
API_URL = "http://crawlerapi.labeb.com/api/PCCrawler/Crawl?StoreId=6021"

# 读取CSV并预处理数据
df = pd.read_csv("carrefour-uae-items-final.csv").fillna("").astype(str)

# 按catalog_uuid分组处理
for catalog_uuid, group in df.groupby("catalog_uuid"):
    # 分别获取英文和阿拉伯文行
    en_row = group[group["lang"] == "en"].iloc[0].to_dict() if "en" in group["lang"].values else None
    ar_row = group[group["lang"] == "ar"].iloc[0].to_dict() if "ar" in group["lang"].values else None
    
    payload = {}
    
    # 构建Payload
    if en_row and ar_row:
        # 同时存在两种语言，添加row和nextRow
        payload["row"] = {
            key: en_row[key] for key in ["LabebStoreId", "catalog_uuid", "lang", "cat_0_name", 
                                       "cat_1_name", "cat_2_name", "cat_3_name", "catalogname", 
                                       "description", "properties", "price", "price_before_discount", 
                                       "externallink", "Rating", "delivery", "discount", "instock"]
        }
        payload["row"]["images"] = ast.literal_eval(en_row["encoded_images"])
        
        payload["nextRow"] = {
            key: ar_row[key] for key in ["LabebStoreId", "catalog_uuid", "lang", "cat_0_name", 
                                       "cat_1_name", "cat_2_name", "cat_3_name", "catalogname", 
                                       "description", "properties", "price", "price_before_discount", 
                                       "externallink", "Rating", "delivery", "discount", "instock"]
        }
        payload["nextRow"]["images"] = ast.literal_eval(ar_row["encoded_images"])
    else:
        # 仅存在一种语言，添加row
        single_row = en_row if en_row else ar_row
        payload["row"] = {
            key: single_row[key] for key in ["LabebStoreId", "catalog_uuid", "lang", "cat_0_name", 
                                       "cat_1_name", "cat_2_name", "cat_3_name", "catalogname", 
                                       "description", "properties", "price", "price_before_discount", 
                                       "externallink", "Rating", "delivery", "discount", "instock"]
        }
        payload["row"]["images"] = ast.literal_eval(single_row["encoded_images"])
    
    # 发送请求
    print(json.dumps(payload, indent=4, ensure_ascii=False))
    response = requests.post(API_URL, headers=HEADERS, json=payload)
    print(f"Catalog UUID {catalog_uuid} 响应: {response.content.decode()}")

关键修正说明 #

• 按catalog_uuid分组：避免重复处理同一商品的不同语言行，确保每个商品仅生成一次Payload并发送一次请求
• 准确识别语言行：从分组中分别提取en和ar行，判断是否同时存在两种语言
• 优化Payload结构：双语言场景下将en行放入row、ar行放入nextRow；单语言场景仅保留row
• 简化代码逻辑：通过字典推导式提取字段，减少冗余代码

内容的提问来源于stack exchange，提问作者X-something