如何通过追加异构列值创建指定结构的DataFrame？

需要构建一个两列多行的DataFrame，结构示例如下：

[
 ['text1',[float1, float2, float3]]
 ['text2',[float4, float5, float6]]
 ...
]

其中列名content对应文本内容，embeddings对应浮点数组。

原代码及问题 #

原实现代码：

mycontent = ["i live in space","i live my life to fullest", "dogs live in kennel","we live to eat and not eat to live","cricket lives in heart of every indian","live and let live","my house is in someplace","my office is in someotherplace","chair is red"]

contents_and_embeddings_df = pd.DataFrame(columns=['content','embeddings'])

for content in mycontent:
    embedding = get_embedding(content,engine='textsearchcuriedoc001mc') # 返回浮点列表
    contents_and_embeddings_df.append(pd.DataFrame([content,embedding]))
   

contents_and_embeddings_df

运行后出现弃用警告：

contents_and_embeddings_df.append(pd.DataFrame([content,embedding])) /tmp/ipykernel_15879/3971327095.py:8: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. contents_and_embeddings_df.append(pd.DataFrame([content,embedding]))

且最终DataFrame仅显示表头，无任何数据。后续尝试其他写法也未成功：

for content in mycontent:
    embedding = get_embedding(content,engine='textsearchcuriedoc001mc')
    # pd.concat(contents_and_embeddings_df,pd.DataFrame([content,embedding])) --> 无效
    # contents_and_embeddings_df.append(pd.DataFrame([content,embedding])) --> 无效
    tempdf = pd.DataFrame([content,embedding]) # 无效
    # tempdf = pd.DataFrame([content,embedding], columns=['content','embeddings']) --> 编译失败
    contents_and_embeddings_df.add(tempdf) # 无效
 

contents_and_embeddings_df # 显示为空

错误原因 #

1. append方法的使用问题：append不会修改原DataFrame，而是返回新对象，原代码未接收返回值；且该方法已被弃用。
2. 临时DataFrame结构错误：pd.DataFrame([content,embedding])会生成两行一列的结构，而非目标的一行两列，无法匹配表头。
3. concat和add使用错误：concat需要传入DataFrame列表并重新赋值，add是数值运算方法，不适合添加行数据。

解决方案 #

方法1：提前收集数据，一次性创建DataFrame（推荐，效率更高）

避免循环中频繁操作DataFrame，先把所有数据存入列表，再统一构建：

import pandas as pd

mycontent = ["i live in space","i live my life to fullest", "dogs live in kennel","we live to eat and not eat to live","cricket lives in heart of every indian","live and let live","my house is in someplace","my office is in someotherplace","chair is red"]

data_list = []
for content in mycontent:
    embedding = get_embedding(content, engine='textsearchcuriedoc001mc')
    # 按列名组装每行数据
    data_list.append({'content': content, 'embeddings': embedding})

# 一次性生成目标DataFrame
contents_and_embeddings_df = pd.DataFrame(data_list)

方法2：使用pd.concat循环添加（不推荐，适合小数据量）

若必须循环添加行，需正确构造临时DataFrame并重新赋值：

import pandas as pd

mycontent = ["i live in space","i live my life to fullest", "dogs live in kennel","we live to eat and not eat to live","cricket lives in heart of every indian","live and let live","my house is in someplace","my office is in someotherplace","chair is red"]

contents_and_embeddings_df = pd.DataFrame(columns=['content','embeddings'])

for content in mycontent:
    embedding = get_embedding(content, engine='textsearchcuriedoc001mc')
    # 构造一行两列的临时DataFrame，匹配表头
    temp_df = pd.DataFrame([[content, embedding]], columns=['content','embeddings'])
    # 用concat合并并重新赋值，ignore_index重置索引避免重复
    contents_and_embeddings_df = pd.concat([contents_and_embeddings_df, temp_df], ignore_index=True)

内容的提问来源于stack exchange，提问作者Manu Chadha