如何用Python的pysimilar或DamerauLevenshtein批量对比多文本相似度

报错原因 #

pysimilar库的compare()方法仅支持单次传入两个单字符串或文件路径进行对比，你直接传入pandas的Series类型（DataFrame的整列数据）不符合参数类型要求，因此触发TypeError。

解决方法 #

场景1：批量对比DataFrame两列的逐行文本相似度 #

对DataFrame按行遍历，每次取出单行的两个字符串传入compare()方法即可，修改后代码如下：

from summarizers import Summarizers 
import pandas as pd
from pysimilar import compare

summ = Summarizers() 
data = ["The NN-CS89L offers next-level cooking convenience. Its four distinct cooking methods - steaming, baking, grilling and microwaving ensure your meals are cooked or reheated to perfection. Its multi-function capabilities can be combined to save time without compromising taste, texture or nutritional value. It’s the all-in-one kitchen companion designed for people with a busy lifestyle.", "These slim and stylish bodies are packed with high performance. The attractive compact designs and energy-saving functions help Panasonic Blu-ray products consume as little power as possible. You can experience great movie quality with this ultra-fast booting DMP-BD89 Full HD Blu-ray disc player. After starting the player, the time it takes from launching the menu to playing a disc is much shorter than in conventional models. The BD89 also allows for smart home networking (DLNA) and provides access to video on demand, so that home entertainment is more intuitive, more comfortable, and lots more fun."] 

df = pd.DataFrame(data, columns=['original_text'])
df['summary'] = df['original_text'].apply(summ)

# 逐行计算相似度，结果存入新列
df['similarity_score'] = df.apply(lambda row: compare(row['original_text'], row['summary']), axis=1)

场景2：4篇及以上多文本交叉相似度对比 #

如果需要计算多文本两两之间的相似度，可以先把所有待对比文本存入列表，再遍历所有文本组合计算得分：

from itertools import combinations
import pandas as pd
from pysimilar import compare

# 待对比的多文本列表，可直接从DataFrame列提取，如text_list = df['original_text'].tolist()
text_list = [
    "文本1内容",
    "文本2内容",
    "文本3内容",
    "文本4内容"
    # 可继续添加更多文本
]

# 计算所有两两组合的相似度
similarity_records = []
for text_a, text_b in combinations(text_list, 2):
    similarity_records.append({
        "text_a": text_a,
        "text_b": text_b,
        "similarity_score": compare(text_a, text_b)
    })

# 转换为DataFrame方便后续分析
result_df = pd.DataFrame(similarity_records)

如果需要生成完整的n*n相似度矩阵（每个文本和其余所有文本的得分），可以使用嵌套循环实现：

sim_matrix = []
for text_a in text_list:
    row = [compare(text_a, text_b) for text_b in text_list]
    sim_matrix.append(row)

# 生成带行列标签的相似度矩阵DataFrame
matrix_df = pd.DataFrame(sim_matrix, index=text_list, columns=text_list)

内容的提问来源于stack exchange，提问作者Sitraka F