You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

替换Fuzzywuzzy为Rapidfuzz后匹配无结果问题咨询

问题:替换Fuzzywuzzy为Rapidfuzz后匹配无结果?

我正在开发一套基于相似度匹配的系统,用于映射客户端数据与中央数据库的行数据,采用混合方法处理产品的厂商、型号、变体等特征——针对字符串类型字段,根据字段特性选择模糊逻辑或SBERT嵌入计算相似度。

原本模型运行效果尚可但速度偏慢,为提升性能,我将Fuzzywuzzy替换为更快的Rapidfuzz,仅修改了导入语句(从from fuzzywuzzy import fuzz改为from rapidfuzz import fuzz)。测试发现模糊分数计算结果和Fuzzywuzzy完全一致,但最终匹配无结果返回。

我尝试过将返回类型转换为float、调整匹配阈值等操作,均未解决问题;只有当阈值设为0时,匹配才恢复正常,但不确定这种方案是否合理。

相关代码如下:

def cached_fuzzy_score(a, b):
     return fuzz.token_set_ratio(a, b)/ 100
def get_candidates_vectorized(bank_make, central_df, threshold=60):
    # Use fuzzy matching on make names
    make_scores = central_df['make_name'].apply(
        lambda x: fuzz.token_set_ratio(bank_make, x)
    )
    return central_df[make_scores > threshold].index.tolist()


# Optimized scoring function
def calculate_scores_batch(bank_rows, central_indices, central_df,
                           model_vectorizer, segment_vectorizer,
                           model_matrix, segment_matrix,
                           segment_embeddings, identity_embeddings,
                           central_variant_embeddings, central_identity_embeddings,
                           weights):
    results = []

    bank_models = [row['bank_model'] for row in bank_rows]
    model_sims = batch_semantic_similarity(bank_models, model_vectorizer, model_matrix)

    # Convert embeddings to torch tensors
    import torch
    bank_segment_embeddings = torch.stack(segment_embeddings)
    bank_identity_embeddings = torch.stack(identity_embeddings)

    sbert_segment_sims = util.pytorch_cos_sim(bank_segment_embeddings, central_variant_embeddings)
    sbert_identity_sims = util.pytorch_cos_sim(bank_identity_embeddings, central_identity_embeddings)

    for i, bank_row in enumerate(bank_rows):
        best_score = 0
        best_match_idx = None

        for central_idx in central_indices[i]:
            central_row = central_df.iloc[central_idx]

            make_score = cached_fuzzy_score(bank_row['bank_make'], central_row['make_name'])
            model_score = model_sims[i, central_idx]
            segment_score = sbert_segment_sims[i][central_idx].item()

            fuel_score = cached_fuzzy_score(
                str(bank_row.get('extracted_fuel_type', '')), str(central_row['fuel_type'])
            )
            transmission_score = cached_fuzzy_score(
                str(bank_row.get('transmission_type', '')), str(central_row['transmission'])
            )
            displacement_score = cached_fuzzy_score(
                str(bank_row.get('extracted_displacement', '')), str(central_row['displacement_formatted'])
            )

            identity_score = sbert_identity_sims[i][central_idx].item()

            # BS rating comparison
            bank_bs = bank_row.get('bs_rating')
            central_bs = central_row.get('bs_rating')
            if bank_bs is not None and central_bs is not None:
                try:
                    bs_score = 1.0 - (abs(float(bank_bs) - float(central_bs)) / 3.0)
                    bs_score = max(bs_score, 0.0)
                except:
                    bs_score = cached_fuzzy_score(str(bank_bs), str(central_bs))
            else:
                bs_score = 0.0

            total_score = (
                weights['make'] * make_score +
                weights['model'] * model_score +
                weights['segment'] * segment_score +
                weights['fuel'] * fuel_score +
                weights['transmission'] * transmission_score +
                weights['displacement'] * displacement_score +
                weights.get('bs', 0) * bs_score +
                weights.get('identity', 0.1) * identity_score
            )

            if total_score > best_score:
                best_score = total_score
                best_match_idx = central_idx

        results.append((best_match_idx, best_score))

    return results

内容的提问来源于stack exchange,提问作者Prabhjit Singh

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.12 23:30:01