如何生成满足特质词与配对频率平衡的指定数量三元组？

我们正在设计一项实验，让受试者从60个特质词的三元组中选出异类。需对全部34200个可能的三元组进行下采样（可在数据采集或分析阶段执行），最终三元组需满足以下约束：

• 各特质词的出现频率大致相等；
• 三元组中隐含的特质词配对出现频率大致相等（例如三元组(kind, mean, smart)包含配对(kind, mean; kind, smart; mean, smart)）；
• 所有可能的配对至少出现在一个三元组中；
• 特质词通过两两邻接矩阵形成单一连通分量。

特质词与配对生成代码 #

import pandas as pd
import numpy as np
from itertools import combinations
from collections import deque

traits = ['kind', 'mean', 'smart', 'ambitious', 'sneaky', 'anxious', 'cunning', 'open-minded', 'diligent', 'creative']    
pairs = list(combinations(traits, 2))

三元组生成方法 #

方法1

traits_deque = deque(traits)
triplets = []
for word1, word2 in pairs:
    curr_triplets = []
    while len(curr_triplets) < 3:
        word3 = traits_deque[0]
        if (word1 != word3) and (word2 != word3):
            curr_triplets.append((word1, word2, word3))
        traits_deque.rotate(-1)
    triplets.extend(curr_triplets)

方法2

triplets2 = []
for word1, word2 in pairs:
    traits_curr = traits.copy()
    traits_curr.remove(word1)
    traits_curr.remove(word2)
    third_words = np.random.choice(traits_curr, size=3, replace=False)
    curr_triplets = [(word1, word2, third_words[0]),
                     (word1, word2, third_words[1]),
                     (word1, word2, third_words[2])]
    triplets2.extend(curr_triplets)

方法3

triplets = list(combinations(traits, 3))
triplets_df = pd.DataFrame(triplets)
triplets_df_sample = triplets_df.sample(frac=1)#n=5310)

特质词频率统计方法 #

方法1

print(pd.Series(np.array(triplets).flatten()).value_counts())

配对频率统计方法 #

方法1

all_pairs = []
for triplet in triplets:
    curr_pairs = [set(x) for x in combinations(triplet, 2)]
    all_pairs.extend(curr_pairs)   
ser = pd.Series(all_pairs).value_counts()
print(ser)

方法2

all_pairs = []
for idx, triplet in triplets_df_sample.iterrows():
    curr_pairs = [set(x) for x in combinations(triplet.values, 2)]
    all_pairs.extend(curr_pairs)   
ser = pd.Series(all_pairs).value_counts()
print(ser)

当前问题 #

我们最初尝试生成全部1770种特质词配对，再选择另外3个词组成三元组，但无论采用哪种方法，均遇到同一问题：特质词的出现频率大致均衡，但配对的出现频率不均衡。

解决方案 #

1. 核心思路

以配对频率均衡为核心目标，先保证所有配对至少出现一次（满足约束3、4），再通过迭代补充三元组，让所有配对的出现频率趋近于设定的目标值，同时同步控制特质词的出现频率保持均匀。

2. 代码实现示例

import pandas as pd
import numpy as np
from itertools import combinations
from collections import defaultdict

# 初始化参数
traits = ['kind', 'mean', 'smart', 'ambitious', 'sneaky', 'anxious', 'cunning', 'open-minded', 'diligent', 'creative']
N = len(traits)
target_pair_freq = 3  # 设定每个配对的目标出现次数
all_triplets = set()
pair_freq = defaultdict(int)

# 第一步：生成基础三元组，确保所有配对至少出现一次（保证连通性）
for i in range(N):
    for j in range(i+1, N):
        word1, word2 = traits[i], traits[j]
        # 选一个未与当前配对重复的互补词，优先保证连通性
        candidates = [w for w in traits if w not in (word1, word2)]
        word3 = candidates[0]
        triplet = tuple(sorted((word1, word2, word3)))
        all_triplets.add(triplet)
        # 更新初始配对频率
        for pair in combinations(triplet, 2):
            sorted_pair = tuple(sorted(pair))
            pair_freq[sorted_pair] += 1

# 第二步：迭代补充三元组，均衡配对频率
while max(pair_freq.values()) - min(pair_freq.values()) > 0 or any(v < target_pair_freq for v in pair_freq.values()):
    # 筛选出当前频率最低且未达目标的配对
    min_freq = min(pair_freq.values())
    low_freq_pairs = [p for p in pair_freq if pair_freq[p] == min_freq and pair_freq[p] < target_pair_freq]
    if not low_freq_pairs:
        break
    
    # 随机选一个低频配对
    selected_pair = np.random.choice(low_freq_pairs)
    word1, word2 = selected_pair
    # 选择当前出现次数最少的特质词作为互补词，保证特质频率均匀
    candidates = [w for w in traits if w not in (word1, word2)]
    trait_counts = defaultdict(int)
    for triplet in all_triplets:
        for t in triplet:
            trait_counts[t] += 1
    # 按出现次数排序候选词
    candidates_sorted = sorted(candidates, key=lambda x: trait_counts[x])
    word3 = candidates_sorted[0]
    
    # 生成新三元组并去重
    new_triplet = tuple(sorted((word1, word2, word3)))
    if new_triplet not in all_triplets:
        all_triplets.add(new_triplet)
        # 更新配对频率
        for pair in combinations(new_triplet, 2):
            sorted_pair = tuple(sorted(pair))
            pair_freq[sorted_pair] += 1

# 转换为最终列表
final_triplets = list(all_triplets)

# 验证结果
print("特质词频率：")
print(pd.Series(np.array(final_triplets).flatten()).value_counts())

print("\n配对频率：")
all_pairs = []
for triplet in final_triplets:
    curr_pairs = [tuple(sorted(x)) for x in combinations(triplet, 2)]
    all_pairs.extend(curr_pairs)
print(pd.Series(all_pairs).value_counts())

3. 针对60个特质词的优化建议

当特质词数量扩展到60时，直接迭代效率较低，可做以下优化：

• 预先计算特质词的目标出现次数：target_trait_freq = (N-1)*target_pair_freq（每个特质词参与N-1个配对，每个配对出现target_pair_freq次，对应该词的总出现次数）
• 采用批量生成逻辑：为多个低频配对批量分配互补词时，轮询选择不同的特质词，避免单个词过度使用
• 用矩阵替代字典跟踪频率：使用二维数组存储配对频率，一维数组存储特质词频率，减少循环计算开销

内容的提问来源于stack exchange，提问作者figgy