联邦平均（Federated Averaging）测试集准确率始终相同问题求助

在IoT-23轻量版数据集上实现联邦平均（Federated Averaging），预处理、模型定义、客户端训练及聚合代码如下，目前遇到的核心问题：无论聚合多少个客户端的权重，测试集准确率始终完全相同（如聚合3组或全部9组时结果一致），但单独使用各客户端权重测试时准确率存在差异。

数据划分代码 #

X_train, X_test, Y_train, Y_test = train_test_split(X, y, random_state=10, test_size=0.2)
test_set = pd.concat([X_test, Y_test], axis=1)
test_set.to_csv('./dataset/test_set.csv')

num_sets = 9

set_size = len(X_train) // num_sets
train_sets = np.array_split(train_set, num_sets)
for i in range(num_sets):
    train_sets[i].to_csv(f'./dataset/train{i}.csv')#type:ignore

模型结构代码 #

model = models.Sequential([
        layers.Input(shape=(24,)),
        layers.Dense(150,activation='relu'),
        layers.Dense(80,activation='relu'),
        layers.Dropout(0.2),
        layers.Dense(7, activation='softmax')
    ])
loss_fn = losses.CategoricalFocalCrossentropy(alpha=0.2)
model.compile(loss=loss_fn, optimizer='rmsprop', metrics=['accuracy'])

客户端训练代码 #

for client_number in range(9):
    model = models.load_model('./model/model.keras')
    train_data=pd.read_csv(f"./dataset/train{client_number}.csv",dtype='float')
    X_train = train_data.iloc[:, 1:-7]
    y_train = train_data.iloc[:, -7:]
    
    base_model.fit(X_train, y_train, epochs=5)

    model.save_weights(f'./weights/weight{client_number}.weights.h5')#type:ignore

联邦平均聚合代码 #

权重聚合函数 #

def sum_weights(weight_list):
    averaged_weights = list()
    #get the average grad accross all client gradients
    for grad_list_tuple in zip(*weight_list):
        layer_mean = tf.math.reduce_sum(grad_list_tuple, axis=0)
        averaged_weights.append(layer_mean)
        
    return sum_weights

聚合执行代码 #

model=models.load_model('./model/model.keras')
for i in range(3):
    model.load_weights(f'./weights/weight{i}.weights.h5')#type:ignore
    org_weights.append(model.get_weights())#type:ignore

average_weights = sum_weights(org_weights)
model.set_weights(average_weights)
    
loss, accuracy = model.evaluate(X, y)#type:ignore
print(f'Org Aggregate no.{j} accuracy: {accuracy}')

核心问题定位 #

1. 聚合函数逻辑错误：当前sum_weights仅对客户端权重做求和操作，未除以客户端数量求平均。权重求和会导致参数值被放大，模型输出的softmax结果趋向于固定类别，最终准确率无变化。
2. 训练变量名混淆：客户端训练时加载的是model，但调用base_model.fit，变量不匹配可能导致保存的权重并非实际训练后的结果（虽单独测试准确率不同，但需修正避免潜在问题）。
3. 评估数据集错误：聚合后用全量数据X,y评估，而非划分好的测试集，结果无参考意义且可能掩盖问题。
4. 数据划分笔误：原代码中train_sets = np.array_split(train_set, num_sets)的train_set未定义，应为train_set = pd.concat([X_train, Y_train], axis=1)。

修正步骤 #

1. 修复权重聚合函数 #

将求和改为求平均（适配等分数据集的联邦平均逻辑）：

def average_weights(weight_list):
    averaged_weights = list()
    client_count = len(weight_list)
    for grad_list_tuple in zip(*weight_list):
        # 求和后除以客户端数量得到平均权重
        layer_mean = tf.math.reduce_sum(grad_list_tuple, axis=0) / client_count
        averaged_weights.append(layer_mean)
        
    return averaged_weights

2. 修正客户端训练变量名 #

确保训练的是加载的模型：

for client_number in range(9):
    model = models.load_model('./model/model.keras')
    train_data = pd.read_csv(f"./dataset/train{client_number}.csv", dtype='float')
    X_train = train_data.iloc[:, 1:-7]
    y_train = train_data.iloc[:, -7:]
    
    # 替换base_model为model，确保训练当前加载的模型
    model.fit(X_train, y_train, epochs=5)

    model.save_weights(f'./weights/weight{client_number}.weights.h5')

3. 修正评估数据集 #

使用划分好的测试集进行评估：

# 加载测试集
test_set = pd.read_csv('./dataset/test_set.csv', dtype='float')
X_test = test_set.iloc[:, 1:-7]
Y_test = test_set.iloc[:, -7:]

# 测试聚合3个客户端的情况
model = models.load_model('./model/model.keras')
org_weights = []
for i in range(3):
    model.load_weights(f'./weights/weight{i}.weights.h5')
    org_weights.append(model.get_weights())

avg_weights = average_weights(org_weights)
model.set_weights(avg_weights)
    
loss, accuracy = model.evaluate(X_test, Y_test)
print(f'Aggregate 3 clients accuracy: {accuracy}')

# 测试聚合9个客户端的情况
org_weights = []
for i in range(9):
    model.load_weights(f'./weights/weight{i}.weights.h5')
    org_weights.append(model.get_weights())

avg_weights = average_weights(org_weights)
model.set_weights(avg_weights)
    
loss, accuracy = model.evaluate(X_test, Y_test)
print(f'Aggregate 9 clients accuracy: {accuracy}')

4. 修正数据划分笔误 #

补充train_set的定义：

X_train, X_test, Y_train, Y_test = train_test_split(X, y, random_state=10, test_size=0.2)
test_set = pd.concat([X_test, Y_test], axis=1)
test_set.to_csv('./dataset/test_set.csv')

# 补充train_set定义
train_set = pd.concat([X_train, Y_train], axis=1)

num_sets = 9
set_size = len(X_train) // num_sets
train_sets = np.array_split(train_set, num_sets)
for i in range(num_sets):
    train_sets[i].to_csv(f'./dataset/train{i}.csv')

内容的提问来源于stack exchange，提问作者AdityaDN