使用AIF360对抗去偏模型遇TensorFlow变量重复定义报错求助

问题场景 #

使用基于TensorFlow的AIF360库中AdversarialDebiasing模型时，未手动编写TensorFlow代码，却出现变量重复定义的错误：

ValueError: Variable debiased_classifier_20710/classifier_model/W1 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?

相关代码片段 #

sess = tf.Session()
debiased_model = AdversarialDebiasing(privileged_groups = privileged_groups, unprivileged_groups = unprivileged_groups,  scope_name = f'debiased_classifier_{int(np.random.random()*100000)}',  debias = True, sess = sess)
model = debiased_model.fit(train_df)

完整报错栈 #

File "python3.10/site-packages/aif360/algorithms/transformer.py", line 27, in wrapper
new_dataset = func(self, *args, **kwargs)
File "python3.10/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py", line 152, in fit
self.pred_labels, pred_logits = self._classifier_model(self.features_ph, self.features_dim,  self.keep_prob)
File "python3.10/site-packages/aif360/algorithms/inprocessing/adversarial_debiasing.py", line  84, in _classifier_model
W1 = tf.get_variable('W1', [features_dim, self.classifier_num_hidden_units],
File "python3.10/site-packages/tensorflow/python/ops/variable_scope.py", line 1616, in  get_variable
return get_variable_scope().get_variable(
File "python3.10/site-packages/tensorflow/python/ops/variable_scope.py", line 1326, in  get_variable
return var_store.get_variable(
File "python3.10/site-packages/tensorflow/python/ops/variable_scope.py", line 582, in  get_variable
return _true_getter(
File "python3.10/site-packages/tensorflow/python/ops/variable_scope.py", line 535, in  _true_getter
return self._get_single_variable(
File "python3.10/site-packages/tensorflow/python/ops/variable_scope.py", line 891, in  _get_single_variable
raise ValueError(err_msg)
ValueError: Variable debiased_classifier_20710/classifier_model/W1 already exists, disallowed.  Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?

解决思路 #

1. 重置TensorFlow默认图并管理Session生命周期 #

TensorFlow的默认计算图会保留之前创建的变量，即使使用随机命名的scope_name，只要在同一个图里重复运行代码就会冲突。每次创建模型前重置默认图，并在训练完成后关闭Session：

import tensorflow as tf
import numpy as np

# 重置默认计算图，清除之前的变量
tf.reset_default_graph()
# 创建新的Session
sess = tf.Session()

# 初始化模型
debiased_model = AdversarialDebiasing(
    privileged_groups=privileged_groups,
    unprivileged_groups=unprivileged_groups,
    scope_name=f'debiased_classifier_{int(np.random.random()*100000)}',
    debias=True,
    sess=sess
)
model = debiased_model.fit(train_df)

# 训练结束后关闭Session，释放资源
sess.close()

2. 避免重复调用同一模型实例的fit方法 #

如果多次调用同一个AdversarialDebiasing实例的fit方法，会在同一个变量作用域下重复创建变量。每次训练都要新建模型实例，或确保每次训练都重置图和Session。

3. 显式指定变量作用域的复用（进阶） #

如果需要复用变量，可以在创建模型前手动设置变量作用域的复用策略：

tf.reset_default_graph()
sess = tf.Session()

with tf.variable_scope(f'debiased_classifier_{int(np.random.random()*100000)}', reuse=tf.AUTO_REUSE):
    debiased_model = AdversarialDebiasing(
        privileged_groups=privileged_groups,
        unprivileged_groups=unprivileged_groups,
        scope_name=f'debiased_classifier_{int(np.random.random()*100000)}',
        debias=True,
        sess=sess
    )
    model = debiased_model.fit(train_df)

sess.close()

内容的提问来源于stack exchange，提问作者surbhi rathore