TensorFlow简单稠密图复用报错：原因及优雅解决方法

背景与代码 #

我是TensorFlow新手，尝试通过简单程序实现稠密层的保存与恢复。

初始化并保存图的代码 #

from __future__ import absolute_import, division, print_function
import os
import tensorflow as tf

x = tf.constant([[1], [2], [3], [4]], dtype=tf.float32,name = "x" )
y_true = tf.constant([[0], [-1], [-2], [-3]], dtype=tf.float32, name = "y_t")
linear_model = tf.layers.Dense(units=1, name = "sutej")
y_pred = linear_model(x)
loss = tf.losses.mean_squared_error(labels=y_true, predictions=y_pred)
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

for i in range(1000):
    _, loss_value = sess.run((train, loss))
    print(loss_value)

all_vars = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
print(all_vars)
for v in all_vars:
    v_ = sess.run(v)
    print(v_)

print(sess.run(y_pred))

saver = tf.train.Saver()
saver.save(sess, '/home/sutej/Tensorflow/newsave/newsave',global_step=1000)

恢复图的代码 #

from __future__ import absolute_import, division, print_function
import os
import tensorflow as tf

sess = tf.Session()
saver = tf.train.import_meta_graph('/home/sutej/Tensorflow/newsave/newsave-1000.meta')
saver.restore(sess,tf.train.latest_checkpoint('/home/sutej/Tensorflow/newsave/'))

graph = tf.get_default_graph()
x=graph.get_tensor_by_name('x:0')
y_true=graph.get_tensor_by_name('y_t:0')

graph = tf.get_default_graph()
all_vars = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
print(all_vars)
for v in all_vars:
    v_ = sess.run(v)
    print(v_)

print(sess.run('sutej/kernel:0'))
print(sess.run(tf.layers.dense(inputs=x,units=1,name = 'sutej', reuse=True)))

报错信息 #

恢复后能看到图及常量、偏置、权重已加载，但最后一行调用稠密层时出现报错：

[<tf.Variable 'sutej/kernel:0' shape=(1, 1) dtype=float32_ref>, <tf.Variable 'sutej/bias:0' shape=(1,) dtype=float32_ref>]
[[-0.98440635]]
[0.95415276]
[[-0.98440635]]
Traceback (most recent call last):
  File "load_saver.py", line 26, in <module>
    print(sess.run(tf.layers.dense(inputs=x,units=1,name = 'sutej', reuse=True)))
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/layers/core.py", line 253, in dense
    return layer.apply(inputs)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 828, in apply
    return self.__call__(inputs, *args, **kwargs)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 699, in __call__
    self.build(input_shapes)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/layers/core.py", line 138, in build
    trainable=True)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 546, in add_variable
    partitioner=partitioner)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/training/checkpointable.py", line 436, in _add_variable_with_custom_getter
    **kwargs_for_getter)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1317, in get_variable
    constraint=constraint)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1079, in get_variable
    constraint=constraint)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 425, in get_variable
    constraint=constraint)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 394, in _true_getter
    use_resource=use_resource, constraint=constraint)
  File "/home/sutej/.local/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 751, in _get_single_variable
    "reuse=tf.AUTO_REUSE in VarScope?" % name)
ValueError: Variable sutej/kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

我的需求 #

我不想手动进行核与偏置的运算，希望像原程序中print(sess.run(y_pred))那样优雅复用，请问错误原因是什么？该如何解决？

错误原因分析 #

咱们拆解下报错的核心问题：

1. 变量作用域不对齐：你保存模型时，tf.layers.Dense是在默认变量作用域下生成的kernel和bias变量，但恢复图后直接调用tf.layers.dense并设置reuse=True时，当前的变量作用域并没有和已恢复变量的作用域关联，TensorFlow相当于“找错了地方”，自然找不到对应的变量。
2. 没必要重新构建层：你保存的图里已经有现成的y_pred计算节点了——它就是原程序中linear_model(x)的输出结果，完全不需要再调用tf.layers.dense重新生成预测，直接复用这个已有的节点即可。

解决方法 #

方法一：直接复用已保存的y_pred张量（推荐）

这是最贴合你需求的方式，和原程序的用法完全一致：
修改恢复代码的最后两行，替换为：

# 先在保存代码里打印y_pred的节点名称（保存前加print(y_pred.name)），再用这个名称获取
y_pred = graph.get_tensor_by_name('sutej/BiasAdd:0')
print(sess.run(y_pred))

小提示：如果不确定节点名称，可以在保存图的代码中加入print(y_pred.name)，输出的字符串就是你需要的张量名称。

方法二：通过变量作用域正确复用层

如果你确实需要重新构建层并复用变量，要确保变量作用域正确设置：
修改恢复代码的最后部分为：

with tf.variable_scope('sutej', reuse=tf.AUTO_REUSE):
    y_pred = tf.layers.dense(inputs=x, units=1, name='sutej')
print(sess.run(y_pred))

不过这种方式不如直接获取已有张量优雅，更推荐第一种方法。

额外建议 #

作为TensorFlow新手，你可以尝试切换到TensorFlow 2.x版本，它的模型保存与恢复机制（比如Model.save和tf.keras.models.load_model）更加直观易用，不需要手动处理图和变量作用域的底层问题，能避开很多这类坑。

内容的提问来源于stack exchange，提问作者sutej kulgod