TensorFlow：合并图后从两个检查点恢复变量问题求助

Hey there! Let's work through this checkpoint loading issue you're hitting after merging Graph A and Graph B. I’ve run into similar headaches before, so here’s a practical, step-by-step breakdown to fix it:

1. First, Avoid Naming Conflicts with Variable Scopes #

The biggest culprit here is usually duplicate variable names (like both graphs having a conv2d/kernel). Start by wrapping each graph's construction in a unique variable scope to keep their variables separate:

# Build/import Graph A under its own scope
with tf.variable_scope('graph_a'):
    a_layer2 = build_or_import_graph_a()  # Your existing code to get Graph A's 2nd layer

# Build/import Graph B under its own scope, then connect to Graph A's layer
with tf.variable_scope('graph_b'):
    # Replace Graph B's 3rd layer input with a_layer2
    b_layer3 = connect_a_layer2_to_b_third_layer(a_layer2)

2. Load Checkpoints Separately with Targeted Savers #

TensorFlow's tf.train.Saver lets you load subsets of variables. Create separate savers for each graph's variables, then load them one after another:

# Grab all variables belonging to Graph A
graph_a_vars = [var for var in tf.global_variables() if var.name.startswith('graph_a/')]
saver_a = tf.train.Saver(graph_a_vars)

# Grab all variables belonging to Graph B
graph_b_vars = [var for var in tf.global_variables() if var.name.startswith('graph_b/')]
saver_b = tf.train.Saver(graph_b_vars)

# Load weights in a session
with tf.Session() as sess:
    # Initialize all variables first (covers any new/unchanged variables)
    sess.run(tf.global_variables_initializer())
    
    # Load Graph A's checkpoint
    saver_a.restore(sess, '/path/to/your/graph_a_checkpoint')
    
    # Load Graph B's checkpoint
    saver_b.restore(sess, '/path/to/your/graph_b_checkpoint')
    
    # Now your merged graph is ready to use!

3. Fix Mismatched Variable Names with Custom Maps #

If your checkpoint variable names don't line up with the merged graph (e.g., original Graph B's conv3/kernel is now graph_b/conv3/kernel), use a variable map to bridge the gap:

# Map checkpoint variable names to merged graph variables
graph_a_var_map = {
    'conv2/kernel': tf.get_default_graph().get_tensor_by_name('graph_a/conv2/kernel:0'),
    'conv2/bias': tf.get_default_graph().get_tensor_by_name('graph_a/conv2/bias:0'),
    # Add all other Graph A variables you need to load
}

graph_b_var_map = {
    'conv3/kernel': tf.get_default_graph().get_tensor_by_name('graph_b/conv3/kernel:0'),
    'conv3/bias': tf.get_default_graph().get_tensor_by_name('graph_b/conv3/bias:0'),
    # Add all other Graph B variables you need to load
}

# Create savers with the custom maps
saver_a = tf.train.Saver(graph_a_var_map)
saver_b = tf.train.Saver(graph_b_var_map)

# Follow the same session loading steps as above

4. Simplify with tf.train.init_from_checkpoint (Recommended for TF 1.x) #

If you're using TensorFlow 1.x, init_from_checkpoint is a cleaner way to initialize variables from multiple checkpoints without writing custom savers:

# Map Graph A's checkpoint variables to its scope in the merged graph
tf.train.init_from_checkpoint(
    '/path/to/graph_a_checkpoint',
    {'': 'graph_a/'}  # Empty prefix in checkpoint maps to 'graph_a/' in merged graph
)

# Map Graph B's checkpoint variables to its scope
tf.train.init_from_checkpoint(
    '/path/to/graph_b_checkpoint',
    {'': 'graph_b/'}
)

# Initialize variables – the specified ones will auto-load from checkpoints
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    # Merged graph is ready to go!

Quick Notes to Avoid Headaches #

• Double-check that the variable shapes (like kernel size, channel count) match between the checkpoints and your merged graph – mismatches will throw errors.
• Any new variables (like if you added a small adapter layer between A's 2nd and B's 3rd layer) will be initialized with default values, which is totally fine.

内容的提问来源于stack exchange，提问作者Ale Solano