TensorFlow中基于损失函数历史的神经网络训练实现问题

Alright, let's break down how to solve this problem. The core issue here is that using placeholders to track loss history breaks TensorFlow's automatic differentiation chain—since placeholders are treated as external inputs, the framework can't trace how they connect to your model's parameters. Instead, we need to keep track of your model's outputs f_k directly within TensorFlow's computation graph so the automatic differentiation tools can follow the full path from parameters to loss.

Key Approach #

Instead of placeholders, use TensorArray (a TensorFlow-native data structure for dynamic sequences) or a list of variables to store each iteration's f_k value. This ensures TensorFlow maintains the gradient flow between your model parameters and the historical outputs used in your loss function.

Example with TensorFlow 2.x (Eager Execution) #

TF2.x's eager execution makes this straightforward with tf.GradientTape and TensorArray:

import tensorflow as tf

# Define your model (example simple dense network)
class SimpleModel(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.dense = tf.keras.layers.Dense(10, activation='relu')
        self.output_layer = tf.keras.layers.Dense(1)
    
    def call(self, inputs):
        x = self.dense(inputs)
        return self.output_layer(x)

# Initialize model, optimizer, and TensorArray to track f_k history
model = SimpleModel()
optimizer = tf.keras.optimizers.Adam()
max_iterations = 20
f_history = tf.TensorArray(tf.float32, size=max_iterations+1, dynamic_size=False)

# Dummy input (replace with your actual data)
input_data = tf.random.normal((32, 5))

# Training loop
for T in range(1, max_iterations+1):
    with tf.GradientTape() as tape:
        # Compute current f_k
        f_k = model(input_data)
        # Store current f_k in the history array
        f_history = f_history.write(T, f_k)
        
        # Calculate loss only when we have enough history (T >=2, since we need f_{k-1})
        if T >= 2:
            loss = 0.0
            # Sum ||f_k - f_{k-1}|| from k=T to 20
            for k in range(T, max_iterations+1):
                f_k_val = f_history.read(k)
                f_k_minus_1_val = f_history.read(k-1)
                loss += tf.norm(f_k_val - f_k_minus_1_val, ord=2)
    
    # Compute gradients and update parameters only when loss is defined
    if T >=2:
        gradients = tape.gradient(loss, model.trainable_variables)
        optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    
    # Optional: print progress
    if T >=2:
        print(f"Iteration {T}, Loss: {loss.numpy():.4f}")

# Convert TensorArray to a regular tensor if needed
f_history = f_history.stack()

Example with TensorFlow 1.x (Static Graph) #

If you're using TF1.x with static graphs, you'll need to build the graph first and use sessions to run operations:

import tensorflow as tf

tf.disable_eager_execution()

# Define model
def model(inputs):
    dense = tf.layers.dense(inputs, 10, activation='relu')
    output = tf.layers.dense(dense, 1)
    return output

# Build graph
input_data = tf.placeholder(tf.float32, shape=(None, 5))
max_iterations = 20

# TensorArray to track f_k history
f_history = tf.TensorArray(tf.float32, size=max_iterations+1, dynamic_size=False)

# Iteration loop in graph
def body(T, f_history):
    f_k = model(input_data)
    f_history = f_history.write(T, f_k)
    
    # Compute loss
    loss = tf.constant(0.0)
    for k in range(T, max_iterations+1):
        f_k_val = f_history.read(k)
        f_k_minus_1_val = f_history.read(k-1)
        loss += tf.norm(f_k_val - f_k_minus_1_val, ord=2)
    
    # Compute gradients and update
    optimizer = tf.train.AdamOptimizer()
    gradients = tf.gradients(loss, tf.trainable_variables())
    train_op = optimizer.apply_gradients(zip(gradients, tf.trainable_variables()))
    
    return T+1, f_history, loss, train_op

# Initialize loop variables
T = tf.constant(1)
loop_cond = lambda T, *args: tf.less_equal(T, max_iterations)
_, final_f_history, final_loss, _ = tf.while_loop(loop_cond, body, [T, f_history])

# Run session
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    dummy_input = tf.random.normal((32,5)).eval()
    for step in range(1, max_iterations+1):
        if step >=2:
            _, loss_val = sess.run([train_op, final_loss], feed_dict={input_data: dummy_input})
            print(f"Iteration {step}, Loss: {loss_val:.4f}")
        else:
            sess.run(body, feed_dict={input_data: dummy_input})

Why This Works #

By using TensorArray, we're keeping all f_k values within TensorFlow's computation graph. This means when we compute the loss as the sum of L2 norms between consecutive f values, TensorFlow can trace back through each f_k to the model's parameters—allowing tf.GradientTape (TF2.x) or tf.gradients (TF1.x) to compute valid gradients for training.

Placeholders fail here because they don't have a connection to the model's parameters in the graph; TensorFlow can't know how to compute gradients from a placeholder input back to your model weights.

内容的提问来源于stack exchange，提问作者Krishnan