GradientDescentOptimizer输入参数解析及代码报错排查咨询

Hey there! Let's break down why your code is throwing that error and fix it so you can experiment with the optimizer just like you did with activation functions.

Why You're Getting the TypeError #

The core issue here is that your loss_data and train_data have no computational relationship. TensorFlow can only compute gradients when the loss depends on the variables you're trying to optimize. Right now, loss_data is just a standalone variable—changing train_data doesn't affect it at all. So when you call compute_gradients, TensorFlow returns None for the gradient of train_data with respect to loss_data, and sess.run() can't handle fetching None.

Fixing the Code: Create a Dependent Loss #

To mimic the "black box" approach you used for activation functions, let's define a simple loss that directly depends on train_data. For example, let's use loss = train_data ** 2 (a basic quadratic loss—we know the minimum should be at 0, so it's easy to verify the optimizer's behavior).

Minimal Working Example #

Here's a revised code snippet that lets you see gradients and test the optimizer:

import tensorflow as tf

# Define our variable to optimize
train_data = tf.Variable([20.0], dtype=tf.float32, name='train')
# Define a loss that DEPENDS on train_data
loss = train_data ** 2  # Simple quadratic loss

# Initialize the optimizer
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.02)

# Calculate gradients (this will now have a valid value)
gradients = optimizer.compute_gradients(loss, var_list=[train_data])
# Create an operation to apply the gradients (equivalent to minimize())
apply_grads = optimizer.apply_gradients(gradients)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    
    # Check initial gradients
    print("Initial gradient:", sess.run(gradients))
    # The gradient of x² w.r. to x is 2x, so initial gradient should be [40.0]
    
    # Run a few optimization steps
    for step in range(5):
        sess.run(apply_grads)
        current_val, current_loss = sess.run([train_data, loss])
        print(f"Step {step+1}: Train value = {current_val[0]:.4f}, Loss = {current_loss[0]:.4f}")

What This Does #

• Dependent Loss: By tying loss to train_data, TensorFlow can compute valid gradients. For loss = x², the gradient is 2x—you can verify this matches the initial output.
• compute_gradients: Returns a list of (gradient, variable) pairs. Now this will have a numeric gradient instead of None.
• apply_gradients: Takes the gradient pairs and updates the variables (this is what minimize() does under the hood—it combines compute_gradients and apply_gradients).

Using the "Black Box" Approach #

Just like you tested activation functions by feeding inputs and checking outputs, you can tweak this code to experiment with the optimizer:

• Change the learning rate (try 0.1 vs 0.001 to see how fast the value converges to 0)
• Use different loss functions (e.g., loss = tf.abs(train_data - 5) to optimize towards 5)
• Add more variables and see how the optimizer handles multiple gradients

Simplified Version with minimize() #

If you just want to use the full minimize() method (instead of splitting into compute/apply), here's an even shorter snippet:

import tensorflow as tf

train_data = tf.Variable([20.0], dtype=tf.float32)
loss = train_data ** 2
optimizer = tf.train.GradientDescentOptimizer(0.02)
train_op = optimizer.minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for step in range(5):
        sess.run(train_op)
        print(f"Step {step+1}: Value = {sess.run(train_data)[0]:.4f}")

This does the same thing as the previous example—minimize() handles computing gradients and applying them in one step.

内容的提问来源于stack exchange，提问作者Chris Snow