TensorFlow代码出现异常类型错误：Maximum操作输入类型不匹配

Let's break down what's causing this error and fix it step by step.

What's Going Wrong? #

Your error message flags a type mismatch in the Maximum operation—one input is float32, but the other doesn't match. Here are the key issues in your code:

1. Mixing Variable Assignment with Tensor Operations
   You initialized curMaxAbs and maxi as tf.Variable, but then overwrote them directly with the output of tf.cond (a tf.Tensor). In TensorFlow's graph mode, this doesn't update the original variable—it creates a new tensor with the same name, leading to unexpected type or context issues later on.

2. Potential Type Mismatch in Gradient Tensors
   If any gradient tensor g in grads isn't float32, tf.reduce_max(tf.abs(g)) will return a tensor of a different type (e.g., int32 if g is integer-type). Passing this mismatched type to tf.maximum alongside curMaxAbs (a float32 variable) triggers the error.

3. Python Integer vs. TensorFlow Tensor
   When using lambda: i in your tf.cond for maxi, i is a plain Python integer, not a TensorFlow int32 tensor. This can cause implicit conversion bugs that ripple through your computation graph.

Fixed Code Snippet #

Here's the corrected version addressing all these issues:

curMaxAbs = tf.Variable(-1.0, tf.float32)
maxi = tf.Variable(-1, tf.int32)

for i, g in enumerate(grads):
    # Ensure maxG is float32, even if g has a different type
    maxG = tf.cast(tf.reduce_max(tf.abs(g)), tf.float32)
    oCurMaxAbs = curMaxAbs.value()  # Get the variable's current value as a tensor
    
    # Update curMaxAbs properly using assign()
    curMaxAbs.assign(tf.cond(
        tf.greater(maxG, oCurMaxAbs),
        lambda: maxG,
        lambda: oCurMaxAbs
    ))
    
    # Update maxi: cast loop index to tf.int32 and use assign()
    maxi.assign(tf.cond(
        tf.greater(maxG, oCurMaxAbs),
        lambda: maxi.value(),
        lambda: tf.cast(i, tf.int32)
    ))

# This line will now run without type errors
if not tf.equal(curMaxAbs, tf.maximum(curMaxAbs, maxG)):
    # Your logic here
    pass

Key Fixes Explained #

• Use assign() for Variable Updates: Instead of overwriting variables directly, assign() modifies the original tf.Variable in place, keeping the tensor context consistent.
• Explicit Type Casting: tf.cast(maxG, tf.float32) ensures maxG matches curMaxAbs's float32 type, even if your gradients are a different type.
• Convert Python Integers to Tensors: tf.cast(i, tf.int32) turns the loop index into a proper TensorFlow tensor, matching maxi's int32 type and avoiding conversion bugs.
• Retrieve Variable Values Safely: oCurMaxAbs = curMaxAbs.value() gets the variable's current tensor value, which is more reliable than using the variable directly in conditional operations.

内容的提问来源于stack exchange，提问作者dashnick