Ubuntu16.04+Python3.5实现REINFORCE自定义损失函数遇tf.Tensor转bool报错

Hey there, let's work through your two issues step by step—first the critical error that's stopping your REINFORCE code, then the CPU instruction warning that's just a performance heads-up.

This is the main problem breaking your custom loss function. Here's why it happens:
Keras (when using TensorFlow as backend) builds computation graphs, so tensors are not immediate Python values—you can't use them in native Python boolean checks like if tensor: or if tensor > 0. All conditional logic has to use TensorFlow's graph-compatible operations.

Common Mistake (What You Might Have Done) #

If your loss function looked something like this, it'll throw that exact error:

def bad_reinforce_loss(y_true, y_pred):
    # y_true = advantage values, y_pred = policy action probabilities
    if y_true > 0:  # ❌ Using tensor as Python bool—this is invalid
        return -y_true * tf.math.log(y_pred)
    else:
        return 0.0

Correct Implementation #

For REINFORCE, the loss is fundamentally -advantage * log(action_probability). You can implement this entirely with tensor operations, no Python-level conditionals needed:

def reinforce_loss(y_true, y_pred):
    # Calculate log probabilities of the chosen actions
    log_probs = tf.math.log(y_pred)
    # Compute REINFORCE loss: negative advantage-weighted log probs
    loss = -tf.multiply(y_true, log_probs)
    # Return mean loss across the batch
    return tf.reduce_mean(loss)

If you do need conditional logic later, use TensorFlow's tf.cond() or tf.where() instead of Python if/else:

def conditional_reinforce_loss(y_true, y_pred):
    log_probs = tf.math.log(y_pred)
    # TensorFlow-native conditional branch
    loss = tf.cond(
        tf.greater(tf.reduce_mean(y_true), 0.0),
        lambda: -tf.multiply(y_true, log_probs),
        lambda: tf.zeros_like(log_probs)
    )
    return tf.reduce_mean(loss)

That log message about AVX2/FMA is just telling you your CPU supports faster instruction sets, but the pre-built TensorFlow binary you installed wasn't compiled to use them. It won't crash your code—it just means your model might run a bit slower than it could.

Quick Fix to Hide the Warning #

Add these lines at the very start of your script to filter out INFO-level TensorFlow logs:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

Optional: Optimize Performance (Advanced) #

If you want to squeeze out more speed, you can compile TensorFlow from source with AVX2/FMA optimizations enabled. This is more involved, though, and usually unnecessary for small-scale REINFORCE experiments.

内容的提问来源于stack exchange，提问作者D1cvv0ng