Keras模型拟合报错‘constant folding failed: Unsupported type:21’及异常求助

Hey there, let's break down your problem step by step—this is a mix of a graph optimization warning and a model training issue, so we'll tackle both:

First, that error comes from TensorFlow's Grappler optimizer, which handles graph optimizations like constant folding (precomputing static values to speed up training). The "type 21" refers to tf.string—Grappler's constant folding logic doesn't support string tensors right now, so it throws this warning when it encounters them in your computation graph.

The good news? This is an optimization-stage warning, not a critical error—it won't stop your training, but it might slightly reduce training efficiency since that optimization step fails. It's unlikely to be the direct cause of your loss/accuracy problems, but we can fix it if it's cluttering your logs.

Your main issue here is almost certainly a data or configuration mismatch with your binary classification setup, not the Grappler warning. Let's walk through the most likely culprits:

1. Label format mismatch: When using binary_crossentropy, your labels need to be single-value floats (0.0 or 1.0) with a shape like (num_samples,). If your labels are one-hot encoded (e.g., [0,1] or [1,0]) or integer types, the loss calculation will be completely wrong—leading to sky-high loss and inverted accuracy (since the model is learning the opposite of what it should).
2. Data pipeline mix-up: Double-check that you're loading text and labels in the correct order. If you accidentally swapped them (feeding labels as text and vice versa), the model will be trying to "predict" text from labels, which explains why flipping the predictions gives good results.
3. Optimizer learning rate: The default RMSprop learning rate (0.001) might be too high for your dataset. A learning rate that's too large causes the model's parameters to oscillate wildly instead of converging, leading to rising loss.

Let's go through actionable steps to resolve both issues:

1. Fix Your Label Setup #

• Convert your labels to float32 single values (not integers or one-hot vectors):

  labels = tf.cast(labels, tf.float32)
  

• Verify the shape: Print a batch of labels—they should look like array([0., 1., 0.], dtype=float32) (shape (batch_size,), not (batch_size, 2)).

2. Validate Your Data Pipeline #

• Print a few examples from your dataset to confirm text and labels are correctly paired:

  for text, label in your_dataset.take(3):
      print(f"Text: {text.numpy()}")
      print(f"Label: {label.numpy()}\n")
  

• Make sure there's no accidental swapping or misalignment here.

3. Adjust the Optimizer #

• Lower the RMSprop learning rate to a smaller value, like 0.0001:

  optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.0001)
  

• This will help the model converge smoothly instead of overshooting the optimal parameters.

4. Suppress the Grappler Warning (Optional) #

If the error logs are annoying and you don't mind losing a bit of optimization speed, disable constant folding:

tf.config.optimizer.set_experimental_options({"constant_folding": False})

5. Double-Check Your Model Output Layer #

For binary classification, your final layer should be a single neuron with a sigmoid activation:

model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

If you're using Dense(2) with softmax, you should switch to sparse_categorical_crossentropy instead—this is another common misconfiguration that causes loss issues.

内容的提问来源于stack exchange，提问作者OmriP