Keras中to_categorical()默认返回float64而非float32的原因及疑问

1. Why does to_categorical() default to returning float64 instead of float32? #

There are a few practical reasons behind this implementation choice:

• NumPy convention alignment: NumPy’s default floating-point type is float64, and Keras operations have historically leaned into NumPy’s behavior for consistency. Since to_categorical() builds one-hot arrays using NumPy under the hood, it inherits this default type.
• Negligible performance overhead: One-hot encoding is a lightweight operation—unlike heavy model layers like convolutions or matrix multiplications, using float64 here doesn’t create a noticeable performance bottleneck, even on GPUs. Optimizing for float32 wasn’t a critical priority for this specific function.
• Precision safety: Float64 offers higher precision than float32, which eliminates any risk of subtle precision loss in edge cases (like certain evaluation metric calculations or debugging scenarios). For a utility function like this, playing it safe with higher precision made sense during initial development.
• Historical legacy: Early Keras versions were built on frameworks like Theano, which also defaulted to float64 in many contexts. This choice carried over as Keras evolved, even as float32 became the standard for GPU-optimized model weights.

2. Should we rely on this return type, and is explicit astype() conversion necessary? #

Short answer: Never rely on the unstated return type—and yes, you should always add an explicit astype() conversion if you need a specific type like float32. Here’s why:

• No API guarantee: The Keras documentation doesn’t specify the return type of to_categorical(), which means this is an implementation detail, not a stable part of the interface. Maintainers could change the default to float32 (or another type) in a future update without warning, breaking code that assumes float64.
• GPU compatibility: While modern GPUs can handle float64, they’re heavily optimized for float32 (and even smaller types like float16/bfloat16). Converting your one-hot labels to float32 ensures they match the data type of your model’s weights (almost always float32 by default), avoiding unnecessary runtime type casting that slows down training.
• Code robustness: Explicitly setting the type makes your code self-documenting. Anyone reading it will immediately know what data type you expect for labels, instead of having to infer it from the to_categorical() implementation.

Example of explicit conversion:

from tensorflow.keras.utils import to_categorical

# Convert labels to float32 one-hot encoding
one_hot_labels = to_categorical(labels, num_classes=10).astype('float32')

内容的提问来源于stack exchange，提问作者Lingxi