Keras自定义全局标准差池化层Dimension报错与输出NaN求解

问题1：自定义GlobalStdPooling1D层报错TypeError: float() argument must be a string or a number, not 'Dimension' #

错误原因 #

• 父类调用错误：GlobalStdPooling1D继承自自定义的GlobalPooling1D，但__init__中super错误传入了GlobalAveragePooling1D作为父类
• 输出形状返回值类型错误：compute_output_shape方法直接返回了tf.TensorShape对象，旧版本Keras要求返回Python原生的列表/元组，tf.TensorShape中的动态维度会以Dimension类型存在，无法被Keras直接解析
• 代码缩进错误：抽象父类GlobalPooling1D的get_config方法缩进错误，不属于类成员
• 张量类型调用错误：mask处理阶段错误调用inputs[0].dtype，inputs本身就是输入张量而非张量列表

修复后的自定义层实现 #

from keras.layers import Layer, InputSpec, Conv1D, Dense, Input
from keras.models import Model
import keras.backend as backend
from keras.optimizers import Adam
from keras.utils import conv_utils
import tensorflow as tf

class GlobalPooling1D(Layer):
    """Abstract class for different global pooling 1D layers."""
    def __init__(self, data_format='channels_last', keepdims=False, **kwargs):
        super(GlobalPooling1D, self).__init__(**kwargs)
        self.input_spec = InputSpec(ndim=3)
        self.data_format = conv_utils.normalize_data_format(data_format)
        self.keepdims = keepdims

    def compute_output_shape(self, input_shape):
        input_shape = tf.TensorShape(input_shape).as_list()
        if self.data_format == 'channels_first':
            if self.keepdims:
                return (input_shape[0], input_shape[1], 1)
            else:
                return (input_shape[0], input_shape[1])
        else:
            if self.keepdims:
                return (input_shape[0], 1, input_shape[2])
            else:
                return (input_shape[0], input_shape[2])

    def call(self, inputs):
        raise NotImplementedError

    def get_config(self):
        config = {'data_format': self.data_format, 'keepdims': self.keepdims}
        base_config = super(GlobalPooling1D, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

class GlobalStdPooling1D(GlobalPooling1D):
    def __init__(self, data_format='channels_last', epsilon=1e-8, **kwargs):
        # 修正父类调用
        super(GlobalStdPooling1D, self).__init__(data_format=data_format,** kwargs)
        self.supports_masking = True
        self.epsilon = epsilon

    def call(self, inputs, mask=None):
        steps_axis = 1 if self.data_format == 'channels_last' else 2
        if mask is not None:
            mask = tf.cast(mask, inputs.dtype)
            mask = tf.expand_dims(mask, 2 if self.data_format == 'channels_last' else 1)
            inputs = inputs * mask
            sum_count = tf.reduce_sum(mask, axis=steps_axis, keepdims=True)
            mean = tf.reduce_sum(inputs, axis=steps_axis, keepdims=True) / tf.maximum(sum_count, 1)
            variance = tf.reduce_sum(tf.square(inputs - mean) * mask, axis=steps_axis, keepdims=True) / tf.maximum(sum_count, 1)
            std = tf.sqrt(variance + self.epsilon)
            if not self.keepdims:
                std = tf.squeeze(std, axis=steps_axis)
            return std
        else:
            # 加极小值防止方差为0出现NaN
            return backend.std(inputs, axis=steps_axis, keepdims=self.keepdims) + self.epsilon

    def get_config(self):
        config = super(GlobalStdPooling1D, self).get_config()
        config.update({'epsilon': self.epsilon})
        return config

问题2：Lambda层输出全为NaN #

错误原因 #

• 数值稳定性问题：backend.std默认使用无偏估计（分母为n-1），当序列长度为1时分母为0，或者方差趋近于0时开根号会出现NaN
• 未添加数值截断的极小epsilon参数

额外问题：原代码中输出维度为1却使用softmax激活，softmax会对所有维度输出做归一化，单个维度的softmax结果永远是1，无法训练，二分类场景请换成sigmoid激活+二元交叉熵损失。

修复后的Lambda层实现 #

def model():
    input_m = Input(shape = (1000, 750))
    con1d_m_5 = Conv1D(768, 5, activation='relu')(input_m)
    # 加入epsilon避免数值不稳定
    std_m_5 = Lambda(lambda x : backend.std(x, axis = 1, keepdims=False) + 1e-8)(con1d_m_5)
    output = Dense(1, activation='sigmoid')(std_m_5)

    model1 = Model(inputs = [input_m], outputs = [output])
    model1.compile(loss = 'binary_crossentropy', optimizer=Adam(1e-5))
    print(model1.summary())
    return model1

内容的提问来源于stack exchange，提问作者big_Z_0909