如何将tf.Variable转为Keras张量以实现两层权重自定义正则化

The error you're seeing happens because you're passing raw tf.Variable objects (the layer weights) directly as inputs to a Keras Layer. Keras tensors need a _keras_history attribute to track their place in the model graph, which regular Variables don't have. Here's how to fix this properly:

Solution: Add Regularization Loss Directly to the Model #

Instead of creating a separate Layer that takes weights as inputs, compute the regularization term directly in your model function and add it to the model using model.add_loss(). This avoids the need to pass Variables as inputs entirely.

Here's the modified code:

import tensorflow as tf
from tensorflow.keras.layers import Input, Flatten, Dense, Layer
from tensorflow.keras.models import Model
from tensorflow.keras import backend as K

def model():
    input1 = Input(shape=(224, 224, 3))
    input2 = Input(shape=(224, 224, 3))
    inp1 = Flatten()(input1)
    inp2 = Flatten()(input2)
    
    layer1 = Dense(1024, activation="sigmoid")
    x1_1 = layer1(inp1)
    x2_1 = layer1(inp2)
    
    layer2 = Dense(1024, activation="sigmoid")
    x1_2 = layer2(inp1)
    x2_2 = layer2(inp2)
    
    # Calculate your custom regularization term
    layer1_wt = layer1.trainable_weights[0]
    layer2_wt = layer2.trainable_weights[0]
    reg = K.dot(K.transpose(layer1_wt), layer2_wt)
    reg_norm = K.sqrt(K.sum(K.square(reg)))
    
    # Add the regularization loss to the model
    model = Model([input1, input2], [x1_2, x2_2])
    model.add_loss(reg_norm)
    
    return model

if __name__ == "__main__":
    m = model()
    m.summary()  # This should work without errors now

Key Changes Explained: #

1. Removed the CustomRegularization Layer: We don't need it anymore because we're adding the loss directly to the model.
2. Compute Regularization in the Model: We calculate the norm of the dot product of the two layer weights right in the model function.
3. Use model.add_loss(): This method accepts tensor expressions involving trainable Variables and adds them to the model's total loss during training. No need to pass Variables as inputs to layers.

Alternative: Reusable Custom Regularization Layer #

If you want to keep the regularization logic encapsulated in a reusable layer, you can modify it to take the target layers as initialization arguments instead of weight inputs:

class CustomRegularization(Layer):
    def __init__(self, target_layer1, target_layer2, **kwargs):
        super(CustomRegularization, self).__init__(**kwargs)
        self.target_layer1 = target_layer1
        self.target_layer2 = target_layer2
    
    def call(self, inputs):
        # Access weights directly from the target layers
        layer1_wt = self.target_layer1.trainable_weights[0]
        layer2_wt = self.target_layer2.trainable_weights[0]
        reg = K.dot(K.transpose(layer1_wt), layer2_wt)
        reg_norm = K.sqrt(K.sum(K.square(reg)))
        self.add_loss(reg_norm)
        # Return the input unchanged (acts as a pass-through layer)
        return inputs

def model():
    input1 = Input(shape=(224, 224, 3))
    input2 = Input(shape=(224, 224, 3))
    inp1 = Flatten()(input1)
    inp2 = Flatten()(input2)
    
    layer1 = Dense(1024, activation="sigmoid")
    x1_1 = layer1(inp1)
    x2_1 = layer1(inp2)
    
    layer2 = Dense(1024, activation="sigmoid")
    x1_2 = layer2(inp1)
    x2_2 = layer2(inp2)
    
    # Apply the reusable regularization layer (pass any model tensor as input)
    x1_2_reg = CustomRegularization(layer1, layer2)(x1_2)
    
    model = Model([input1, input2], [x1_2_reg, x2_2])
    return model

This approach keeps your regularization logic modular while avoiding the _keras_history error, since we're not passing Variables as inputs—instead, we reference the layers directly in the custom layer.

内容的提问来源于stack exchange，提问作者Skand Vishwanath Peri