如何在Keras中实现节点连接任意的前馈神经网络？

Hey Oscar, great question! You’re absolutely right that the Sequential API (and the older Graph class, which is now deprecated) defaults to full connectivity between consecutive layers. But don’t worry—Keras has a far more flexible tool for building custom network topologies: the Functional API. It’s designed exactly for cases like yours where you need non-standard connections, partial layer links, or skip connections.

Let’s break down how to tackle both your use cases:

1. Building Non-Fully Connected Feedforward Layers #

If you want to exclude specific connections between neurons (e.g., the first node of layer 2 doesn’t connect to the second node of layer 3), you have two straightforward approaches:

Option 1: Use a Custom Weight Constraint #

You can define a constraint that masks out specific weight values, ensuring they stay at 0 and aren’t updated during training. Here’s how:

from keras.layers import Input, Dense
from keras.models import Model
from keras.constraints import Constraint
import tensorflow as tf

# Define a custom constraint to mask specific connections
class ConnectionMask(Constraint):
    def __init__(self, mask_matrix):
        self.mask = mask_matrix
    
    def __call__(self, weights):
        # Multiply weights by the mask to zero out unwanted connections
        return weights * self.mask

# Step 1: Define input layer
input_layer = Input(shape=(10,))

# Step 2: First hidden layer (fully connected)
hidden_layer_1 = Dense(3, activation='relu')(input_layer)

# Step 3: Create a mask for the second hidden layer
# Mask shape: (num_neurons_in_prev_layer, num_neurons_in_current_layer)
# We'll zero out the connection from hidden_layer_1's 1st neuron to hidden_layer_2's 2nd neuron
connection_mask = tf.constant([
    [1.0, 0.0],  # Allow connection to 1st neuron, block to 2nd
    [1.0, 1.0],  # Allow both connections
    [1.0, 1.0]   # Allow both connections
], dtype=tf.float32)

# Step 4: Second hidden layer with the custom mask constraint
hidden_layer_2 = Dense(
    2,
    activation='relu',
    kernel_constraint=ConnectionMask(connection_mask)
)(hidden_layer_1)

# Step 5: Output layer
output_layer = Dense(1, activation='sigmoid')(hidden_layer_2)

# Build and compile the model
model = Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='binary_crossentropy')

Option 2: Manually Modify Layer Weights #

If you prefer a more direct approach, you can initialize the layer’s weights, zero out the unwanted connections, and freeze those specific weights to prevent updates:

# After defining hidden_layer_2 (before connecting it to the input)
hidden_layer_2 = Dense(2, activation='relu')
# Get the initial kernel weights (shape: 3x2)
kernel_weights = hidden_layer_2.get_weights()[0]
# Zero out the unwanted connection
kernel_weights[0, 1] = 0.0
# Reassign the weights
hidden_layer_2.set_weights([kernel_weights, hidden_layer_2.get_weights()[1]])
# Freeze the kernel weights so the masked connection stays zero
hidden_layer_2.trainable_weights[0].assign(kernel_weights)

# Now connect the layer to hidden_layer_1
hidden_layer_2 = hidden_layer_2(hidden_layer_1)

2. Adding Skip (Jump) Connections #

Skip connections (like those in ResNets) are trivial to implement with the Functional API—you just connect layers directly to non-consecutive layers, then combine their outputs (via concatenation, addition, etc.). Here are two common examples:

Example 1: Concatenated Skip Connection #

Connect the input layer directly to a later hidden layer by concatenating their outputs:

from keras.layers import concatenate

# Reuse the input_layer and hidden_layer_2 from earlier
# Add a skip connection: concatenate input layer with hidden_layer_2
concat_skip = concatenate([input_layer, hidden_layer_2])
# Output layer uses the concatenated tensor
output_layer = Dense(1, activation='sigmoid')(concat_skip)

model = Model(inputs=input_layer, outputs=output_layer)

Example 2: Residual Skip Connection (Addition) #

For layers with matching output shapes, you can add their outputs directly (this is the core of ResNet architectures):

from keras.layers import add

# Define a hidden layer with the same output shape as hidden_layer_1
hidden_layer_2 = Dense(3, activation='relu')(hidden_layer_1)
# Add the original hidden_layer_1 output to hidden_layer_2's output
residual_connection = add([hidden_layer_1, hidden_layer_2])
# Pass the combined output to the final layer
output_layer = Dense(1, activation='sigmoid')(residual_connection)

model = Model(inputs=input_layer, outputs=output_layer)

A quick note: The old Graph class is no longer recommended for new projects—Functional API is the standard for custom models in modern Keras (and TensorFlow Keras).

内容的提问来源于stack exchange，提问作者Oscar Gabriel Reyes Pupo