在含3个隐藏层的DNN中，TensorFlow环境下能否仅用一个W作为权重？

Great question! Let’s break this down clearly: while what you’re proposing is technically feasible to code, it’s strongly not recommended for practical deep learning workflows. Here’s the breakdown:

1. It’s technically possible… but clunky #

You could theoretically define one large tensor W and slice out sub-tensors for each layer’s weights. For example, if your network has an input layer (784 units), 3 hidden layers (256, 128, 64 units), and an output layer (10 units), you could shape W to cover all layer weight dimensions, then use tensor indexing to extract the parts needed for each layer:

import tensorflow as tf

input_dim = 784
h1, h2, h3, output_dim = 256, 128, 64, 10

# Define one giant weight tensor
total_rows = input_dim + h1 + h2 + h3
total_cols = h1 + h2 + h3 + output_dim
W = tf.Variable(tf.random.normal((total_rows, total_cols)))

def forward(x):
    # Slice W for layer 1: input -> h1
    w1 = W[:input_dim, :h1]
    x = tf.nn.relu(tf.matmul(x, w1) + tf.Variable(tf.zeros(h1)))
    # Slice W for layer 2: h1 -> h2
    w2 = W[input_dim:input_dim+h1, h1:h1+h2]
    x = tf.nn.relu(tf.matmul(x, w2) + tf.Variable(tf.zeros(h2)))
    # Slice W for layer 3: h2 -> h3
    w3 = W[input_dim+h1:input_dim+h1+h2, h1+h2:h1+h2+h3]
    x = tf.nn.relu(tf.matmul(x, w3) + tf.Variable(tf.zeros(h3)))
    # Slice W for output layer: h3 -> output
    w_out = W[input_dim+h1+h2:, h1+h2+h3:]
    return tf.matmul(x, w_out) + tf.Variable(tf.zeros(output_dim))

This code will run without errors, but that’s where the positives end.

2. The core problem: broken gradient updates #

TensorFlow’s automatic differentiation tracks gradients per variable, not per slice of a variable. When you use different parts of W for different layers, all the gradients from each layer will be combined into a single gradient tensor for the entire W during backpropagation.

This means updates to the "W1 slice" will interfere with updates to the "W2 slice" and vice versa—completely breaking the independent weight learning that makes deep neural networks work. Your model will likely never converge, or will converge to a terrible, unstable solution.

3. Other practical downsides #

• Unmaintainable code: You’ll have to manually track slice indices for every layer. Changing even one layer’s size means recalculating the entire W shape and all slice positions—easy to mess up, hard to debug.
• No access to framework best practices: TensorFlow’s high-level APIs (like tf.keras.layers.Dense) handle weight initialization, regularization, and training logic automatically. Using a single W forces you to hand-write all this, leading to redundant, error-prone code.
• No efficiency gains: A single large tensor uses the same total memory as multiple smaller tensors (since parameter count is identical). Slicing operations may even add overhead, especially on GPUs where data continuity matters for speed.

Final Recommendation #

Stick with separate weight variables (W1, W2, W3). This is the standard approach in deep learning—frameworks are designed around it, it’s easier to debug/maintain, and it ensures your model trains correctly.

内容的提问来源于stack exchange，提问作者chiu0619