技术问询：神经网络中哪些层使用激活函数？输入层是否使用？

Great questions—let's break this down in plain terms, like we're chatting through a coffee break:

1. Which layers in a neural network use activation functions? #

• Hidden layers: Almost every hidden layer relies on an activation function. The core reason for stacking hidden layers is to learn complex, non-linear patterns in data, and functions like ReLU, GELU, Sigmoid, or Tanh are what introduce that critical non-linearity. Without them, even a deep neural network would behave just like a single linear model—totally wasting the power of depth.
• Output layers: Whether you use an activation here depends entirely on your task:
  • For multi-class classification: Softmax is standard, as it converts raw outputs into class probabilities that sum to 1.
  • For binary classification: Sigmoid is common, squashing outputs to a 0-1 range that represents the probability of the positive class.
  • For regression tasks (predicting continuous values): You might skip an activation entirely (using a "linear activation," which is just the identity function) or use ReLU if you need to enforce non-negative outputs.

2. Does the input layer use an activation function? #

Nope, the input layer does not use an activation function. Its only job is to pass your raw (or preprocessed) feature data straight to the first hidden layer. Activation functions exist to transform data as it moves through the model to capture complexity—there’s no need to apply that transformation to your initial input values.

内容的提问来源于stack exchange，提问作者user8262209