能否用反向传播拟合非神经网络模型？以泰勒多项式为例探讨

Great question—this cuts to the heart of what backpropagation actually is, and why it’s so closely tied to neural networks. Let’s break this down:

First: Backpropagation Isn’t Exclusive to Neural Networks #

At its core, backpropagation is just an efficient way to compute gradients using the chain rule for complex computational graphs. There’s nothing inherent to it that restricts it to neural networks. You absolutely could model a Taylor polynomial as a computational graph, compute gradients via backpropagation, and optimize its coefficients ($c_0, c_1, ..., c_n$) using gradient descent or another iterative optimizer.

So Why Don’t We See This Done Often? #

The answer boils down to practicality and the unique properties of the models involved:

• Taylor polynomials have closed-form solutions
  For fitting a Taylor polynomial to data, we don’t need iterative methods like gradient descent. We can solve for the coefficients directly using linear least squares (since the Taylor polynomial is a linear combination of basis functions $(x-a)^k$). This gives us an exact solution in one step, which is far faster and more precise than iterating with backpropagation + gradient descent. Why use a sledgehammer when a nail gun works better?

• Neural networks require iterative optimization
  Neural networks are composed of nested nonlinear transformations (e.g., activation functions layered on top of linear weights). There’s no closed-form solution for their parameters—we can’t just solve a linear system to find optimal weights. Backpropagation’s superpower is that it efficiently computes gradients for thousands (or millions) of interdependent parameters in these deep, nested structures, making iterative optimization feasible.

• Taylor polynomials are limited to local approximation
  Taylor polynomials are designed to approximate functions locally around a point $a$. For global function fitting, high-degree Taylor polynomials suffer from severe overfitting (the Runge phenomenon) and are rarely used in practice. Neural networks, by contrast, excel at modeling complex, non-local nonlinear patterns—so the use case for backpropagation here is far more common.

When Would You Use Backpropagation for a Taylor Polynomial? #

That said, there are edge cases where this makes sense:

• If you want to add regularization (e.g., L2 penalty on coefficients) to prevent overfitting in high-degree polynomials. While regularized least squares still has a closed-form solution, using backprop + gradient descent lets you experiment with custom regularization terms more flexibly.
• If your dataset has noisy or non-standard constraints that make the closed-form solution less ideal (e.g., weighted least squares with non-uniform weights).

Wrap-Up #

Backpropagation is a general tool for gradient computation, not a neural network-specific trick. It’s just that neural networks are the type of model where this tool provides the most value—since they lack closed-form solutions and have complex parameter dependencies. For Taylor polynomials and other models with simpler, solvable structures, we tend to use more efficient methods instead.

内容的提问来源于stack exchange，提问作者Edv Beq