是否存在神经网络可100%分类而RBF SVM无法实现的场景及2D两类数据案例

Let's break down your two questions with clear, concrete explanations to make things easy to follow.

1. Are there scenarios where a neural network can hit 100% accuracy, but an RBF SVM can't? #

Absolutely. The core difference comes down to the types of functions each model can represent:

• RBF SVMs live in a Reproducing Kernel Hilbert Space (RKHS) defined by the RBF kernel. Functions in this space are inherently continuous and have a built-in "smoothness" constraint—they can't capture wildly erratic, non-smooth patterns.
• Neural networks (even shallow ones with enough hidden units, or deep networks) have a far larger hypothesis space. Thanks to the Universal Approximation Theorem, they can approximate nearly any function (including discontinuous or infinitely oscillating ones) to arbitrary precision, as long as you give them enough parameters to work with.

A perfect example is a classification task where the decision boundary oscillates infinitely. Think of classifying points based on the sign of sin(1/x) for x ∈ (0,1] (with x=0 assigned to one class). As x gets closer to 0, this function bounces between positive and negative infinitely many times. Since RBF SVMs can only model continuous functions, they'll never perfectly capture this boundary—tweaking kernel parameters like gamma or C won't fix it, and there will always be misclassifications. A neural network, though, can be sized to learn these tiny, rapid oscillations and nail 100% accuracy on a dense set of points.

2. Is there a 2D two-class dataset that a neural network can classify perfectly, but an RBF SVM can't? #

Yep, and we can adapt the previous example to 2D to make it tangible. Here's how to build such a dataset:

• Class 1: All points (x, y) where x ∈ (0, 1], y ∈ [0, 1], and sin(1/x) > 0, plus every point (0, y) for y ∈ [0, 1].
• Class 2: All points (x, y) where x ∈ (0, 1], y ∈ [0, 1], and sin(1/x) < 0.

Why does this work?

• A sufficiently large neural network (like a shallow MLP with enough hidden units) can learn the infinitely wiggly boundary between the two classes. It can model the rapid shifts near x=0 and correctly classify every point in the dataset.
• An RBF SVM, however, is stuck with continuous decision boundaries. The true boundary here has infinitely many crossings near x=0, which no continuous function can replicate perfectly. No matter how you adjust the RBF kernel's width or regularization, the SVM's boundary will be a smooth approximation that misses some oscillations, leading to unavoidable mistakes.

Note that this is an edge case—for most real-world datasets with smooth boundaries, RBF SVMs often perform on par with neural networks. But these examples highlight the fundamental limits of what each model can represent.

内容的提问来源于stack exchange，提问作者user6250685