如何基于额外数据训练k-NN模型以绘制MNIST数据集学习曲线

Nice work getting the MNIST data loaded! Drawing learning curves for k-NN is a great way to visualize how your model's performance improves (or plateaus) as you feed it more training data. Let's walk through the full process, starting with fixing your data prep and then building the curve step by step.

Step 1: Complete Data Preprocessing #

First, let's finish converting your MNIST data to numpy arrays and add normalization (critical for k-NN, since it relies on distance metrics):

import numpy as np
import matplotlib.pyplot as plt
from mnist import MNIST
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Load MNIST data
mndata = MNIST('./data')
images_train, labels_train = mndata.load_training()
images_test, labels_test = mndata.load_testing()

# Convert list data to numpy arrays
X_train = np.array(images_train)
y_train = np.array(labels_train)
X_test = np.array(images_test)
y_test = np.array(labels_test)

# Normalize pixel values to [0, 1] (k-NN performs much better with scaled features)
X_train = X_train / 255.0
X_test = X_test / 255.0

Step 2: Generate Learning Curve Data #

Unlike parametric models, k-NN doesn't "train" in the traditional sense—it just stores training data. For the learning curve, we'll test model performance on increasing sizes of training data:

# Define training set sizes to evaluate (adjust based on your computational power)
train_sizes = [100, 500, 1000, 2000, 5000, 10000, 20000]
train_accuracies = []
test_accuracies = []

# Choose a k value (start with 5, then tune later)
k = 5

for size in train_sizes:
    # Grab a subset of the training data
    X_subset = X_train[:size]
    y_subset = y_train[:size]
    
    # Initialize and fit k-NN (n_jobs=-1 uses all CPU cores for speed)
    knn = KNeighborsClassifier(n_neighbors=k, n_jobs=-1)
    knn.fit(X_subset, y_subset)
    
    # Calculate accuracy on the training subset
    train_pred = knn.predict(X_subset)
    train_acc = accuracy_score(y_subset, train_pred)
    train_accuracies.append(train_acc)
    
    # Calculate accuracy on the test set
    test_pred = knn.predict(X_test)
    test_acc = accuracy_score(y_test, test_pred)
    test_accuracies.append(test_acc)
    
    print(f"Training size: {size} | Train Accuracy: {train_acc:.4f} | Test Accuracy: {test_acc:.4f}")

Step 3: Plot the Learning Curve #

Now let's visualize the results to see how performance scales with training data:

plt.figure(figsize=(10, 6))
plt.plot(train_sizes, train_accuracies, label='Training Accuracy', marker='o', color='blue')
plt.plot(train_sizes, test_accuracies, label='Test Accuracy', marker='s', color='orange')

plt.xlabel('Training Set Size')
plt.ylabel('Accuracy')
plt.title(f'Learning Curve for k-NN (k={k}) on MNIST')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

Key Notes for Better Results #

• Tune k: Try values like 3, 7, or 9 to see how the curve changes. Lower k may cause overfitting (high training accuracy, low test accuracy early on), while higher k is more generalized but slower.
• Random Sampling: Instead of taking the first N samples, randomly select subsets to avoid bias from ordered data:

  random_indices = np.random.choice(len(X_train), size, replace=False)
  X_subset = X_train[random_indices]
  y_subset = y_train[random_indices]
  

• Cross-Validation: For more reliable validation, split your training set into train/validation subsets instead of using the test set directly (use sklearn.model_selection.train_test_split).
• Performance: k-NN is slow on large datasets. If you want to test the full 60k training samples, expect longer runtime—stick to smaller sizes for quick iterations.

内容的提问来源于stack exchange，提问作者Fallen Apart