K近邻插补（KNN Imputation）原理、文档及K值选择咨询

Hey there! Since you already have a solid grasp of KNN classifiers, KNN imputation will feel familiar—it’s just a logical extension of that core idea, repurposed to fill in missing values instead of predicting categories. Let’s break this down clearly:

At its heart, KNN imputation relies on the same intuition as KNN classification: similar samples tend to have similar feature values. Here’s the step-by-step process for filling a missing value in a specific feature for a target sample:

1. Calculate distances: For the sample with a missing value, compute its distance to every other sample that has a complete set of values for the relevant features. Common distance metrics include Euclidean (for continuous data) or Manhattan (for ordinal/count data). Critically, when calculating distance, you only use features that have valid values in both the target sample and the comparison sample—this avoids skewing the distance with missing data.
2. Select nearest neighbors: Pick the top K samples with the smallest distances to the target sample.
3. Impute the missing value:
   • For numerical features: Use the mean or median of the K nearest neighbors’ values for that feature (median is better if the data has outliers).
   • For categorical features: Use the mode (most frequent value) among the K neighbors’ values for that feature.

You’re right to ask about K selection—there’s overlap with KNN classification, but some key differences too:

Similarities #

• Avoid extreme K values: A tiny K (like 1) makes your imputation highly sensitive to outliers or noise in individual samples (overfitting). A huge K (like half your dataset) will smooth out local patterns too much, leading to imputed values that don’t reflect the target sample’s true group.
• Cross-validation is your friend: The best way to pick K is to use a validation set. Take a subset of your complete data, artificially introduce missing values, then test different K values. Choose the K that gives the smallest imputation error (e.g., RMSE for numerical features, accuracy for categorical).

Key Differences #

• Impact of K: In KNN classification, K directly changes the decision boundary between classes. In KNN imputation, K affects how "local" or "global" your imputed values are: smaller K leans into local group patterns, larger K averages across broader segments of your data.
• Missing data context: If your dataset has a high proportion of missing values, you might need a larger K to ensure you have enough complete neighbor samples to draw from—though you still have to balance this against losing local relevance.

For a quick concrete example: Suppose you have a customer sample with a missing "age" value, but valid values for "income" ($50k), "education" (bachelor’s), and "location" (urban). You pick K=3 nearest neighbors, whose ages are 28, 30, and 32. You’d impute the missing age as 30 (the mean) for this sample.

内容的提问来源于stack exchange，提问作者Yohann L.