数据插补的选择、评估与报告：通用插补模型验证等技术问询

Hey there! Let's break this down step by step since you're already up to speed on model checking for multiple imputation (MI) but want to extend that to methods like random forest imputation and hotdeck, plus figure out how to pick the best model, evaluate performance, and report it properly in your work.

Random Forest Imputation #

Since random forest imputation leverages tree-based predictions, its checks tie closely to the model's inherent properties:

• Out-of-Bag (OOB) Error: Most implementations (like R's missForest package) calculate OOB error automatically—this is the error from predicting the subset of data not used to build each tree. For continuous variables, look at the mean squared error (MSE); for categorical variables, track the classification error rate. A stable, low OOB error means the model is making reliable predictions for missing values.
• Convergence Tracking: If you're using an iterative random forest imputer, plot the OOB error across iterations. Once the error stops decreasing (or fluctuates minimally), the model has converged—no need to keep iterating.
• Residual & Distribution Checks: For continuous variables, compute residuals (observed value - imputed value) for cases where you know the true value (either naturally complete or simulated missing). Check that residuals are randomly distributed around 0 with no obvious patterns. For categorical variables, use a confusion matrix to compare imputed categories against true values.

Hotdeck Imputation #

Hotdeck relies on matching "donor" cases (with complete data) to "recipient" cases (with missing values), so checks focus on match quality and distribution preservation:

• Match Quality: Calculate the standardized mean difference (SMD) between donors and recipients for each matching variable. An SMD < 0.1 indicates good balance—meaning donors and recipients are similar on the variables you used to match.
• Distribution Similarity: Compare the distribution of the imputed variable (mean, median, frequency counts, variance) to the distribution of the observed values. Use histograms, QQ-plots, or statistical tests (like the Kolmogorov-Smirnov test) to confirm they align closely.
• Donor Pool Sensitivity: Test different donor pool sizes (e.g., nearest 1 donor vs. nearest 5) and see how much the imputed values change. If results vary drastically, your donor pool choice is driving the outcome—you'll need to justify why you picked a specific size.

Your hunch about testing multiple methods is exactly right! Here's how to narrow it down:

• Predictive Accuracy Test: Use a "missing data simulation" approach: take a subset of your complete data, artificially set some values to missing, then use each imputation method to fill them in. Compare the imputed values to the true values using metrics like MSE (continuous) or accuracy (categorical). The method with the lowest error is a strong candidate.
• Preserve Data Relationships: Check if the imputation method maintains the original correlations and covariance between variables. For example, compare the correlation matrix of the original data to the correlation matrix of the imputed data. A method that keeps these relationships intact is better for downstream analyses (like regression).
• Stability Check: Run the imputation method multiple times (with different random seeds, if applicable) and look at key statistics (e.g., mean of an imputed variable, regression coefficients from your final model). If the results don't vary much, the method is stable.
• Computational Practicality: Don't overlook speed! For large datasets, random forest imputation can be slow, while hotdeck is often faster. Balance performance with how long you can wait for results.

Beyond the checks above, keep these in mind:

• Missing Mechanism Fit: If you suspect your data is MAR (missing at random), make sure your imputation method accounts for variables related to missingness (e.g., include those variables in random forest predictors or hotdeck matching). For MNAR (missing not at random), no method is perfect—be sure to report this uncertainty.
• Sensitivity Analysis: Run your final analysis with multiple imputation methods. If the conclusions (e.g., significant predictors, effect sizes) are consistent across methods, your results are robust. If they differ, report all outcomes and discuss the implications.

To make your work reproducible and transparent, include these details:

• Missing Data Overview: Report the missing rate for each variable, describe the missing data pattern (e.g., "30% of participants missing BMI, mostly those who skipped the physical exam"), and state your assumption about the missing mechanism (MAR/MCAR).
• Imputation Method Details:
  • For random forest: Specify the package used, number of trees, OOB error, and convergence criteria.
  • For hotdeck: List the matching variables, donor pool size, and whether you used sequential or random hotdeck.
• Model Checking Results: Share key performance metrics (e.g., "missForest achieved an OOB MSE of 2.3 for BMI, with residuals randomly distributed") and any sensitivity analysis findings.
• Uncertainty: If using multiple imputations (even with random forest), report the pooled statistics and their standard errors. For single-imputation methods like hotdeck, acknowledge the limitation and suggest that multiple imputations could be used to quantify uncertainty.

As you guessed, the core idea is to test multiple methods and select the one that minimizes predictive error, preserves data structure, and leads to stable, robust final results.

内容的提问来源于stack exchange，提问作者sma