Hey there! Let's tackle your two questions around evaluating clustering accuracy for mixed-data algorithms like K-Prototypes and selecting the right validity metrics—this is a really common challenge when working with both numerical and categorical features, so I’m glad you asked.

The approach depends entirely on whether you have ground truth labels for your data:

• With labeled data (supervised validation)
  You can use agnostic metrics that only care about how well clustering results match the true groups, not the data type itself. Top choices include:

  • Adjusted Rand Index (ARI): Fixes the bias of the raw Rand Index (which tends to score higher as the number of clusters increases). It ranges from -1 (worse than random) to 1 (perfect match), and it’s my go-to for labeled mixed data.
  • Normalized Mutual Information (NMI): Measures the mutual information between clusters and true labels, normalized to avoid bias from cluster count. Ranges from 0 (no overlap) to 1 (perfect alignment).
  • Both metrics work seamlessly because they only compare the grouping of samples, not the underlying feature types.

• Without labeled data (unsupervised validation)
  You’ll need to use internal metrics that assess cluster tightness (how similar samples are within a cluster) and separation (how distinct clusters are from each other). For K-Prototypes, this means adapting metrics to use the algorithm’s combined distance function:
  d = w * euclidean(numeric_features) + (1-w) * hamming(categorical_features)
  where w is the weight balancing numerical and categorical distances. We’ll dive deeper into these adapted metrics in the next section.

Most standard metrics are built for numerical data, but you can adapt or use specialized metrics for mixed data. Here’s how to pick and apply them:

External Metrics (With Ground Truth) #

Stick to the label-agnostic ones I mentioned above:

• Prioritize ARI over raw Rand Index—it’s more reliable for datasets with varying cluster counts.
• NMI is great if you want to focus on information overlap between clusters and true labels, especially when clusters are imbalanced.

Internal Metrics (No Ground Truth) #

Adapt classic numerical metrics to work with mixed data, or use categorical-specific ones:

• Mixed-data Silhouette Coefficient: Replace the standard Euclidean distance with K-Prototypes’ combined distance. For each sample, calculate how similar it is to its own cluster (a) vs. the nearest other cluster (b). The silhouette score is (b - a)/max(a,b). Average this across all samples—scores close to 1 mean well-separated clusters, while scores near 0 mean overlapping clusters.
• Mixed-data Davies-Bouldin Index (DBI): Again, use the combined distance to compute average within-cluster distance and between-cluster distance. DBI is the average ratio of within-cluster distance to between-cluster distance for each pair of clusters. Lower scores mean better clustering (tight, separated clusters).
• Entropy-based Cluster Homogeneity: For categorical features, calculate the entropy of each cluster’s categorical distribution. Lower entropy means the cluster has more consistent categorical values. You can combine this with numerical within-cluster variance to get a single composite score (e.g., weight variance and entropy based on feature importance).
• Chi-Square Test for Cluster Separation: Use the chi-square test to check if categorical feature distributions differ significantly between clusters. A high chi-square statistic (with a low p-value) means clusters have distinct categorical profiles, which is a sign of good clustering.

Practical Tips #

• Combine multiple metrics: Don’t rely on just one. For example, pair a silhouette score with DBI—if both point to good clustering, you can be more confident.
• Tune distance weights: If your categorical features are more important, adjust the w parameter in K-Prototypes’ distance function, and make sure your validity metrics use the same weight.
• Check convergence: For K-Prototypes, monitor the training loss (sum of combined distances from samples to their cluster centers). If the loss plateaus, it’s a sign the algorithm has found stable clusters—this is a simple but effective sanity check.

内容的提问来源于stack exchange，提问作者Jack shephard