使用sklearn.cluster.DBSCAN(metric='russellrao')的正确数据格式咨询

Great question! Let’s break down exactly what’s happening here, and get you past that all--1 result (all points marked as noise) you’re seeing.

First, let’s clarify the core requirement for the russellrao metric:

• It’s exclusively designed for binary data—that means your input matrix must only contain 0s and 1s, where each feature represents a binary attribute (like "present/absent", "yes/no").
• In scikit-learn, the russellrao distance is calculated as:
  distance = 1 - (number of features where both samples are 1) / (total number of features)
  So the distance ranges from 0 (perfect match, all overlapping 1s) to 1 (no overlapping 1s at all).

Why are you getting all -1s? #

The default DBSCAN parameter eps=0.5 is tuned for Euclidean distance, not russellrao. If every pair of samples in your data has a russellrao distance greater than 0.5, DBSCAN can’t find any core samples (samples with enough neighbors within eps), so every point gets labeled as noise (-1).

Step-by-Step Fix & Correct Data Format #

1. Ensure your data is a binary matrix #

Your input npy = df2.values must be strictly 0s and 1s. If your original data isn’t binary, use Binarizer to convert it:

from sklearn.preprocessing import Binarizer
# Adjust the threshold based on your data (e.g., convert values >0.5 to 1, others to 0)
binarizer = Binarizer(threshold=0.5)
binary_data = binarizer.fit_transform(df2.values)

2. Analyze your distance distribution #

Before running DBSCAN, check the actual range of russellrao distances between your samples to pick a sensible eps:

from sklearn.metrics.pairwise import pairwise_distances
distance_matrix = pairwise_distances(binary_data, metric="russellrao")
print(f"Minimum distance: {distance_matrix.min()}")
print(f"Maximum distance: {distance_matrix.max()}")
print(f"Average distance: {distance_matrix.mean()}")

For example, if your average distance is 0.7, try setting eps=0.8—this will include more sample pairs in each other’s neighborhoods.

3. Tune DBSCAN parameters and re-run #

Use your cleaned binary data and adjusted parameters:

# Adjust eps based on your distance analysis, and min_samples based on dataset size
y_pred = DBSCAN(metric="russellrao", eps=0.8, min_samples=5).fit_predict(binary_data)

Note: min_samples controls how many neighbors a point needs to be a core sample—too small and you’ll get tiny, meaningless clusters; too large and you’ll end up back with all noise.

Common Pitfalls to Avoid #

• Don’t use non-binary data: russellrao doesn’t make sense with continuous values or discrete non-0/1 data—this will lead to invalid distance calculations that break DBSCAN.
• Don’t rely on default eps: The default eps=0.5 is for Euclidean distance, which has a completely different scale than russellrao (0-1 range). Always tune eps based on your actual distance distribution.

内容的提问来源于stack exchange，提问作者Lucas