如何获取数据表列间相关性、识别高度相关列并绘制热力图

Hey there! No worries at all about being a forum newbie—we all started somewhere, and your question is totally clear 😊 Let's walk through each step you need, using your DataFrame mc:

Step 1: Calculate Correlation Between All Columns #

First, we'll use pandas' built-in corr() method to generate a correlation matrix. This matrix shows the Pearson correlation coefficient (ranging from -1 to 1) between every pair of numerical columns:

# Calculate the correlation matrix
correlation_matrix = mc.corr()

Note: This only works for numerical columns. If your DataFrame has non-numeric columns, filter them out first with:
mc_numeric = mc.select_dtypes(include=['number'])
Then use mc_numeric.corr() instead.

Step 2: Identify the Column Pair with the Highest Correlation #

The correlation matrix has 1s along the diagonal (each column is perfectly correlated with itself), so we need to exclude those to find the strongest cross-column correlation:

# Unstack the matrix to get all pairwise correlations
corr_pairs = correlation_matrix.unstack()

# Remove self-correlations (where value is 1)
corr_pairs = corr_pairs[corr_pairs != 1]

# Find the pair with the highest absolute correlation
highest_corr_pair = corr_pairs.abs().idxmax()
highest_corr_value = corr_pairs.loc[highest_corr_pair]

# Print the result
print(f"Columns with the strongest correlation: {highest_corr_pair}")
print(f"Correlation coefficient: {round(highest_corr_value, 4)}")

This will give you the exact column names and their correlation score—positive means they move together, negative means they move in opposite directions.

Step 3: Plot a Correlation Heatmap #

To visualize the correlations clearly, we'll use seaborn's heatmap function (you'll need to have seaborn and matplotlib installed first):

import seaborn as sns
import matplotlib.pyplot as plt

# Set up the plot size
plt.figure(figsize=(10, 8))

# Create the heatmap with annotations, color scale, and formatted values
sns.heatmap(
    correlation_matrix,
    annot=True,       # Show correlation values on the heatmap
    cmap='coolwarm',  # Color scale from blue (negative) to red (positive)
    fmt='.2f',        # Format values to 2 decimal places
    linewidths=0.5    # Add lines between cells for readability
)

# Add a title and display the plot
plt.title('Correlation Heatmap of Data Columns', fontsize=14)
plt.show()

The heatmap makes it easy to spot patterns at a glance—darker reds mean strong positive correlations, darker blues mean strong negative correlations.

Feel free to ask if you run into any issues with the code, or if you need clarification on any part!

内容的提问来源于stack exchange，提问作者tpt369