针对响应率数据的3-Way ANOVA适配方案及替代统计测试咨询

Great question! Let’s work through this together since your aggregated response rate data doesn’t fit the standard assumptions of 3-way ANOVA (which expects individual-level, independent observations). Here are your most robust options in Python:

Why Standard 3-Way ANOVA Isn’t Ideal #

ANOVA assumes your dependent variable is a continuous, normally distributed measure from individual respondents. Your response rate is a summary statistic (proportion) tied to group sizes, so using it directly in ANOVA would ignore the variation in sample sizes across groups and violate key assumptions, leading to unreliable results.

Option 1: Expand to Individual-Level Data & Use Logistic Regression #

If your total population size isn’t prohibitively large, you can expand your aggregated data into individual rows (each representing one person: 1 for a responder, 0 for non-responder). This lets you use logistic regression, which is designed for binary outcomes and can test main effects and interactions just like ANOVA.

Python Code Example: #

import pandas as pd
from statsmodels.formula.api import logit

# Your original aggregated data
raw_data = pd.DataFrame({
    'HairColor': ['Brown', 'Brown', 'Brown', 'Brown', 'Black', 'Black', 'Black', 'Black'],
    'EyeColor': ['Blue', 'Blue', 'Green', 'Green', 'Blue', 'Blue', 'Green', 'Green'],
    'ShoeSize': [12, 13, 12, 13, 12, 13, 12, 13],
    'Respondants': [3, 4, 5, 8, 2, 5, 2, 4],
    'PopulationSize': [10, 20, 5, 10, 20, 10, 10, 20]
})

# Expand into individual-level observations
individual_data = []
for _, row in raw_data.iterrows():
    # Add rows for responders (Response = 1)
    individual_data.extend([{
        'HairColor': row['HairColor'],
        'EyeColor': row['EyeColor'],
        'ShoeSize': row['ShoeSize'],
        'Response': 1
    }] * row['Respondants'])
    # Add rows for non-responders (Response = 0)
    non_responders = row['PopulationSize'] - row['Respondants']
    individual_data.extend([{
        'HairColor': row['HairColor'],
        'EyeColor': row['EyeColor'],
        'ShoeSize': row['ShoeSize'],
        'Response': 0
    }] * non_responders)

individual_df = pd.DataFrame(individual_data)

# Fit logistic regression with 3-way interaction
model = logit("Response ~ HairColor * EyeColor * ShoeSize", data=individual_df).fit()
print(model.summary())

• Check the p-values in the summary to determine if main effects (HairColor, EyeColor, ShoeSize) or their interactions are statistically significant.

Option 2: Use Generalized Linear Models (GLM) with Binomial Family (No Data Expansion) #

If expanding data isn’t feasible (e.g., huge population sizes), use a GLM with a binomial distribution. This directly uses your aggregated counts (respondents vs. total population) and avoids expanding rows entirely—this is the most efficient approach for your data.

Python Code Example: #

import pandas as pd
from statsmodels.formula.api import glm
from statsmodels.genmod.families import Binomial

# Your original aggregated data (same as above)
raw_data = pd.DataFrame({
    'HairColor': ['Brown', 'Brown', 'Brown', 'Brown', 'Black', 'Black', 'Black', 'Black'],
    'EyeColor': ['Blue', 'Blue', 'Green', 'Green', 'Blue', 'Blue', 'Green', 'Green'],
    'ShoeSize': [12, 13, 12, 13, 12, 13, 12, 13],
    'Respondants': [3, 4, 5, 8, 2, 5, 2, 4],
    'PopulationSize': [10, 20, 5, 10, 20, 10, 10, 20]
})

# Fit GLM with binomial distribution (logit link)
model = glm(
    formula="Respondants ~ HairColor * EyeColor * ShoeSize",
    data=raw_data,
    family=Binomial(link='logit'),
    weights=raw_data['PopulationSize']
).fit()

print(model.summary())

• This model treats each group as a binomial trial (success = response, trials = population size). The results are statistically equivalent to the expanded logistic regression but run much faster for large datasets.

Option 3: Weighted Least Squares (WLS) with Logit Transformation #

If you prefer a linear model framework (similar to ANOVA), you can transform your response rate to a logit scale (to handle proportional data) and use weighted least squares, where weights are tied to group size (to account for variance in response rates).

Python Code Example: #

import pandas as pd
import numpy as np
from statsmodels.formula.api import wls

raw_data = pd.DataFrame({
    'HairColor': ['Brown', 'Brown', 'Brown', 'Brown', 'Black', 'Black', 'Black', 'Black'],
    'EyeColor': ['Blue', 'Blue', 'Green', 'Green', 'Blue', 'Blue', 'Green', 'Green'],
    'ShoeSize': [12, 13, 12, 13, 12, 13, 12, 13],
    'Respondants': [3, 4, 5, 8, 2, 5, 2, 4],
    'PopulationSize': [10, 20, 5, 10, 20, 10, 10, 20]
})

# Calculate response rate and logit-transform it (add small epsilon to avoid 0/1 issues)
raw_data['ResponseRate'] = raw_data['Respondants'] / raw_data['PopulationSize']
raw_data['LogitResponse'] = np.log((raw_data['ResponseRate'] + 1e-6) / (1 - raw_data['ResponseRate'] + 1e-6))

# Fit WLS model with group size as weights
model = wls(
    formula="LogitResponse ~ HairColor * EyeColor * ShoeSize",
    data=raw_data,
    weights=raw_data['PopulationSize']
).fit()

print(model.summary())

• The logit transformation converts proportions to a continuous scale that’s closer to normally distributed, and weights ensure groups with larger populations have more influence on the model.

Final Recommendation #

Skip standard ANOVA entirely—your data is proportional, not continuous, so logistic regression (expanded data) or binomial GLM (aggregated data) are statistically appropriate choices. The binomial GLM is the most efficient, especially for large datasets.

内容的提问来源于stack exchange，提问作者user19154333