使用Python评估A/B测试结果及相关假设检验问题

Hey folks, let's walk through how to work with conversion rate calculations and hypothesis testing for your A/B test results—now that you've collected data on user conversions for both site variants.

转化率计算公式 #

First, let's define our terms clearly:

• Control group: Total users = c_t, Converted users = c_n
• Variant group: Total users = v_t, Converted users = v_n

The conversion rates for each group are calculated as:
$$
\mathrm{CR}_c = \frac{c_n}{c_t} \quad \mathrm{CR}_v = \frac{v_n}{v_t}
$$

常见的假设检验类型 #

Depending on what you're trying to prove, you'll choose one of these null hypothesis frameworks:

双侧检验 #

This is for when you want to check if there's any significant difference between the two groups (no matter which performs better):

• Null hypothesis: $H_0: \mathrm{CR}_c = \mathrm{CR}_v$
• Alternative hypothesis: $H_a: \mathrm{CR}_c \neq \mathrm{CR}_v$

单侧检验（检验控制组是否更优） #

The original text cuts off here, but this is the standard complementary setup for the given null hypothesis:

• Null hypothesis: $H_0: \mathrm{CR}_c > \mathrm{CR}_v$
• Typical alternative hypothesis (to complete the test): $H_a: \mathrm{CR}_c \leq \mathrm{CR}_v$

单侧检验（检验变体组是否更优） #

Another common directional test focuses on whether the variant outperforms the control:

• Null hypothesis: $H_0: \mathrm{CR}_c \geq \mathrm{CR}_v$
• Alternative hypothesis: $H_a: \mathrm{CR}_c < \mathrm{CR}_v$

Once you've selected the right test, you can use methods like the z-test or chi-squared test to compute the p-value and decide whether to reject the null hypothesis based on your chosen significance level (usually $\alpha = 0.05$).

内容的提问来源于stack exchange，提问作者Dror Atariah