配对设计下5点Likert item得分差异显著性检验咨询

Alright, let's walk through how to tackle this problem—since every student rated both lectures, we're dealing with paired data (same participants providing two responses), which changes the statistical tests we should use compared to independent groups. Here are the most common and appropriate methods, ordered by practicality:

1. Wilcoxon Signed-Rank Test (Nonparametric, Preferred for Likert Data) #

Likert scales are ordinal (not truly continuous), so nonparametric tests are often the safer bet because they don't assume normality of the data. The Wilcoxon Signed-Rank Test is the go-to paired alternative to the t-test here:

• What it tests: Whether the median difference between Lecture A and Lecture B scores is significantly different from 0.
• How it works:
  • Calculate the difference (Score A - Score B) for each student.
  • Ignore any differences that are exactly 0.
  • Rank the absolute values of these differences.
  • Sum the ranks for positive differences and negative differences, then compare these sums to determine if one group is consistently higher/lower.
• Example R code:

  # Assume you have vectors scoreA and scoreB with the 1-5 Likert values
  wilcox.test(scoreA, scoreB, paired = TRUE, alternative = "two.sided")
  

• Interpretation: If the p-value is below your significance threshold (usually 0.05), you can conclude there's a significant difference between the two lectures' scores.

2. Paired t-Test (Parametric, Conditional Use) #

You can use a paired t-test if you're willing to treat the 5-point Likert scale as continuous data (assigning 1 = Very Disagree, 5 = Very Agree) and your data meets these assumptions:

• The differences between paired scores are approximately normally distributed (check with a Q-Q plot or Shapiro-Wilk test).
• Sample size is large enough (the central limit theorem can help here if normality is a bit off).
• How it works: Tests whether the mean difference between the two scores is significantly different from 0.
• Example R code:

  t.test(scoreA, scoreB, paired = TRUE, alternative = "two.sided")
  

• Caveat: Many statisticians argue against treating Likert as continuous, so use this only if you're confident the scale behaves like an interval measure here.

3. Bowker's Test (For Testing Distributional Symmetry) #

If you want to look beyond just median/mean differences and test whether the entire distribution of responses differs between the two lectures (e.g., more students "Strongly Agreed" with Lecture A vs. B), Bowker's Test is the right choice. It's an extension of the McNemar Test for paired categorical data with more than two categories:

• What it tests: Whether the frequency of students rating Lecture A higher than B is symmetric to those rating B higher than A (across all response categories).
• Example R code (using the DescTools package):

  library(DescTools)
  # Create a contingency table of paired responses
  contingency_table <- table(scoreA, scoreB)
  BowkerTest(contingency_table)
  

• Interpretation: A significant p-value means the response distributions for the two lectures are not symmetric—i.e., there's a meaningful difference in how students rated them.

Quick Practical Steps Before Testing #

1. Descriptive Stats First: Calculate means, medians, and interquartile ranges for each lecture, then plot a paired boxplot or stacked bar chart to visualize the differences. This helps you understand the direction and magnitude of the difference before running significance tests.
2. Check Assumptions: For parametric tests (like the paired t-test), verify normality of the differences. For nonparametric tests, no assumptions about normality are needed.

内容的提问来源于stack exchange，提问作者Ong Zhi Qiang