复杂数据集评分者间分析：Fleiss' Kappa结果计数异常求助

我的数据集为行为谱（ethogram）响应数据，由5名评分者分别对53个视频（涉及28只不同犬只）进行评分。最终数据集包含55列、265行，前3列用于标识评分者及对应评分的视频。我已将数据拆分为**有序（ordinal）和分类（categorical）**子集。

数据融化代码 #

# Melt the data----                              
# Convert all non-identifier columns to character type

ordinal <- ordinal %>%
  mutate(across(-c(Code, Data.Collection.Number, Assessor), as.character))

ordinal_melted_data <- ordinal %>%
  pivot_longer(cols = -c(Code, Data.Collection.Number, Assessor),
               names_to = "variable",
               values_to = "value") 

##Categorical 
# Convert all non-identifier columns to character type

categorical <- categorical %>%
  mutate(across(-c(Code, Data.Collection.Number, Assessor), as.character))

# Melt the data
categorical_melted_data <- categorical %>%
  pivot_longer(cols = -c(Code, Data.Collection.Number, Assessor),
               names_to = "variable",
               values_to = "value")

Fleiss' Kappa分析异常问题 #

我使用融化后的数据尝试进行Fleiss' Kappa和ICC分析，但分类数据的Fleiss' Kappa得出的评分者（3名）和受试者（28个）数量与实际不符，实际应为5名评分者、53个视频。

分析代码 #

# Get unique variable names from categorical_melted_data
variable_names <- unique(categorical_melted_data$variable)

# Initialize a list to store results
results <- list()

# Iterate over each variable
for (variable in variable_names) {
  # Subset data for the current variable
  variable_data <- categorical_melted_data[categorical_melted_data$variable == variable, ]
  
  # Print unique raters and subjects
  cat("Variable:", variable, "\n")
  cat("Unique raters:", unique(variable_data$Assessor), "\n")
  cat("Unique subjects:", unique(variable_data$Code), "\n")
  
  # Create the contingency table
  contingency_table <- table(variable_data$Code, variable_data$value)
  
  # Calculate Fleiss' Kappa
  fleisskappa_result <- kappam.fleiss(contingency_table)
  
  # Store the results
  results[[variable]] <- fleisskappa_result$value
}

# Print results along with raters and subjects details
for (variable in variable_names) {
  cat("Variable:", variable, "\n")
  
  # Display details on raters and subjects
  cat("Number of raters:", ncol(contingency_table), "\n")
  cat("Number of subjects:", nrow(contingency_table), "\n")
  
  # Display Fleiss' Kappa value for the variable
  cat("Fleiss' Kappa:", results[[variable]], "\n\n")
}

补充说明 #

融化后的数据格式为5列：Video Code、Assessor、Data Collection Number、variable、value

内容的提问来源于stack exchange，提问作者Jenny