基于个体注视数据计算三方对话中的互视阶段

问题背景 #

处理三方对话场景下的注视数据，需要计算互视阶段：即说话者注视某听者，同时该听者回视说话者的重叠时间段。已知数据包含每个标注为A、B、C的说话者在任意时刻的注视对象及起止时间，数据结构如下：

原始数据示例 #

# A tibble: 10 × 5
   Utterance                  Role      Gaze_pair start   end
   <chr>                      <chr>     <chr>     <int> <int>
 1 what's steam punk exactly? Speaker   AC          299   700
 2 what's steam punk exactly? Recipient AC            0   601
 3 what's steam punk exactly? Recipient BC            0   355
 4 what's steam punk exactly? Recipient BC          411   700
 5 where are you guys from?   Speaker   AC           10   109
 6 where are you guys from?   Speaker   AC          678   750
 7 where are you guys from?   Recipient AC           50   900
 8 where are you guys from?   Speaker   BC          509   568
 9 where are you guys from?   Speaker   BC          800   900
10 where are you guys from?   Recipient BC            0   900

互视计算示例 #

以第一个对话what's steam punk exactly?为例：

• Speaker A在299-700注视Recipient C
• Recipient C在0-601回视Speaker A
• 两者的重叠时间段299-601即为互视阶段

期望输出 #

在原始数据基础上新增MutG_start和MutG_end列，标记对应行的互视时间段；无互视时填充0-0：

# A tibble: 10 × 7
   Utterance                  Role      Gaze_pair start   end MutG_start MutG_end
   <chr>                      <chr>     <chr>     <int> <int>      <int>    <int>
 1 what's steam punk exactly? Speaker   AC          299   700        299      601
 2 what's steam punk exactly? Recipient AC            0   601        299      601
 3 what's steam punk exactly? Recipient BC            0   355          0        0
 4 what's steam punk exactly? Recipient BC          411   700          0        0
 5 where are you guys from?   Speaker   AC           10   109         50      109
 6 where are you guys from?   Speaker   AC          678   750        678      750
 7 where are you guys from?   Recipient AC           50   900         50      109
 8 where are you guys from?   Speaker   BC          509   568        509      568
 9 where are you guys from?   Speaker   BC          800   900        800      900
10 where are you guys from?   Recipient BC            0   900        509      568

实现方案（R语言） #

使用dplyr、purrr和intervals包处理区间交集，支持同一对话中多个互视阶段的计算：

步骤1：加载数据和依赖包 #

library(dplyr)
library(purrr)
library(intervals)

# 加载可复现数据
gaze_data <- structure(list(Utterance = c("what's steam punk exactly?", "what's steam punk exactly?", 
"what's steam punk exactly?", "what's steam punk exactly?", "where are you guys from?", 
"where are you guys from?", "where are you guys from?", "where are you guys from?", 
"where are you guys from?", "where are you guys from?"), Role = c("Speaker", 
"Recipient", "Recipient", "Recipient", "Speaker", "Speaker", 
"Recipient", "Speaker", "Speaker", "Recipient"), Gaze_pair = c("AC", 
"AC", "BC", "BC", "AC", "AC", "AC", "BC", "BC", "BC"), start = c(299L, 
0L, 0L, 411L, 10L, 678L, 50L, 509L, 800L, 0L), end = c(700L, 
601L, 355L, 700L, 109L, 750L, 900L, 568L, 900L, 900L)), row.names = c(NA, 
-10L), class = c("tbl_df", "tbl", "data.frame"))

步骤2：预处理数据 #

生成配对key统一双向注视对，拆分注视者与被注视者：

gaze_data <- gaze_data %>%
  mutate(
    # 生成无序配对key，确保AC/CA对应同一组
    pair_key = map_chr(Gaze_pair, ~paste(sort(strsplit(.x, "")[[1]]), collapse = "")),
    # 拆分注视者和被注视者
    gazer = substr(Gaze_pair, 1, 1),
    gazed = substr(Gaze_pair, 2, 2)
  )

步骤3：定义区间交集计算函数 #

计算两组时间区间的所有重叠部分：

calculate_mutual_gaze <- function(speaker_intervals, recipient_intervals) {
  if (length(speaker_intervals) == 0 || length(recipient_intervals) == 0) {
    return(tibble(MutG_start = integer(), MutG_end = integer()))
  }
  
  # 转换为区间对象
  speaker_int <- Intervals(speaker_intervals)
  recipient_int <- Intervals(recipient_intervals)
  
  # 计算交集
  intersections <- interval_intersection(speaker_int, recipient_int)
  
  if (nrow(intersections) == 0) {
    return(tibble(MutG_start = 0L, MutG_end = 0L))
  } else {
    return(tibble(
      MutG_start = intersections[,1],
      MutG_end = intersections[,2]
    ))
  }
}

步骤4：分组计算互视区间并合并结果 #

# 按对话和配对key分组计算
mutual_gaze_results <- gaze_data %>%
  group_by(Utterance, pair_key) %>%
  summarise(
    # 提取Speaker的注视区间
    speaker_intervals = list(
      map2(start[Role == "Speaker"], end[Role == "Speaker"], c)
    ),
    # 提取Recipient的回视区间（仅保留对Speaker的回视）
    recipient_intervals = list(
      map2(start[Role == "Recipient" & gazer == setdiff(strsplit(pair_key, "")[[1]], gazer[Role == "Speaker"])], 
           end[Role == "Recipient" & gazer == setdiff(strsplit(pair_key, "")[[1]], gazer[Role == "Speaker"])], 
           c)
    ),
    # 计算互视区间
    mutual_intervals = list(calculate_mutual_gaze(unlist(speaker_intervals, recursive = FALSE), 
                                                  unlist(recipient_intervals, recursive = FALSE))),
    .groups = "drop"
  )

# 合并回原数据并处理空值
final_data <- gaze_data %>%
  left_join(mutual_gaze_results, by = c("Utterance", "pair_key")) %>%
  unnest(mutual_intervals, keep_empty = TRUE) %>%
  mutate(
    MutG_start = replace_na(MutG_start, 0L),
    MutG_end = replace_na(MutG_end, 0L)
  ) %>%
  select(Utterance, Role, Gaze_pair, start, end, MutG_start, MutG_end)

# 查看最终结果
print(final_data)

关键逻辑说明 #

1. 配对统一：通过pair_key将双向注视（如A→C和C→A）归为同一组，确保能匹配双方的注视数据。
2. 区间交集：使用intervals包处理多区间的重叠计算，支持同一对话中存在多个互视阶段的场景。
3. 空值处理：无互视区间时自动填充0-0，符合期望输出格式。

内容的提问来源于stack exchange，提问作者Chris Ruehlemann