使用dtplyr时在函数中引用新生成列的报错问题排查

问题场景 #

我有lazy_tbl格式的数据，使用dtplyr::lazy_dt()转换后，想在同一个mutate操作中先创建avgH、avgD、avgA三列，接着引用这三个新列生成prob1列，但执行代码时触发以下报错：

Error in `[.data.table`(copy(`_DT20`), , `:=`(c("avgH", "avgD", "avgA",  : 
  Variable '1' is not found in calling scope. Looking in calling scope because this symbol was prefixed with .. in the j= parameter.

但如果先调用as_tibble()将lazy_dt转换为普通tibble，再计算prob1列，就能得到预期结果。

报错原因 #

1. 语法符号冲突：pmap_dbl中用到的..1/..2/..3是tidyverse体系里用于引用匿名函数参数的语法，但在data.table的作用域规则中，..前缀是用来引用上层环境变量的标记。当dtplyr把dplyr代码转换为data.table语法时，会错误地将..1识别为要调用上层环境中名为1的变量，导致“找不到变量”的报错。
2. 延迟计算的作用域限制：lazy_dt基于data.table的延迟执行机制，同一mutate块内的新列虽然可以被后续表达式引用，但tidyverse的行级迭代函数（如pmap系列）在data.table的j表达式中运行时，会触发作用域解析冲突，无法正确识别刚生成的列。

解决方法 #

方法1：拆分mutate步骤 #

把prob1的计算放到单独的mutate调用中，让avgH/avgD/avgA列先被正确创建并纳入lazy_dt的上下文：

library(dtplyr)
library(dplyr)
library(implied)

possibly_mean <- possibly(mean, otherwise = NA)

ready_to_calculate %>% 
  lazy_dt() %>% 
  mutate(avgH = map_dbl(odds_1x2, ~possibly_mean(.x$home_odds, na.rm = TRUE)),
         avgD = map_dbl(odds_1x2, ~possibly_mean(.x$draw_odds, na.rm = TRUE)),
         avgA = map_dbl(odds_1x2, ~possibly_mean(.x$away_odds, na.rm = TRUE))) %>%
  mutate(prob1 = pmap_dbl(list(avgH, avgD, avgA), 
                          ~implied::implied_probabilities(c(..1, ..2, ..3), method = "wpo")$probabilities[1,1])) %>%
  as_tibble() # 最后按需转换为tibble格式

方法2：避免使用..参数语法 #

改用显式的匿名函数参数命名，替换..1/..2/..3，彻底避免与data.table的作用域标记冲突：

ready_to_calculate %>% 
  lazy_dt() %>% 
  mutate(avgH = map_dbl(odds_1x2, ~possibly_mean(.x$home_odds, na.rm = TRUE)),
         avgD = map_dbl(odds_1x2, ~possibly_mean(.x$draw_odds, na.rm = TRUE)),
         avgA = map_dbl(odds_1x2, ~possibly_mean(.x$away_odds, na.rm = TRUE)),
         prob1 = pmap_dbl(list(avgH, avgD, avgA), 
                          function(h, d, a) {
                            implied::implied_probabilities(c(h, d, a), method = "wpo")$probabilities[1,1]
                          })) %>%
  as_tibble()

方法3：使用向量化函数替代行级迭代 #

用Vectorize把implied_probabilities包装为向量化函数，直接对整列进行操作，跳过行级迭代的作用域问题：

vec_implied <- Vectorize(function(h, d, a) {
  if (any(is.na(c(h, d, a)))) return(NA)
  implied::implied_probabilities(c(h, d, a), method = "wpo")$probabilities[1,1]
})

ready_to_calculate %>% 
  lazy_dt() %>% 
  mutate(avgH = map_dbl(odds_1x2, ~possibly_mean(.x$home_odds, na.rm = TRUE)),
         avgD = map_dbl(odds_1x2, ~possibly_mean(.x$draw_odds, na.rm = TRUE)),
         avgA = map_dbl(odds_1x2, ~possibly_mean(.x$away_odds, na.rm = TRUE)),
         prob1 = vec_implied(avgH, avgD, avgA)) %>%
  as_tibble()

测试数据 #

ready_to_calculate <- structure(list(home_team = c("Arsenal", "Leeds"), away_team = c("Aston Villa", 
"Wolves"), odds_1x2 = list(structure(list(bookmaker = c("10Bet", 
"188BET", "1xBet", "888sport", "bet-at-home", "bet365", "Betfair", 
"Betsafe", "Betsson", "BetVictor", "Betway", "bwin", "ComeOn", 
"Interwetten", "Pinnacle", "Unibet", "William Hill", "Betfair Exchange"
), home_odds = c(2.05, 2.02, 1.84, 1.93, 2.02, 1.85, 1.91, 2.05, 
2.05, 1.95, 1.91, 1.9, 2.05, 1.95, 2.04, 1.96, 1.91, 1.97), draw_odds = c(3.6, 
3.6, 3.85, 3.6, 3.6, 3.75, 3.7, 3.65, 3.65, 3.6, 3.6, 3.5, 3.65, 
3.6, 3.65, 3.7, 3.6, 3.61), away_odds = c(3.65, 3.8, 4.41, 4.1, 
3.65, 4, 4.2, 3.7, 3.7, 4, 4, 4, 3.75, 3.85, 3.87, 4.1, 3.9, 
3.95)), class = "data.frame", row.names = c(NA, 18L)), NA)), class = "data.frame", row.names = c(NA, 
-2L))

内容的提问来源于stack exchange，提问作者Hakki