基于多准则重塑数据：寻求高效替代循环的实现方案

针对大数据量场景，完全不需要用for循环，推荐用向量化分组操作实现，以下是两种高效方案（R语言）：

先构建原始数据 #

route_id <- c('FR00020604','FR00020604','FR00020604','FR00020604','FR00026117','FR00026117','FR00026117','FR00026117','FR00026117')
flow_type <- c('STD','STD','NSTD','NSTD','STD','STD','STD','STD','STD')
prod <- c('654495','654495','654495','654495','732919','732919','732921','732921','732921')
route_seg_num <- c('01','02','01','02','01','02','01','02','03')
ship_from <- c('S22491IE02','S22521DE09','S22491IE02','S22521DE09','S00745BR01','S01480PA01','S00745BR01','S01480PA01','S100142300')
ship_to <- c('S22521DE09','S85879GB03','S22521DE09','S85879GB03','S01480PA01','S100142300','S01480PA01','S100142300','S100153252')

table1 <- data.frame(route_id, flow_type, prod, route_seg_num, ship_from, ship_to, stringsAsFactors = FALSE)

方案1：用dplyr（易读性优先） #

适合需要清晰代码结构的场景，处理十万级数据无压力：

library(dplyr)

# 按route_id/flow_type/prod分组，聚合为完整路径+关键节点
table2 <- table1 %>%
  group_by(route_id, flow_type, prod) %>%
  arrange(route_seg_num, .by_group = TRUE) %>%
  summarise(
    start_point = first(ship_from),
    end_point = last(ship_to),
    total_segments = n(),
    full_route = paste(c(ship_from, last(ship_to)), collapse = " -> ")
  ) %>%
  ungroup()

# 如果table2需要把各路段拆分为单独列（长转宽）
table2_wide <- table1 %>%
  group_by(route_id, flow_type, prod) %>%
  arrange(route_seg_num, .by_group = TRUE) %>%
  mutate(seg_label = paste0("seg", route_seg_num)) %>%
  pivot_wider(
    names_from = seg_label,
    values_from = c(ship_from, ship_to),
    names_glue = "{seg_label}_{.value}"
  ) %>%
  ungroup()

方案2：用data.table（性能优先） #

专门针对超大数据量（百万级以上）优化，速度比dplyr更快：

library(data.table)

setDT(table1)

# 聚合为完整路径+关键节点
table2 <- table1[order(route_seg_num), 
                 .(
                   start_point = first(ship_from),
                   end_point = last(ship_to),
                   total_segments = .N,
                   full_route = paste(c(ship_from, last(ship_to)), collapse = " -> ")
                 ), 
                 by = .(route_id, flow_type, prod)]

# 长转宽拆分路段为单独列
table1[, seg_label := paste0("seg", route_seg_num)]
table2_wide <- dcast(
  table1,
  route_id + flow_type + prod ~ seg_label,
  value.var = c("ship_from", "ship_to"),
  sep = "_"
)

关键说明 #

两种方案都基于分组向量化操作，避免了循环的低效遍历，data.table在内存使用和处理速度上更适合极端大数据场景；dplyr代码更直观，便于维护。可根据table2的具体结构（聚合路径/拆分路段列）选择对应代码。

内容的提问来源于stack exchange，提问作者user1765608