You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

替代For Loop:百万级数据动态过滤与汇总的高效方案

处理500万条数据的空间邻域高效汇总方案

我有一个包含500万条记录的数据集,结构如下:

library(dplyr)

Data <- tibble(
  X = runif(5000000, min=200000, max=400000),
  Y = runif(5000000, min=400000, max=500000),
  UID = 1:5000000,
  Count = runif(5000000, min=1, max=2000)
)

需要为每个UID计算四个空间范围(200m、400m、800m、1600m)内的以下指标:

  • 范围内其他点的数量
  • 这些点的Count字段总和
  • 密度值,计算公式:Count总和 / (点数量 * 0.000625)

最终输出要包含每个UID及其对应四个范围的所有汇总结果。

我用for循环实现了需求,但运行耗时极长,代码如下:

Summary <- NULL

for (n in 1:nrow(Data)){
  Data_Row <- Data[n,]

  Row_200 <- Data %>% 
    filter(between(X, Data_Row$X-200, Data_Row$X+200) & between(Y, Data_Row$Y-200, Data_Row$Y+200)) %>%
    summarise(
      Instances_200 = n(), 
      Count_200 = sum(Count), 
      Calculation_200 = Count_200/(Instances_200*0.000625)
    )

  Row_400 <- Data %>% 
    filter(between(X, Data_Row$X-400, Data_Row$X+400) & between(Y, Data_Row$Y-400, Data_Row$Y+400)) %>%
    summarise(
      Instances_400 = n(), 
      Count_400 = sum(Count), 
      Calculation_400 = Count_400/(Instances_400*0.000625)
    )

  Row_800 <- Data %>% 
    filter(between(X, Data_Row$X-800, Data_Row$X+800) & between(Y, Data_Row$Y-800, Data_Row$Y+800)) %>%
    summarise(
      Instances_800 = n(), 
      Count_800 = sum(Count), 
      Calculation_800 = Count_800/(Instances_800*0.000625)
    )

  Row_1600 <- Data %>% 
    filter(between(X, Data_Row$X-1600, Data_Row$X+1600) & between(Y, Data_Row$Y-1600, Data_Row$Y+1600)) %>%
    summarise(
      Instances_1600 = n(), 
      Count_1600 = sum(Count), 
      Calculation_1600 = Count_1600/(Instances_1600*0.000625)
    )

  Summary_Row <- bind_cols(Data[n,"UID"], Row_200, Row_400, Row_800, Row_1600)

  Summary <- bind_rows(Summary, Summary_Row)
}

我了解lapply、map这类函数,但不知道如何用它们高效完成这种动态过滤与汇总操作,求可行的解决方案。


内容的提问来源于stack exchange,提问作者Chris

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.16 01:55:38