为数据框添加重复观测行计数并提取全部重复行

当然可以提取完整的重复观测行，以下结合你提供的数据集，分两种常见场景给出解决方案：

首先把你的数据集加载到R环境中：

df <- structure(list(CrimeId = c(160903280L, 160912272L, 160912590L, 160912801L, 160912811L, 160913003L), OriginalCrimeTypeName = c("Assault / Battery", "Homeless Complaint", "Susp Info", "Report", "594", "Ref'd"), OffenseDate = c("2016-03-30T00:00:00", "2016-03-31T00:00:00", "2016-03-31T00:00:00", "2016-03-31T00:00:00", "2016-03-31T00:00:00", "2016-03-31T00:00:00"), CallTime = c("18:42", "15:31", "16:49", "17:38", "17:42", "18:29"), CallDateTime = c("2016-03-30T18:42:00", "2016-03-31T15:31:00", "2016-03-31T16:49:00", "2016-03-31T17:38:00", "2016-03-31T17:42:00", "2016-03-31T18:29:00"), Disposition = c("REP", "GOA", "GOA", "GOA", "REP", "GOA"), Address = c("100 Block Of Chilton Av", "2300 Block Of Market St", "2300 Block Of Market St", "500 Block Of 7th St", "Beale St/bryant St", "16th St/pond St"), City = c("San Francisco", "San Francisco", "San Francisco", "San Francisco", "San Francisco", "San Francisco"), State = c("CA", "CA", "CA", "CA", "CA", "CA"), AgencyId = c("1", "1", "1", "1", "1", "1"), Range = c(NA, NA, NA, NA, NA, NA), AddressType = c("Premise Address", "Premise Address", "Premise Address", "Premise Address", "Intersection", "Intersection")), row.names = c(NA, 6L), class = "data.frame")

1. 提取整行完全重复的观测行 #

整行完全重复指所有列的值完全一致的行。用duplicated()函数结合反向判断，就能保留所有重复行（包括第一次出现的重复项）：

# 提取所有整行重复的行
full_duplicates <- df[duplicated(df) | duplicated(df, fromLast = TRUE), ]

注：你提供的数据集中CrimeId是唯一标识，所以没有整行完全重复的行，运行后会返回空数据框。

2. 提取基于指定列重复的观测行 #

多数场景下，你可能更关注某些关键列重复的行（比如数据中Address重复的情况），只需指定判断重复的列即可：

单列重复示例（以Address为例） #

# 提取Address列重复的所有行
address_duplicates <- df[duplicated(df$Address) | duplicated(df$Address, fromLast = TRUE), ]

运行后会返回数据集中第2、3行，这两行的Address均为2300 Block Of Market St。

多列重复示例（比如Address+City+State） #

如果需要基于多列判断重复，按以下方式指定列：

# 提取多列重复的所有行
multi_col_duplicates <- df[duplicated(df[, c("Address", "City", "State")]) | duplicated(df[, c("Address", "City", "State")], fromLast = TRUE), ]

用dplyr包的简洁写法 #

如果你习惯使用tidyverse工具链，用dplyr的filter()函数会更直观：

library(dplyr)

# 整行重复
full_duplicates_dplyr <- df %>% filter(duplicated(.) | duplicated(., fromLast = TRUE))

# 指定列重复（以Address为例）
address_duplicates_dplyr <- df %>% filter(duplicated(Address) | duplicated(Address, fromLast = TRUE))

内容的提问来源于stack exchange，提问作者Kevin Santos