R语言模糊匹配相似公司名 为同主体分配相同group_id
企业分支机构group_id匹配问题
问题背景
最初基于公司名称精确匹配规则合并dt、dt2两个数据集,目的是为第二个数据集dt2分配group_id分组标识。当前需要为group_id为空的记录填充正确值:这类记录属于同一企业的不同分支机构,仅公司名称的文本表述存在差异。
初始精确匹配合并使用的代码如下:
dt2 <- merge(dt2,dt[,c("psn_name_PAT","group_id")],by = c("psn_name_PAT"),all.x = T,all.y = F)
样例数据
psn_name_PAT n name_std group_id 1 CHICHIBU FUJI COMPANY 2 CHICHIBU FUJI NA 2 FUJI CERAMICS 1 FUJI CERAMICS NA 3 FUJI CHEMICAL 1 FUJI 606 4 FUJI ELECTRIC COMPANY 439 FUJI 606 5 FUJI ELECTRIC CORPORATE RESEARCH & DEVELOPMENT 5 FUJI 606 6 FUJI ELECTRIC CORPORATE RESEARCH AND DEVELOPMENT 12 FUJI 606
期望输出结果
psn_name_PAT n name_std group_id 1 CHICHIBU FUJI COMPANY 2 CHICHIBU FUJI 606 2 FUJI CERAMICS 1 FUJI CERAMICS 606 3 FUJI CHEMICAL 1 FUJI 606 4 FUJI ELECTRIC COMPANY 439 FUJI 606 5 FUJI ELECTRIC CORPORATE RESEARCH & DEVELOPMENT 5 FUJI 606 6 FUJI ELECTRIC CORPORATE RESEARCH AND DEVELOPMENT 12 FUJI 606
之前尝试通过如下代码对dt2做非精确词匹配,但该方法会匹配到错误的group_id,无法关联到正确的所属公司(如FUJI主体):
dt2 <- dt2 %>% mutate(group_id=ifelse(grepl(paste0("\\<",fuz$name_std,"\\b", collapse = "|"), name_std),fuz$group_id,NA))
完整数据集结构
dt数据集
structure(list(psn_name_PAT = c("FUJI CHEMICAL", "FUJI ELECTRIC COMPANY", "FUJI ELECTRIC CORPORATE RESEARCH & DEVELOPMENT", "FUJI ELECTRIC CORPORATE RESEARCH AND DEVELOPMENT", "FUJI ELECTRIC CORPORATION RESEARCH AND DEVELOPMENT", "FUJI ELECTRIC SYSTEMS COMPANY", "FUJI ELECTRIC TECHNOLOGY COMPANY", "FUJI MACHINE MANUFACTURING COMPANY", "FUJITSU", "FUJITSU GENERAL", "FUJIMI", "FUJIFILM", "FUJIFILM ELECTRONIC MATERIALS U.S.A.", "FUJIFILM MANUFACTURING EUROPE", "FUJIKURA", "FUJI MACHINERY MFG. & ELECTRONICS COMPANY", "FUJI XEROX COMPANY", "FUJIKIN", "FUJIKOSHI MACHINERY CORPORATION", "HONGFUJIN PRECISION INDUSTRY (SHENZHEN) COMPANY", "TOSHIBA AMERICA RESEARCH", "TOSHIBA CORPORATION", "TOSHIBA MACHINE COMPANY", "TOSHIBA MATERIALS COMPANY", "TOSHIBA MEDICAL SYSTEMS CORPORATION", "TOSHIBA SOLUTIONS CORPORATION" ), name_std = c("FUJI", "FUJI", "FUJI", "FUJI", "FUJI", "FUJI", "FUJI", "FUJI", "FUJITSU", "FUJITSU", "FUJIMI", "FUJIFILM", "FUJIFILM", "FUJIFILM", "FUJIKURA", "FUJI MFG", "FUJI XEROX", "FUJIKIN", "FUJIKOSHI", "HONGFUJIN", "TOSHIBA", "TOSHIBA", "TOSHIBA", "TOSHIBA", "TOSHIBA", "TOSHIBA"), n = c(45L, 45L, 45L, 45L, 45L, 45L, 45L, 45L, 6L, 6L, 4L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), group_id = c(606L, 606L, 606L, 606L, 606L, 606L, 606L, 606L, 614L, 614L, 613L, 609L, 609L, 609L, 612L, 607L, 608L, 610L, 611L, 705L, 1631L, 1631L, 1631L, 1631L, 1631L, 1631L)), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, -26L))
dt2数据集
structure(list(psn_name_PAT = c("CHICHIBU FUJI COMPANY", "FUJI CERAMICS", "FUJI CHEMICAL", "FUJI ELECTRIC COMPANY", "FUJI ELECTRIC CORPORATE RESEARCH & DEVELOPMENT", "FUJI ELECTRIC CORPORATE RESEARCH AND DEVELOPMENT", "FUJI ELECTRIC CORPORATION RESEARCH AND DEVELOPMENT", "FUJI ELECTRIC DEVICE TECHNOLOGY COMPANY", "FUJI ELECTRIC SYSTEMS COMPANY", "FUJI ELECTRIC TECHNOLOGY COMPANY", "FUJI ELECTROCHEMICAL COMPANY", "FUJI FILM MICRODEVICES COMPANY", "FUJI HEAVY IND", "FUJI MACHINE MANUFACTURING COMPANY", "FUJI MACHINERY MFG. & ELECTRONICS COMPANY", "FUJI SEIKI MACHINE WORKS", "FUJI XEROX COMPANY", "FUJIFILM", "FUJIFILM DIAMATIX", "FUJIFILM DIMATIX", "FUJIFILM ELECTRONIC IMAGING", "FUJIFILM ELECTRONIC MATERIALS U.S.A.", "FUJIFILM MANUFACTURING EUROPE", "FUJIKIN", "FUJIKOSHI KIKAI KOGYO", "FUJIKOSHI MACHINERY CORPORATION", "FUJIKURA", "FUJIMA", "FUJIMI", "FUJINON CORPORATION", "FUJITSU", "FUJITSU AMD SEMICONDUCTOR", "FUJITSU AMD SEMICONDUCTOR LIMITED (FASL)", "FUJITSU AND SEMICONDUCTOR", "FUJITSU AUTOMATION", "FUJITSU COMPONENT", "FUJITSU DISPLAY TECHNOLOGIES CORPORATION", "FUJITSU FRONTECH", "FUJITSU GENERAL", "FUJITSU HITACHI PLASMA DISPLAY", "FUJITSU LIMITED KABUSHIKI KAISHA TOSHIBA", "FUJITSU MEDIA DEVICES", "FUJITSU MICROCOMPUTER SYSTEMS", "FUJITSU MICROELECTRONICS", "FUJITSU MICROELECTRONICS LMIITED", "FUJITSU MIYAGI ELECTRONICS", "FUJITSU QUANTUM DEVICES", "FUJITSU SEMICONDUCTOR", "FUJITSU TAKAMISAWA COMPONENT", "FUJITSU TEN", "FUJITSU VLSI", "FUJITSU YAMANASHI ELECTRONICS", "HONGFUJIN PRECISION INDUSTRY (SHENZHEN) COMPANY", "KYUSHU FUJITSU ELECTRONICS", "NEC TOSHIBA SPACE SYSTEMS", "SUZUKA FUJI XEROX COMPANY", "TOSHIBA AMERICA ELECTRONIC COMPONENTS", "TOSHIBA AMERICA RESEARCH", "TOSHIBA AUTOMATION COMPANY", "TOSHIBA AVE CORPORATION", "TOSHIBA CERAMICS COMPANY", "TOSHIBA COMPONENTS COMPANY", "TOSHIBA CORPORATION", "TOSHIBA KIKAI", "TOSHIBA LIGHTING & TECHNOLOGY CORPORATION", "TOSHIBA MACHINE COMPANY", "TOSHIBA MATERIALS COMPANY", "TOSHIBA MATSUSHITA DISPLAY TECHNOLOGY COMPANY", "TOSHIBA MEDICAL SYSTEMS CORPORATION", "TOSHIBA MICROELECTRONICS CORPORATION", "TOSHIBA MOBILE DISPLAY COMPANY", "TOSHIBA SHIBAURA DENKI", "TOSHIBA SILICONE COMPANY", "TOSHIBA SOLUTIONS CORPORATION", "TOSHIBA TEC CORPORATION", "TOSHIBA TECHNO CENTER" ), n = c(2L, 1L, 1L, 439L, 5L, 12L, 1L, 43L, 38L, 1L, 1L, 1L, 1L, 2L, 2L, 4L, 129L, 549L, 1L, 5L, 1L, 1L, 2L, 5L, 1L, 1L, 35L, 1L, 7L, 20L, 2430L, 7L, 1L, 2L, 1L, 2L, 4L, 2L, 1L, 1L, 1L, 36L, 1L, 126L, 1L, 1L, 56L, 312L, 1L, 3L, 25L, 1L, 6L, 13L, 1L, 1L, 14L, 1L, 1L, 1L, 19L, 1L, 5609L, 7L, 5L, 5L, 15L, 22L, 8L, 1L, 2L, 1L, 1L, 1L, 3L, 4L), name_std = c("CHICHIBU FUJI", "FUJI CERAMICS", "FUJI", "FUJI", "FUJI", "FUJI", "FUJI", "FUJI DEVICE", "FUJI", "FUJI", "FUJI ELECTROCHEMICAL", "FUJI MICRODEVICES", "FUJI HEAVY IND", "FUJI", "FUJI MFG", "FUJI SEIKI WORKS", "FUJI XEROX", "FUJIFILM", "FUJIFILM DIAMATIX", "FUJIFILM DIMATIX", "FUJIFILM IMAGING", "FUJIFILM", "FUJIFILM", "FUJIKIN", "FUJIKOSHI KIKAI", "FUJIKOSHI", "FUJIKURA", "FUJIMA", "FUJIMI", "FUJINON", "FUJITSU", "FUJITSU AMD SEMICONDUCTOR", "FUJITSU AMD SEMICONDUCTOR FASL", "FUJITSU SEMICONDUCTOR", "FUJITSU AUTOMATION", "FUJITSU COMPONENT", "FUJITSU DISPLAY", "FUJITSU FRONTECH", "FUJITSU", "FUJITSU HITACHI PLASMA DISPLAY", "FUJITSU KABUSHIKI KAISHA TOSHIBA", "FUJITSU DEVICES", "FUJITSU MICROCOMPUTER", "FUJITSU MICROELECTRONICS", "FUJITSU MICROELECTRONICS LMIITED", "FUJITSU MIYAGI", "FUJITSU QUANTUM DEVICES", "FUJITSU SEMICONDUCTOR", "FUJITSU TAKAMISAWA COMPONENT", "FUJITSU TEN", "FUJITSU VLSI", "FUJITSU YAMANASHI", "HONGFUJIN", "KYUSHU FUJITSU", "NEC TOSHIBA SPACE", "SUZUKA FUJI XEROX", "TOSHIBA COMPONENTS", "TOSHIBA", "TOSHIBA AUTOMATION", "TOSHIBA AVE", "TOSHIBA CERAMICS", "TOSHIBA COMPONENTS", "TOSHIBA", "TOSHIBA KIKAI", "TOSHIBA LIGHTING", "TOSHIBA", "TOSHIBA", "TOSHIBA MATSUSHITA DISPLAY", "TOSHIBA", "TOSHIBA MICROELECTRONICS", "TOSHIBA MOBILE DISPLAY", "TOSHIBA SHIBAURA DENKI", "TOSHIBA SILICONE", "TOSHIBA", "TOSHIBA TEC", "TOSHIBA TECHNO" ), group_id = c(NA, NA, 606L, 606L, 606L, 606L, 606L, NA, 606L, 606L, NA, NA, NA, 606L, 607L, NA, 608L, 609L, NA, NA, NA, 609L, 609L, 610L, NA, 611L, 612L, NA, 613L, NA, 614L, NA, NA, NA, NA, NA, NA, NA, 614L, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 705L, NA, NA, NA, NA, 1631L, NA, NA, NA, NA, 1631L, NA, NA, 1631L, 1631L, NA, 1631L, NA, NA, NA, NA, 1631L, NA, NA)), row.names = c(NA, -76L), class = "data.frame")
解决方案
之前的正则匹配出错核心原因是:短品牌名(比如FUJI)会优先命中包含它的长品牌名(比如FUJITSU/FUJIFILM/FUJIKURA)的记录,导致错配。
解决思路是长标准名优先匹配,避免短词抢占结果,同时加整词边界防止部分字符串误匹配,代码如下:
library(dplyr) # 提取唯一的标准名-分组ID映射,按名称长度降序排列,长名称优先匹配 std_map <- dt %>% distinct(name_std, group_id) %>% mutate(name_len = nchar(name_std)) %>% arrange(desc(name_len)) res_dt <- dt2 # 遍历映射表,仅填充group_id为空的记录 for(i in 1:nrow(std_map)){ current_std <- std_map$name_std[i] current_gid <- std_map$group_id[i] # 匹配规则:name_std以当前标准名开头,加整词边界避免跨词匹配 match_pattern <- paste0("^", current_std, "\\b") fill_idx <- which(is.na(res_dt$group_id) & grepl(match_pattern, res_dt$name_std)) res_dt$group_id[fill_idx] <- current_gid }
运行后可以验证:
- 样例中
CHICHIBU FUJI、FUJI CERAMICS会正确匹配到FUJI对应的606 FUJIFILM DIAMATIX会优先被更长的FUJIFILM匹配到609,不会被短词FUJI抢占KYUSHU FUJITSU ELECTRONICS会匹配到FUJITSU对应的614TOSHIBA AUTOMATION会匹配到TOSHIBA对应的1631
无错配问题。如果业务中存在特殊命名规则,可以调整匹配正则进一步适配。
内容的提问来源于stack exchange,提问作者Amleto
相关产品推荐
相关产品推荐

