R语言：筛选已分词数据框中参议员发言行遇空结果求助

问题背景 #

需要从已分词的国会记录数据框中，筛选出以参议员名称开头的发言行，排除提案、标题类内容，但使用subset()和filter()函数时始终得到空结果。

原代码 #

library(tidytext)
library(dplyr)
library(stringr)

#this is the list of senator names with regex
## "^" to indicate that the names are at the begginning of the string
## "\\s" to indicate that there should be a white space after the ":"
## "(.*) to indicate that it should include all following content
sen_names <- c("^Mr. McCONNELL (R; Kentucky):\\s(.*)","^Mr. JOHNSON (D; South Dakota):\\s(.*)","^Mr. NICKLES (R; Oklahoma):\\s(.*)","^Mr. LEAHY (D; Vermont):\\s(.*)","^Mr. CONRAD (D; North Dakota):\\s(.*)","^Mrs. BOXER (D; California):\\s(.*)","^Mrs. BOXER (D; California):\\s(.*)","^Mr. SHELBY (R:\\s(.*) Alabama):\\s(.*)","^Ms. MURKOWSKI (R; Alaska):\\s(.*)","^Mr. STEVENS (R; Alaska):\\s(.*)","^Mr. KYL (R; Arizona):\\s(.*)","^Mr. McCAIN (R; Arizona):\\s(.*)","^Mr. PRYOR (D; Arkansas):\\s(.*)","^Mrs. LINCOLN (D; Arkansas):\\s(.*)","^Mrs. FEINSTEIN (D; California):\\s(.*)","^Mr. ALLARD (R; Colorado):\\s(.*)","^Mr. CAMPBELL (R; Colorado):\\s(.*)","^Mr. DODD (D; Connecticut):\\s(.*)","^Mr. LIEBERMAN (D; Connecticut):\\s(.*)","^Mr. BIDEN (D; Delaware):\\s(.*)","^Mr. CARPER (D; Delaware):\\s(.*)","^Mr. L GRAHAM (R; South Carolina):\\s(.*)","^Mr. B GRAHAM (D; Florida):\\s(.*)","^Mr. C NELSON (D; Florida):\\s(.*)","^Mr. B NELSON (D; Nebraska):\\s(.*)","^Mr. CHAMBLISS (R; Georgia):\\s(.*)","^Mr. MILLER (D; Georgia):\\s(.*)","^Mr. AKAKA (D; Hawaii):\\s(.*)","^Mr. INOUYE (D; Hawaii):\\s(.*)","^Mr. CRAIG (R; Idaho):\\s(.*)","^Mr. CRAPO (R; Idaho):\\s(.*)","^Mr. DURBIN (D; Illinois):\\s(.*)","^Mr. FITZGERALD (R; Illinois):\\s(.*)","^Mr. BAYH (D; Indiana):\\s(.*)","^Mr. LUGAR (R; Indiana):\\s(.*)","^Mr. GRASSLEY (R; Iowa):\\s(.*)","^Mr. HARKIN (D; Iowa):\\s(.*)","^Mr. BROWNBACK (R; Kansas):\\s(.*)","^Mr. ROBERTS (R; Kansas):\\s(.*)","^Mr. BUNNING (R; Kentucky):\\s(.*)","^Mr. BREAUX (D; Louisiana):\\s(.*)","^Ms. LANDRIEU (D; Louisiana):\\s(.*)","^Ms. COLLINS (R; Maine):\\s(.*)","^Ms. SNOWE (R; Maine):\\s(.*)","^Ms. MIKULSKI (D; Maryland):\\s(.*)","^Mr. SARBANES (D; Maryland):\\s(.*)","^Mr. KENNEDY (D; Massachusetts):\\s(.*)","^Mr. KERRY (D; Massachusetts):\\s(.*)","^Ms. STABENOW (D; Michigan):\\s(.*)","^Mr. LEVIN (D; Michigan):\\s(.*)","^Mr. DAYTON (D; Minnesota):\\s(.*)","^Mr. COLEMAN (R; Minnesota):\\s(.*)","^Mr. COCHRAN (R; Mississippi):\\s(.*)","^Mr. COCHRAN (R; Mississippi):\\s(.*)","^Mr. TALENT (R; Missouri):\\s(.*)","^Mr. BOND (R; Missouri):\\s(.*)","^Mr. BAUCUS (D; Montana):\\s(.*)","^Mr. BURNS (R; Montana):\\s(.*)","^Mr. HAGEL (R; Nebraska):\\s(.*)","^Mr. ENSIGN (R; Nevada):\\s(.*)","^Mr. REID (D; Nevada):\\s(.*)","^Mr. GREGG (R; New Hampshire):\\s(.*)","^Mr. SUNUNU (R; New Hampshire):\\s(.*)","^Mr. CORZINE (D; New Jersey):\\s(.*)","^Mr. LAUTENBERG (D; New Jersey):\\s(.*)","^Mr. BINGAMAN (D; New Mexico):\\s(.*)","^Mr. DOMENICI (R; New Mexico):\\s(.*)","^Mrs. CLINTON (D; New York):\\s(.*)","^Mr. SCHUMER (D; New York):\\s(.*)","^Mr. SCHUMER (D; New York):\\s(.*)","^Mr. SCHUMER (D; New York):\\s(.*)","^Mr. DORGAN (D; North Dakota):\\s(.*)","^Mr. DEWINE (R; Ohio):\\s(.*)","^Mr. VOINOVICH (R; Ohio):\\s(.*)","^Mr. INHOFE (R; Oklahoma):\\s(.*)","^Mr. SMITH (R; Oregon):\\s(.*)","^Mr. WYDEN (D; Oregon):\\s(.*)","^Mr. SANTORUM (R; Pennsylvania):\\s(.*)","^Mr. SPECTER (R; Pennsylvania):\\s(.*)","^Mr. CHAFEE (R; Rhode Island):\\s(.*)","^Mr. REED (D; Rhode Island):\\s(.*)","^Mr. HOLLINGS (D; South Carolina):\\s(.*)","^Mr. DASCHLE (D; South Dakota):\\s(.*)","^Mr. FRIST (R; Tennessee):\\s(.*)","^Mr. ALEXANDER (R; Tennessee):\\s(.*)","^Mr. CORNYN (R; Texas):\\s(.*)","^Mrs. HUTCHISON (R; Texas):\\s(.*)","^Mr. BENNETT (R; Utah):\\s(.*)","^Mr. HATCH (R; Utah):\\s(.*)","^Mr. JEFFORDS (I; Vermont):\\s(.*)","^Mr. ALLEN (R; Virginia):\\s(.*)","^Mr. WARNER (R; Virginia):\\s(.*)","^Ms. CANTWELL (D; Washington):\\s(.*)","^Ms. CANTWELL (D; Washington):\\s(.*)","^Mr. BYRD (D; West Virginia):\\s(.*)","^Mr. ROCKEFELLER (D; West Virginia):\\s(.*)","^Mr. FEINGOLD (D; Wisconsin):\\s(.*)","^Mr. KOHL (D; Wisconsin):\\s(.*)","^Mr. ENZI (R; Wyoming):\\s(.*)","^Mr. THOMAS (R; Wyoming):\\s(.*)")

#this imports the column from a tokenized dataframe
##the tokenized df is a single congressional record
##and it has already been split, I just need the ones that...
#... start with the names and are speeches rather than descriptions...
#... of legislation
aug_01_2003_speeches <- data.frame(aug_01_2003_tkns$cleaned)
colnames(aug_01_2003_speeches) <- c("speeches")
View(aug_01_2003_speeches)


#tried running through subset()
##OUTPUT -> "<0 rows> (or 0-length row.names)"
#NOTE: original object names were aug_o1_speeches
#... changed here for clarity when calling the object
aug_01_2003_x <- subset(aug_01_2003_speeches, sen_names, select = speeches)
aug_01_2003_x <- subset(aug_01_2003_speeches, subset = sen_names, select = speeches)
aug_01_2003_x <- subset(aug_01_2003_speeches, speeches %in% sen_names)
aug_01_2003_x

#tried running through filter()
##OUTPUT -> "<0 rows> (or 0-length row.names)"
aug_01_2003_x <- filter(aug_01_2003_speeches, speeches %in% sen_names, .preserve = T)
aug_01_2003_x

#there are 100 names here, so running each one individually...
#... wouldn't be the best solution in the world

问题原因 #

1. 匹配逻辑错误：%in%是精确字符串匹配，而sen_names是正则表达式模式，speeches列是完整发言内容，两者不可能完全一致，导致无匹配结果。
2. 正则表达式错误：原正则中的括号()未转义，被视为分组语法而非匹配实际括号；同时Mr. SHELBY的正则格式有误，不符合数据实际结构。
3. subset用法错误：直接将sen_names作为subset的条件，而subset需要逻辑向量，语法逻辑不成立。

修正方案 #

步骤1：修复正则表达式 #

转义所有括号，修正格式错误，再将所有正则模式合并为一个，提升匹配效率。

步骤2：用str_detect()匹配正则 #

使用str_detect()检查speeches列是否匹配参议员名称开头的正则模式，替代错误的%in%。

修正后代码 #

library(tidytext)
library(dplyr)
library(stringr)

# 修复正则：转义括号，修正Mr. SHELBY的格式错误
sen_names_fixed <- sen_names %>%
  str_replace_all("\\(", "\\\\(") %>%  # 转义左括号
  str_replace_all("\\)", "\\\\)") %>%  # 转义右括号
  str_replace("^Mr. SHELBY \\(R:\\\\s(.*) Alabama\\):\\\\s(.*)", 
              "^Mr. SHELBY \\(R; Alabama\\):\\\\s.*")  # 修正格式

# 合并所有正则模式为一个，用|分隔
sen_pattern <- str_c(sen_names_fixed, collapse = "|")

# 构建数据框（假设aug_01_2003_tkns已加载）
aug_01_2003_speeches <- data.frame(speeches = aug_01_2003_tkns$cleaned)

# 筛选匹配的行
aug_01_2003_x <- aug_01_2003_speeches %>%
  filter(str_detect(speeches, sen_pattern))

# 查看结果
aug_01_2003_x

额外扩展 #

如果需要提取参议员姓名、党派、州和发言内容，可在筛选后用str_match()捕获：

aug_01_2003_speeches %>%
  filter(str_detect(speeches, sen_pattern)) %>%
  mutate(
    match = str_match(speeches, "^(Mr\\.|Mrs\\.|Ms\\.) ([A-Z\\s]+) \\(([A-Z]); ([A-Za-z\\s]+)\\):\\s(.*)"),
    title = match[,2],
    name = match[,3],
    party = match[,4],
    state = match[,5],
    content = match[,6]
  ) %>%
  select(-match)

内容的提问来源于stack exchange，提问作者Rochelle