构建dfm前处理搭配与复合词：quanteda报错问题咨询

我明白你遇到的问题了——tokens_compound()确实只接受tokens对象，而你直接传入了textstat_collocations()返回的搭配数据框，所以才会触发报错。下面是完整的正确流程，帮你把提取的搭配顺利整合到DFM中：

步骤1：提取搭配（调整阈值适配示例数据） #

首先还是用textstat_collocations()提取2-3词搭配，注意你的示例数据量小，min_count=30会提取不到任何结果，我先把阈值调低到2，你可以根据真实数据再调整：

library(quanteda)
dataset1 <- data.frame( 
  anumber = c(1,2,3), 
  text = c("Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.","It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum", "Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source.")
)

# 提取2-3词搭配，调整min_count适配示例数据
cols <- textstat_collocations(dataset1$text, size = 2:3, min_count = 2)

步骤2：创建tokens对象并合并搭配 #

先把文本转换成tokens对象，再用tokens_compound()将识别到的搭配合并成单个"复合词"：

# 先生成预处理后的tokens对象（和你原来的逻辑一致）
toks <- dataset1 %>% 
  corpus() %>% 
  tokens(remove_punct = TRUE, remove_numbers = TRUE, remove_symbols = TRUE)

# 把搭配合并到tokens中，cols会自动匹配并替换成下划线连接的复合词
toks_compounded <- tokens_compound(toks, pattern = cols)

步骤3：基于合并后的tokens构建DFM #

最后用处理好的tokens生成DFM，这样搭配就会作为单个特征出现在DFM里了：

myDfm <- dfm(toks_compounded)

额外优化建议 #

• 如果想筛选更显著的搭配，可以基于cols里的z值或p_value过滤，比如只保留z值大于3的搭配：

cols_filtered <- cols[cols$z > 3, ]
toks_compounded <- tokens_compound(toks, pattern = cols_filtered)

• 可以用head(toks_compounded)查看效果，能看到像lorem_ipsum这样的合并后的复合词。

内容的提问来源于stack exchange，提问作者Vl_dem