如何从多文本列data.frame创建quanteda语料库？多列处理方案咨询

Great question! When working with a data frame that has multiple text columns, quanteda's text_field parameter does limit you to a single column for initial corpus creation—but there are smarter ways to handle this than just building two separate corpora. Let's walk through the best approaches:

1. Reshape Your Data to Long Format (Recommended) #

The cleanest method is to convert your wide-format data (one row per observation, multiple text columns) into long format (one row per text entry, with a column indicating which text field it comes from). This plays nicely with quanteda's workflow and keeps all your text data in a single corpus, making it easier to manage and analyze.

Step-by-Step Code Example: #

Suppose your data frame df looks like this:

First, use tidyr::pivot_longer to reshape it:

library(tidyr)
library(quanteda)

# Reshape to long format
df_long <- df %>%
  pivot_longer(
    cols = starts_with("text"),  # Target all text columns
    names_to = "text_type",      # Column to track which text field it is
    values_to = "text_content"   # Column holding the actual text
  )

# Now df_long looks like:
# | id | text_type | text_content          |
# |----|-----------|-----------------------|
# | 1  | text1     | "Hello world"         |
# | 1  | text2     | "Goodbye world"       |
# | 2  | text1     | "Quanteda is great"   |
# | 2  | text2     | "Text analysis rocks" |

Then create a single corpus with all your text, keeping track of the original id and text_type as document variables:

corp <- corpus(df_long, text_field = "text_content", docvars = df_long[c("id", "text_type")])

# You can now easily subset the corpus for each text type:
corp_text1 <- corpus_subset(corp, text_type == "text1")
corp_text2 <- corpus_subset(corp, text_type == "text2")

# Or create separate DFMs directly:
dfm_text1 <- dfm(corp_text1)
dfm_text2 <- dfm(corp_text2)

This approach is flexible—you can also analyze both text types together (e.g., compare word frequencies across text1 and text2) without juggling multiple corpora.

2. Merge Two Corpora/DFMs by ID (If You Prefer Separate Initial Corpora) #

If you do want to start with two separate corpora, you absolutely can merge them by id later. The key is to retain the id as a document variable in both corpora, then align and combine their DFMs.

Step-by-Step Code Example: #

# Create two corpora, each using a different text column, and keep the id
corp1 <- corpus(df, text_field = "text1", docvars = df["id"])
corp2 <- corpus(df, text_field = "text2", docvars = df["id"])

# Convert to DFMs
dfm1 <- dfm(corp1)
dfm2 <- dfm(corp2)

# Rename columns to avoid conflicts (since both DFMs might share vocabulary)
colnames(dfm1) <- paste0("text1_", colnames(dfm1))
colnames(dfm2) <- paste0("text2_", colnames(dfm2))

# Ensure both DFMs are sorted by id to match rows correctly
dfm1_sorted <- dfm1[order(docvars(dfm1, "id")), ]
dfm2_sorted <- dfm2[order(docvars(dfm2, "id")), ]

# Merge the DFMs into one
merged_dfm <- cbind(dfm1_sorted, dfm2_sorted)

# Verify the merged DFM has matching ids
docvars(merged_dfm, "id")

Important Notes: #

• Always sort by id before merging to avoid misalignment (especially if your original data wasn't ordered consistently).
• Prefixing column names prevents duplicate vocabulary labels from overlapping in the merged DFM.

Which Approach Should You Choose? #

• Long format is better for most cases: it keeps your data organized, simplifies subsetting, and supports more complex analyses (like cross-text-type comparisons).
• Merging separate DFMs works if you only need to combine the final matrix outputs, but it’s less flexible for intermediate steps (like cleaning or preprocessing text separately).

内容的提问来源于stack exchange，提问作者Ted Mosby