NLP新手使用NLTK生成搭配遇ValueError报错求助

Hey there! Let's break down why you're hitting that too many values to unpack (expected 2) error and how to fix it.

What's Causing the Error? #

Looking at the traceback, the issue stems from a mismatch in your NLTK version. The collocations() method in nltk.Text expects collocation_list() to return simple word pairs (like (word1, word2)), but your current NLTK version returns longer tuples (likely including a score value, e.g., (score, word1, word2)). When the internal code tries to unpack these longer tuples into just two variables, it throws the ValueError.

Two Ways to Resolve It #

Option 1: Downgrade NLTK to a Compatible Version

Older NLTK versions (like 3.5) have collocation_list() returning only word pairs, which works seamlessly with the collocations() method. Run this command in your terminal to downgrade:

pip install nltk==3.5

After downgrading, your original code should run without errors.

Option 2: Use NLTK's Official Collocation Tools (Recommended)

The Text.collocations() method is a basic wrapper—NLTK's official docs now recommend using BigramCollocationFinder for more control and version stability. Here's a working example tailored to your use case:

from nltk.corpus import genesis
from nltk.collocations import BigramCollocationFinder
from nltk.metrics import BigramAssocMeasures

# Load the text tokens
tokens = genesis.words('english-kjv.txt')

# Initialize the bigram collocation finder
finder = BigramCollocationFinder.from_words(tokens)

# Filter out rare word pairs (optional but helpful—adjust the number to your needs)
finder.apply_freq_filter(5)

# Get the top 20 collocations using PMI (Pointwise Mutual Information) scoring
top_collocations = finder.nbest(BigramAssocMeasures.pmi, 20)

# Print results in a readable format
print("; ".join([" ".join(pair) for pair in top_collocations]))

This approach lets you customize filters (like frequency thresholds) and scoring methods (you can swap PMI for likelihood_ratio if preferred), and it won't break with future NLTK updates.

内容的提问来源于stack exchange，提问作者RevolverRakk