GloVe词嵌入是否包含情感？实验与文献结论相悖的原因探究

Great question—this mismatch between your findings and the published literature is a really interesting observation, and there are several key factors that could explain why your Amazon review-trained GloVe embeddings aren’t picking up on sentiment-polar opposites like the studies you cited. Let’s break this down:

1. Dataset-Specific Co-Occurrence Patterns #

GloVe learns embeddings based on word co-occurrence statistics from your training data, so the first thing to check is the nature of your Amazon review corpus:

• Sentiment distribution bias: If your dataset is heavily skewed toward positive reviews (which is common for Amazon products), negative terms might be rare, or they might rarely co-occur with positive terms in contexts that signal opposition. For example, reviewers rarely write "This product is beautiful but sad"—instead, they stick to consistent sentiment in a single review. Without those cross-polar co-occurrences, GloVe has no signal to link "happy" and "sad" as opposites.
• Domain-specific language: Amazon reviews focus on product attributes, so emotional terms are often used in narrow, consistent contexts. Words like "beautiful" will almost always appear alongside other positive adjectives describing the product, reinforcing their shared positive embedding space rather than linking to negative counterparts.

2. Training Parameter Choices #

Your GloVe training setup might be influencing the embeddings’ ability to capture sentiment polarity:

• Window size: If you used a small window (e.g., 2-3 words), the model only captures local co-occurrences. In a positive review, "beautiful" will only be next to other positive words, so its embedding clusters with them. A larger window might capture broader contextual relationships, though this depends on the dataset.
• Vector dimension: Lower-dimensional embeddings (e.g., <100) might not have enough capacity to distinguish between subtle semantic relationships like sentiment polarity. The studies you cited likely used standard dimensions (100-300) which offer more granularity.
• Training convergence: If you stopped training too early, the embeddings might not have fully learned the underlying co-occurrence patterns. Make sure you’ve run enough iterations until the loss stabilizes.

3. Differences in Experimental Setup from the Literature #

The studies you referenced used very different data and methods, which explains the divergent results:

• Yu et al. (2017): They refined pre-trained word embeddings using sentiment intensity scores, starting with a general-purpose embedding that already had some inherent sentiment information. Your embeddings are trained from scratch on a domain-specific dataset, so they don’t inherit that general semantic polarity.
• Tang et al. (2014): They used Twitter data, which is full of short, conversational text where users often express contrasting emotions or use polar terms in close proximity (e.g., "I’m happy today but sad about tomorrow"). Twitter’s diverse, unstructured context provides far more signal for learning opposite sentiment pairs than Amazon’s product-focused reviews.

4. How You’re Evaluating Similarity #

Double-check how you’re retrieving similar words:

• Are you only testing positive terms (like "beautiful")? Try looking up similar words for negative terms (e.g., "terrible", "disappointing")—do you see clusters of negative words, or still positive ones? If it’s the former, your embeddings are capturing sentiment clusters, just not cross-polar links (which aligns with the dataset’s co-occurrence patterns).
• The literature mentions ~30% of top 10 similar words are polar opposites across all vocabulary terms. If you’re only sampling a subset of positive terms, you won’t see that ratio.

Next Steps to Explore #

• Audit your dataset’s sentiment distribution: Use a simple sentiment classifier to count positive/negative/neutral reviews, and check how often polar opposite terms co-occur.
• Adjust GloVe parameters: Try increasing the window size (to 5-10) and vector dimension (to 200-300), then re-train.
• Compare with pre-trained embeddings: Load a general-purpose GloVe embedding (e.g., trained on Wikipedia) and check if it shows the polar opposite pairs the literature describes—this will confirm whether the issue is dataset-specific.
• Try sentiment-refinement techniques: Follow the approaches in Yu et al. or Tang et al. to adjust your domain embeddings using sentiment annotations, which can explicitly inject polarity signals.

内容的提问来源于stack exchange，提问作者Ivy