基于带情感标签词词典的情感分析训练准确率极低求助

Hey there, let's figure out why your sentiment classification model is struggling with low accuracy and walk through actionable fixes that should help!

Your dataset is made up of individual words/phrases directly mapped to sentiment labels—TF-IDF is designed for measuring term importance across longer documents, not for single-term samples. Here's why it's failing:

• For single words, TF (term frequency) is always 1, so TF-IDF values won't have meaningful variation between positive/negative terms.
• Phrases like "too good" get split into separate tokens, breaking the connection between the phrase and its label.

Quick Fixes for TF-IDF: #

• Enable lowercase=True (you currently have it set to False) to avoid treating "Bad" and "bad" as distinct terms—this unifies identical words with the same sentiment.
• Add ngram_range=(1,2) to capture single words and two-word phrases, so "too good" is treated as a single feature instead of two separate tokens.
• Double-check your stop_words list: if it includes sentiment-relevant terms (like "too", which amplifies positive/negative meaning), remove them to preserve critical context.

Low accuracy often starts with messy data. Run these checks first:

• Check label distribution: Use test_df['label'].value_counts() to see if one sentiment class dominates (e.g., 90% negative labels). If so, the model will just guess the majority class instead of learning meaningful patterns. Fix this with oversampling minority classes or undersampling majority ones.
• Spot-check annotations: Randomly pull 100-200 samples to verify labels (e.g., make sure "amazing" isn't tagged as -1). Even a small number of mislabeled samples can derail training.
• Normalize labels: Your labels range from -2 to 3—if you're using a classification model, ensure it's set up to handle multi-class labels, or consider binarizing them (e.g., negative = -2/-1, positive = 2/3) if you only need binary sentiment.

You didn't mention which model you're using, but linear models (like logistic regression) can struggle with sparse TF-IDF features for single-term data. Try these alternatives:

• Multinomial Naive Bayes: This model is optimized for text classification with sparse features (like TF-IDF) and tends to perform well on sentiment tasks with word-level data.
• Word Embeddings + Neural Network: For better semantic understanding, use pre-trained embeddings (GloVe, Word2Vec) or train a custom embedding layer on your dataset. This captures the emotional context of words that TF-IDF misses.

Don't forget these critical steps that are easy to overlook:

• Split your data: Always split into training and test sets (use train_test_split)—training on the full dataset and evaluating on the same data gives you inflated, inaccurate accuracy scores.
• Use cross-validation: Run cross_val_score to ensure your model's performance isn't just luck from a single train/test split.
• Tune hyperparameters: Use GridSearchCV or RandomizedSearchCV to optimize your vectorizer and model parameters (e.g., adjust use_idf in TF-IDF, or alpha in Naive Bayes).

Here's a revised version of your code incorporating these fixes:

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load your dataset (assuming comma-separated txt file)
df = pd.read_csv('your_vocab_file.txt', names=['word', 'label'])

# Split into train/test sets to avoid overfitting
X_train, X_test, y_train, y_test = train_test_split(
    df['word'], df['label'], test_size=0.2, random_state=42
)

# Configure TF-IDF to capture single words and phrases
vectorizer = TfidfVectorizer(
    use_idf=True,
    lowercase=True,
    ngram_range=(1, 2),
    stop_words=None  # Remove stop_words unless you're certain they don't affect sentiment
)

# Transform text data
X_train_vec = vectorizer.fit_transform(X_train)
X_test_vec = vectorizer.transform(X_test)

# Train a Naive Bayes model
model = MultinomialNB()
model.fit(X_train_vec, y_train)

# Evaluate performance
y_pred = model.predict(X_test_vec)
print(f"Test Accuracy: {accuracy_score(y_test, y_pred):.2f}")

内容的提问来源于stack exchange，提问作者Devansh Singh